Metadata-Version: 2.4
Name: leaf-portal
Version: 1.0.16
Summary: LEAF Portal
License-File: LICENSE
Requires-Python: >=3.12,<4.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Dist: asyncpg (>=0.31.0)
Requires-Dist: bcrypt (>=5.0.0)
Requires-Dist: ipykernel (>=7.2.0)
Requires-Dist: nicegui[testing] (>=3.12.0)
Requires-Dist: pandas (>=3.0.3)
Requires-Dist: plotly (>=6.7.0)
Requires-Dist: pyotp (>=2.9.0)
Requires-Dist: python-dotenv (>=1.2.2)
Requires-Dist: qrcode (>=7.4.2)
Description-Content-Type: text/markdown

# leaf-portal

Web portal for the LEAF framework. Built with [NiceGUI](https://nicegui.io) and [asyncpg](https://github.com/MagicStack/asyncpg), backed by a TimescaleDB/PostgreSQL database.

## Features

- **Organisation & department management** — hierarchical grouping of entities
- **User management** — superadmin, org admin, and regular users with bcrypt-hashed passwords; admin impersonation
- **Access management** — time-windowed grants per department/entity
- **Entity management** — hide entities from regular users and the sensor catalog
- **Sensor data explorer** — browse and filter readings by entity, metric, and time range (multi-select)
- **Interactive plots** — Plotly-based time-series visualization
- **Alarm rules** — threshold-based alerts (configurable per-rule check interval) with email notifications on trigger and auto-resolve
- **API token management** — generate and revoke tokens for REST API access
- **REST API** — token-authenticated endpoints for sensor data retrieval (Swagger UI at `/api/docs`)
- **First-run setup wizard** — browser-based DB connection and superadmin creation at `/setup`
- **Password reset** — email-based reset flow (`/forgot-password`, `/reset-password`)

## Requirements

- Python 3.12+
- PostgreSQL 16+ or TimescaleDB
- SMTP server (optional — required for alarm emails and password reset)

## Installation

From PyPI:

```bash
pip install leaf-portal
```

From source:

```bash
poetry install
```

## Configuration

Create a `.env` file in the working directory. All variables are optional at startup — the setup wizard at `/setup` will prompt for DB credentials on first run and persist them to `.env`.

```env
# Database (defaults shown)
PGHOST=timescaledb
PGPORT=5432
PGDATABASE=leaf
PGUSER=postgres
PGPASSWORD=

# Mail (required for alarm emails and password reset)
SMTP_HOST=smtp.example.com
SMTP_PORT=587
SMTP_USER=user@example.com
SMTP_PASSWORD=secret
SMTP_FROM=noreply@example.com

# Portal public URL (used in password-reset emails)
PORTAL_URL=http://localhost:8081

# NiceGUI session secret — change in production
STORAGE_SECRET=change-me-in-production
```

## Running

```bash
leaf-portal
# or
python -m leaf_portal
```

The portal listens on `0.0.0.0:8081` by default.

On first run, navigate to `http://localhost:8081` — you will be redirected to the setup wizard to configure the database connection and create the initial superadmin account.

## REST API

All endpoints require a token passed via the `Authorization: Bearer <token>` header.

Tokens are generated from the **API Tokens** page (`/tokens`).

| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/managements` | List managements accessible to the token |
| `GET` | `/api/data/recent` | Most recent readings across all accessible departments (`?limit=20`) |
| `GET` | `/api/data` | Filtered sensor data — requires `organisation` + `department`; optional: `entity`, `metric` (comma-separated for multiple), `from`, `to` (ISO 8601), `limit` (max 10 000) |

Interactive docs: `/api/docs`

## Development

```bash
poetry install --with dev

# Run tests
poetry run pytest tests/ -v

# Lint and format
poetry run ruff check leaf_portal/ tests/
poetry run ruff format leaf_portal/ tests/
```

## Database

The schema lives at `deploy/deploy.sql` (targets TimescaleDB) and is applied automatically by the setup wizard or on startup when the DB is reachable.

For CI and Docker-based local development, `docker/init_db.py` waits for Postgres to be ready and then applies `deploy/deploy.sql`.

## Backups

A full `pg_dump` of the database doesn't make sense once `sensor_data` grows
large — it would re-export the entire sensor history every day. Instead,
`docker/backup` builds a small standalone image that splits the backup in two:

1. **Operational tables** (`organisation`, `department`, `user_account`,
   `management`, `alarm_*`, `mapper_*`, `api_token`, ...) — these are tiny, so
   they're fully `pg_dump`'d (custom format) every day to `leaf_YYYY-MM-DD.dump`.
   `sensor_data` and TimescaleDB's internal chunk/catalog tables are excluded.
2. **`sensor_data`** — exported per UTC day via `\copy` to
   `sensor_data_YYYY-MM-DD.csv.gz`. The last `ROLLING_DAYS` days are
   re-exported (overwritten) on every run to catch late-arriving readings;
   older files are write-once.

The image is read-only against the database (`leaf_backup_user`, member of
the `backup_readers` role created by `deploy/deploy.sql`) and runs once per
invocation — schedule it with a Kubernetes CronJob (see
`docker/backup/cronjob.example.yaml`) or any host cron running `docker run`.

```bash
docker build -t leaf-backup docker/backup

docker run --rm \
  -e PGHOST=... -e PGUSER=leaf_backup_user -e PGPASSWORD=... -e PGDATABASE=leaf \
  -e ROLLING_DAYS=3 -e KEEP_DUMPS=14 \
  -v /path/to/backups:/backups \
  leaf-backup
```

Or, using `deploy/deploy.py` (builds/pushes via `build-backup`, runs once via
`backup`):

```bash
python3 deploy/deploy.py build-backup   # build & push to the registry

export PGHOST=... PGUSER=leaf_backup_user PGPASSWORD=... PGDATABASE=leaf
export BACKUP_DIR=/path/to/backups      # default: ./backups
python3 deploy/deploy.py backup
```

Restore:

```bash
# 1. Recreate the schema (also recreates the sensor_data hypertable)
python3 deploy/deploy.py schema

# 2. Restore operational tables
pg_restore --data-only --disable-triggers -d <db> leaf_YYYY-MM-DD.dump

# 3. Re-import sensor_data for each day
zcat sensor_data_YYYY-MM-DD.csv.gz | psql -d <db> -c "\copy sensor_data FROM STDIN WITH (FORMAT csv, HEADER true)"
```

The backup image pins its `pg_dump`/`pg_restore` version to the TimescaleDB
version this project targets (`timescale/timescaledb:2.17.2-pg16`) — custom-format
dumps aren't readable by an older `pg_restore`. To back up a different
Postgres major version, change the `FROM` tag in `docker/backup/Dockerfile`
and rebuild. Before dumping anything, `backup.sh` checks that the server's
major version matches its bundled `pg_dump` and exits with an error
(without writing any files) if they've drifted apart.

## Deployment

`deploy/deploy.py` is a helper script for building and running the portal in production. It requires no extra dependencies beyond Docker (and `psql` for remote schema application).

```
python3 deploy/deploy.py <command>
```

| Command | What it does |
|---------|--------------|
| `build` | Builds a multi-arch (`amd64`/`arm64`) Docker image, tags it with the current git tag or short commit hash, and pushes it to `docker-registry.wur.nl/leaf/docker/leaf-portal`. |
| `schema` | Applies `deploy/deploy.sql` to the target database. Locally it runs `psql` inside the `timescaledb` container; against a remote host it calls `psql` directly. |
| `run` | Pulls the portal image from the registry and starts it as a container named `leaf-portal` on port 8081. |
| `stop` | Stops and removes the `leaf-portal` container. |

`schema` requires passwords for the PostgreSQL service accounts it creates — never use defaults:

```bash
export LEAF_PORTAL_PASSWORD=...   # leaf_portal_user  (portal, read+write)
export LEAF_GRAFANA_PASSWORD=...  # leaf_grafana_user (Grafana, read-only)
export LEAF_API_PASSWORD=...      # leaf_api_user     (external API access)
export LEAF_BACKUP_PASSWORD=...   # leaf_backup_user  (backup job, read-only)
```

Typical production flow:

```bash
# 1. Apply the schema (once, or after schema changes)
python3 deploy/deploy.py schema

# 2. Start the portal
python3 deploy/deploy.py run
```

Run `build` only when cutting a new release.

## Page routes

| Route | Description |
|-------|-------------|
| `/` | Redirects to `/dashboard` or `/login` |
| `/login` | Login page |
| `/setup` | First-run setup wizard |
| `/forgot-password` | Password reset request |
| `/reset-password` | Password reset with token |
| `/dashboard` | Overview dashboard |
| `/admin/organisations` | Organisation management |
| `/admin/departments` | Department management |
| `/admin/users` | User management |
| `/admin/access-management` | Access grant management |
| `/admin/mapper` | Entity/metric mapping |
| `/admin/settings` | Application settings |
| `/dept/members` | Department member management |
| `/entities` | Entity management (hide/show from regular users) |
| `/categories` | Category management |
| `/data/explorer` | Sensor data explorer |
| `/data/plots` | Time-series plots |
| `/alarms` | Alarm rules and event history |
| `/tokens` | API token management |
| `/profile` | User profile |
| `/api/docs` | Swagger UI for the REST API |

