Metadata-Version: 2.4
Name: scalio-selftest
Version: 0.2.0
Summary: Test mode of the Scalio apps: nightly self-check (API suite + browser smoke) on test instances
License: Proprietary
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: httpx>=0.27
Provides-Extra: fastapi
Requires-Dist: fastapi>=0.110; extra == "fastapi"
Provides-Extra: ui
Requires-Dist: playwright==1.64.0; extra == "ui"
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"
Requires-Dist: fastapi>=0.110; extra == "dev"
Requires-Dist: mongomock-motor>=0.0.29; extra == "dev"
Requires-Dist: sqlalchemy>=2; extra == "dev"

# scalio-selftest

Test mode of the Scalio apps (scalio-docs ADR 0053, `concepts/test-mode.md`): on our test instances (never dev, demo or production) the app
checks itself every night from the outside (API suite + browser smoke), keeps the history, compares each run with
the one before, reports to Scalio Monitor and shows everything on the page «Tests» (`@scalio-cloud/ui`).
Modelled on the Nexus self-check (`backend/e2e_selfcheck.py`, `backend/test.py`, `backend/ui_smoke.py`).

```python
import scalio_selftest as sst
from scalio_selftest import common

async def basics(ctx):
    await common.basics(ctx)
    await common.sign_in(ctx)
    await common.setup_closed(ctx)
    await common.rights(ctx, [common.Rule("GET", "/api/users", {"admin"})])

SELFTEST = sst.SelfTest(
    "citadel", db, [sst.Suite("Basis", basics)],
    accounts=[sst.account("citadel", "admin"), sst.account("citadel", "operator")],
    upsert_account=upsert, remove_accounts=remove_all,          # the app stores its users itself
    ui=sst.UiSmoke(pages=[sst.Page("/", "Übersicht"), sst.Page("/settings", "Einstellungen")]),
)

app.include_router(make_router(SELFTEST, require_admin))       # scalio_selftest.router, needs [fastapi]
await SELFTEST.startup()                                         # API and worker start: accounts on / off
JOBS += [("citadel_selftest", 30, SELFTEST.tick)]                # worker, under the job lease
```

## Switches

| Env | Meaning |
| --- | --- |
| `TEST_MODE` | `SAFE` (only the app's own data and simulated Microsoft layers) or `FULL` (also the Entra test tenant; only Markus sets it). Anything else: off. |
| `SCALIO_STAGE` | must be `test`, otherwise the test mode stays off even with `TEST_MODE` |
| `TEST_E2E_PUBLIC_URL` | address the runner calls (default `PUBLIC_URL`) |
| `TEST_E2E_RUN_HOUR` | nightly hour 0-23; default 2 |
| `TEST_E2E_TZ` | time zone of the hour, default `Europe/Zurich` |
| `E2E_BROWSER_WS` | Playwright server of the host (container `e2e-browser`), e.g. `ws://e2e-browser:3000/` |
| `E2E_MONITOR_KEY`, `E2E_MONITOR_TOKEN_FILE` | heartbeat to Scalio Monitor after every run |
| `ERROR_REPORT_URL`, `ERROR_REPORT_TOKEN` | failed checks go to Scalio Monitor as errors of kind `e2e` (one GitHub issue per new group, label `e2e`, cockpit tab «Fehler»). There are no mails. |
| `E2E_TRIGGER_TOKENS` | sha256 of the tokens the RC deploy may use for `POST /api/e2e/trigger` |

## Pieces

- `gate`: the two switches above. The UI can lower FULL to SAFE, never raise it.
- `accounts`: one hidden test account per role (`e2e-<role>@<app>.e2e.scaliocloud.com`, no MX), fresh password on every start, kept
  only in the app's database, removed when the test mode is off. `totp()` for test accounts with MFA.
- `store`: `MongoStore(db)` (collections `e2e_state`, `e2e_runs`) or `SqlStore(engine)` (table `e2e_docs`,
  `SqlStore.DDL`, for Kontor). `SelfTest` takes a Motor database or a store.
- `runner`: `SelfTest` (history of 30 runs, progress, request, config), `Context` with
  `check()`, `client(role)`, `defer()` for clean-up, objects named with `ctx.prefix` (`e2e-<run>`).
- `common`: checks every app runs: health, version, headers, sign-in per role, wrong password, rights matrix,
  first-run setup closed, reads without errors.
- `ui`: browser smoke over `E2E_BROWSER_WS` in German (light) and English (dark): sign-in, pages render, no error
  page, no sharp s, console errors as WARN.
- `router`: `/api/e2e/status` (public, verdict only), `health`, `runs`, `progress`, `config`, `run`, `cancel`
  (admin), `trigger` (deploy token).
- `monitor`: heartbeat `POST /ingest/heartbeat/<key>?status=ok|fail`.

Result format (same as Nexus): `{section, name, mode, status, detail}` with `PASS`, `FAIL`, `WARN`, `SKIP`.
