Metadata-Version: 2.4
Name: ts43-test-ecs
Version: 1.3.0
Summary: GSMA TS.43 v13.0 Test Entitlement Configuration Server (ECS)
License: Apache-2.0
Requires-Python: >=3.11
Requires-Dist: aiosqlite<1.0,>=0.20
Requires-Dist: alembic<2.0,>=1.14
Requires-Dist: cachetools<7.0,>=5.5
Requires-Dist: cryptography>=44.0
Requires-Dist: fastapi<1.0,>=0.115
Requires-Dist: httpx<1.0,>=0.28
Requires-Dist: lxml<7.0,>=5.0
Requires-Dist: pycryptodomex<4.0,>=3.21
Requires-Dist: pydantic-settings<3.0,>=2.0
Requires-Dist: pydantic<3.0,>=2.7
Requires-Dist: pyjwt<3.0,>=2.8
Requires-Dist: pyyaml<7.0,>=6.0
Requires-Dist: sqlalchemy[asyncio]<3.0,>=2.0.38
Requires-Dist: uvicorn[standard]<1.0,>=0.34
Provides-Extra: console-test
Requires-Dist: playwright>=1.49; extra == 'console-test'
Provides-Extra: dev
Requires-Dist: bandit>=1.8; extra == 'dev'
Requires-Dist: hypothesis>=6.100; extra == 'dev'
Requires-Dist: lxml-stubs>=0.5; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pip-audit>=2.7; extra == 'dev'
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
Requires-Dist: pytest-cov>=4.1; extra == 'dev'
Requires-Dist: pytest>=8.2; extra == 'dev'
Requires-Dist: ruff>=0.11; extra == 'dev'
Requires-Dist: types-cachetools>=5.5; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mike>=2.2; extra == 'docs'
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Provides-Extra: fcm
Requires-Dist: firebase-admin<7.0,>=6.0; extra == 'fcm'
Provides-Extra: fuzz
Requires-Dist: atheris>=2.3; extra == 'fuzz'
Provides-Extra: lab
Requires-Dist: mobly>=1.12; extra == 'lab'
Description-Content-Type: text/markdown

# TS.43 Test ECS

[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/RamboXWang/ts43-test-ecs/badge)](https://scorecard.dev/viewer/?uri=github.com/RamboXWang/ts43-test-ecs)

Open-source **GSMA TS.43 v13.0 Entitlement Configuration Server (ECS)** for testing
client devices, Android first. A single-process Python server — FastAPI + SQLite +
pure-Python EAP-AKA/AKA′ — with no Docker, no FreeRADIUS, and no Keycloak. Point a
phone (or the bundled protocol-faithful client) at it and exercise real entitlement
flows end-to-end: VoLTE, VoWiFi, eSIM ODSA, satellite, DataBoost, carrier billing,
phone-number verification.

**Status: v1.3.0** — published to PyPI as `ts43-test-ecs`. 914 tests,
TS.43 §-by-§ traceability generated from test markers
(see [`docs/compliance_matrix.md`](docs/compliance_matrix.md)), plus an
AOSP-differential conformance harness with a deterministic replay corpus.

## Contents

- [Why this exists](#why-this-exists)
- [Supported AppIDs](#supported-appids)
- [Quick start](#quick-start)
- [Making your first entitlement request](#making-your-first-entitlement-request)
- [Authentication methods](#authentication-methods)
- [HTTP surface](#http-surface)
- [Test-control (admin) API](#test-control-admin-api)
- [Scenario presets](#scenario-presets)
- [Configuration](#configuration)
- [Testing with a real Android device](#testing-with-a-real-android-device)
- [Architecture](#architecture)
- [Development](#development)
- [Protocol fidelity notes](#protocol-fidelity-notes)
- [Key documentation](#key-documentation)
- [License](#license)

## Why this exists

GSMA TS.43 is the protocol Android and iOS use to ask a carrier "is this SIM
entitled to VoWiFi / eSIM transfer / satellite messaging?". Commercial ECS
deployments are closed, and there was no public server implementation to test
clients against. This project fills that gap for lab use:

- **Real authentication** — full EAP-AKA/AKA′ over the TS.43 HTTP relay, driven by
  a pure-Python Milenage implementation. Works with programmable SIMs
  (known Ki/OPc) and handles real-SIM SQN resynchronization (AUTS).
- **Real documents** — byte-stable OMA WAP provisioning XML and JSON responses,
  golden-file tested against what the AOSP client actually parses.
- **A test-control plane** — seed subscribers, flip entitlement parameters, inject
  faults, apply scenario presets, inspect every request the device made.

## Supported AppIDs

All 13 AppIDs known to the AOSP `service_entitlement` library, plus the
spec-only ap2015:

| AppID | Service | TS.43 § |
|---|---|---|
| ap2003 | VoLTE / VoCel entitlement | 3 |
| ap2004 | VoWiFi entitlement (T&C / address flows) | 4 |
| ap2005 | SMSoIP entitlement | 5 |
| ap2006 | ODSA companion device (eSIM) | 6–8 |
| ap2009 | ODSA primary device (eSIM) | 6–8 |
| ap2010 | Data plan information | 9 |
| ap2011 | ODSA server-initiated requests | 10 |
| ap2012 | DataBoost / premium network capability | 11 |
| ap2013 | Direct carrier billing | 12 |
| ap2014 | App-Server authentication / phone-number verification | 13 |
| ap2015 | Server-to-server operator tokens | 14 |
| ap2016 | Satellite entitlement + SatMode Config (v13) | 15 |
| ap2017 | ODSA cross-platform transfer | 6–8 |

Unknown AppIDs fall back to `EntitlementStatus=0` rather than erroring, matching
commercial server behavior.

## Quick start

The CI-tested, pip-install-based walkthrough lives in the
[Quickstart Guide](docs/quickstart.md) — from `pip install ts43-test-ecs` to a
first simulated-client entitlement fetch.

To run from a source checkout instead (requires Python ≥ 3.11):

```bash
git clone https://github.com/RamboXWang/ts43-test-ecs.git
cd ts43-test-ecs
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
python scripts/generate_certs.py     # local CA + ECDSA server cert + JWT keypair
python -m ts43ecs                        # serves https://127.0.0.1:8443
curl -sk https://127.0.0.1:8443/health
```

`python scripts/generate_certs.py` writes a local CA (`certs/ca.crt`), a server
certificate with lab-friendly SANs, and the ES256 keypair used to sign auth
tokens. The server binds loopback by default and *refuses* to bind non-loopback
addresses with missing or example credentials.

## Making your first entitlement request

Seed a subscriber (3GPP test-SIM credentials), then run the full EAP-AKA
handshake with the bundled `EcsClient` — it implements the device side of the
protocol exactly:

```bash
export ECS_ADMIN_API_KEY=my-lab-key    # set before starting the server

curl -sk -X POST https://127.0.0.1:8443/admin/api/subscribers \
  -H "X-Admin-Key: my-lab-key" -H "Content-Type: application/json" \
  -d '{"imsi":"001010000000001",
       "ki":"000102030405060708090a0b0c0d0e0f",
       "opc":"00000000000000000000000000000000",
       "msisdn":"+15551234567"}'
```

```python
import asyncio, httpx
from ts43ecs.tools.ecs_client.client import EcsClient

async def main():
    async with httpx.AsyncClient(base_url="https://127.0.0.1:8443",
                                 verify="certs/ca.crt") as http:
        resp = await EcsClient(client=http).authenticate(
            imsi="001010000000001",
            ki="000102030405060708090a0b0c0d0e0f",
            opc="00000000000000000000000000000000",
            app="ap2004",
        )
        print(resp.document)          # VERS + TOKEN + ap2004 APPLICATION
        print(resp.document.token)    # fast re-auth token for follow-ups

asyncio.run(main())
```

The document carries the ap2004 characteristic (`EntitlementStatus`,
`TC_Status`, `AddrStatus`, …) plus a fast re-auth token. Subsequent requests
skip EAP entirely — the CLI covers that path:

```bash
python -m ts43ecs.cli client \
  --url https://127.0.0.1:8443 --ca certs/ca.crt \
  --imsi 001010000000001 --app ap2004 --token "<token from above>"
```

CLI flags: `--app` (repeatable), `--json` (JSON instead of XML), `--post`,
`--insecure`, and `--ki`/`--opc` to run the full EAP-AKA handshake from the
command line instead of a token. With neither a token nor Ki/OPc the CLI sends
an unauthenticated request — useful for observing the server's EAP challenge /
511 behavior.

## Authentication methods

The server implements every TS.43 §2.8 authentication path:

| Method | § | Notes |
|---|---|---|
| EAP-AKA / EAP-AKA′ | 2.8.1 | HTTP-relay (JSON body over HTTP 200), Milenage, max 3 attempts, AUTS resync |
| Fast re-auth token | 2.8.5 | JWT (ES256) issued in the TOKEN characteristic; passed as a *query parameter*; bound to the IMSI (mismatch → 511) |
| Temporary tokens | 2.8.6 / 2.8.7 | ODSA websheet flows; renewal is once-only |
| Web-view auth | 2.8.2 | Auto-login, login form, and SMS-OTP (codes readable from the admin OTP outbox) |
| OAuth 2.0 / OIDC | 2.8.3 | Embedded provider under `/oidc/*` — no external IdP |
| S2S operator tokens | 14 | ap2015: `/oauth/token` + `/oauth/introspect`; MDM/App-Server requests present a Bearer token |
| Debug bypass | — | `ECS_ALLOW_AUTH_BYPASS` for SIM-less emulators; disabled by default, refuses example secrets |

## HTTP surface

| Path | Purpose |
|---|---|
| `GET/POST /` | The TS.43 entitlement endpoint — everything a device sends goes here (params as query string, per spec) |
| `GET /health`, `GET /ready` | Liveness / readiness |
| `/oidc/*` | Embedded OIDC provider (authorize, login, otp, token, jwks, callback) |
| `/oauth/token`, `/oauth/introspect` | ap2015 server-to-server tokens |
| `/websheet/odsa`, `/websheet/databoost` (+ `/complete`) | User-facing websheets for ODSA subscription and DataBoost purchase flows |
| `/admin/api/*` | Test-control API (below) |

## Test-control (admin) API

Guarded by `X-Admin-Key` (constant-time compare) plus a source-IP allowlist and
per-IP rate limiting. Highlights:

```
POST   /admin/api/subscribers                 create (also /bulk)
GET    /admin/api/subscribers/{imsi}          DELETE …
POST   /admin/api/subscribers/{imsi}/sqn/advance
PUT    /admin/api/entitlements/{imsi}/{app}   override document parameters
POST   /admin/api/profiles                    eSIM profiles (also /bulk)
GET    /admin/api/odsa/state/{imsi}           PUT /admin/api/odsa/{imsi}/{app}
POST   /admin/api/odsa/{imsi}/{app}/download-ready
POST   /admin/api/odsa/server-initiated
GET    /admin/api/tokens                      POST …/{id}/revoke, POST …/expire
GET    /admin/api/otp-outbox                  SMS-OTP codes for web-auth tests
GET    /admin/api/notifications               POST …/push (§2.6), GET /notif-outbox
GET    /admin/api/traffic                     request inspector (+ /traffic/html)
POST   /admin/api/chaos                       fault injection rules (GET/DELETE)
GET    /admin/api/scenarios                   POST …/{name}/apply
POST   /admin/api/snapshots/{name}            state snapshots (+ /restore)
POST   /admin/api/reset                       global test isolation
GET    /admin/api/sessions                    live EAP session view
```

Chaos injection (delays, error codes, malformed bodies, matched by ts43ecs/imsi/path)
is additionally gated behind `ECS_ALLOW_CHAOS_INJECT=true` and delay-capped.

## Scenario presets

`carriers.yaml` ships ready-made situations, each mapped to a GSMA TS.56 test
case — apply one and the subscribers, overrides, and faults it needs are set up
in one call:

```bash
curl -sk -X POST -H "X-Admin-Key: my-lab-key" \
  https://127.0.0.1:8443/admin/api/scenarios/vowifi_tc_pending/apply
```

Included presets: `vowifi_tc_pending`, `esim_transfer_delayed`,
`satellite_disaster_only`, `token_expired_511`. Add your own by extending the
YAML (`description`, `ts56`, `subscribers`, `overrides`, `settings`,
`expire_tokens`, `chaos`).

## Configuration

Everything is an environment variable with the `ECS_` prefix (or `.env`; see
`.env.example` for the full list). The important ones:

| Variable | Default | Purpose |
|---|---|---|
| `ECS_HOST` / `ECS_PORT` | `127.0.0.1` / `8443` | Bind address (non-loopback requires strong credentials) |
| `ECS_SSL_CERTFILE` / `ECS_SSL_KEYFILE` | `certs/server.crt` / `.key` | TLS material (use a public cert for real devices — see playbook) |
| `ECS_DATABASE_URL` | `sqlite+aiosqlite:///./ecs.db` | Durable state |
| `ECS_ADMIN_API_KEY` | *(unset — admin API disabled)* | Test-control API key |
| `ECS_ADMIN_ALLOWED_IPS` | `127.0.0.1,::1` | Admin source-IP allowlist (empty = allow any) |
| `ECS_ADMIN_RATE_LIMIT_PER_MINUTE` | `600` | Per-IP sliding window; `0` disables |
| `ECS_TOKEN_TTL_SECONDS` | `86400` | Fast-auth token lifetime |
| `ECS_ALLOW_AUTH_BYPASS` / `ECS_ALLOW_CHAOS_INJECT` | `false` | Test-only escape hatches |
| `ECS_SHUTDOWN_GRACE_SECONDS` | `10` | Graceful shutdown drain (0–120) |

Secrets are masked in logs and `Settings.__repr__`.

## Testing with a real Android device

See **[`docs/android_testing_playbook.md`](docs/android_testing_playbook.md)** for
the full guide. The short version:

1. **TLS trust** is the main hurdle — AOSP entitlement clients trust the *system*
   store only. Options: emulator with writable system, Android 14+
   Conscrypt-APEX bind-mount (rooted), or a public certificate (zero device mods).
2. **Point the device at the server**:
   `adb shell cmd phone cc set-value -p imsserviceentitlement.entitlement_server_url_string https://<host>:8443`
   (the same key drives IMS *and* satellite entitlement).
3. **SIM credentials**: programmable SIMs (sysmoISIM) with known Ki/OPc, seeded
   via the admin API. First auth resyncs SQN automatically — that's expected.
4. Watch everything the device does in the traffic inspector
   (`/admin/api/traffic/html`).

## Architecture

```
Device / ecs_client
   │ HTTPS (GET /, query params)
   ▼
transport/router.py ──► EAP relay? ──► eap/ (packet codec → FSM → Milenage)
   │ normalize (case-insensitive params, UA, vers)
   │ chaos (opt-in)
   │ authenticate (token | temp token | S2S bearer | EAP | OIDC | bypass)
   ▼
apps/registry.py ──► per-AppID handler ──► odsa/dispatcher.py (ODSA ops)
   │                       │
   │                 HandlerContext (subscriber, overrides, request)
   ▼
documents/ (DocumentModel → WAP XML or JSON, VERS+TOKEN+APPLICATION)
```

- **Storage split**: SQLite (SQLAlchemy async + Alembic) for durable state —
  subscribers, tokens, overrides, notification registrations; an in-process
  `MemoryStore` (TTL caches, lock-wrapped) for EAP sessions, OTP/notification
  outboxes, traffic log, and chaos rules.
- **Single process, embedded everything**: EAP engine, OIDC provider, websheets,
  and admin plane all live in one FastAPI app (`ts43ecs/main.py:create_app`).
- **`ts43ecs/tools/ecs_client`** implements the *client* side of the protocol and is what
  the integration and performance tests drive — CI needs no hardware.

## Development

```bash
# Gates (all enforced in CI, py3.11–3.13)
ruff check .
mypy app tools                                 # strict
pytest -q -m "not perf" --cov=app --cov-fail-under=90
bandit -q -c pyproject.toml -r ts43ecs/ ts43ecs/tools/ scripts/
pip-audit --skip-editable

# Perf smoke (advisory: 50 concurrent EAP handshakes, ≥100 fast-auth/s)
pytest -q -m perf

# Single test
pytest tests/unit/test_milenage.py -v
pytest -k "test_volte" -v

# Migrations (run from repo root; CI fails on model drift)
alembic upgrade head && alembic check

# Regenerate the matrices after touching @pytest.mark.spec markers / carriers.yaml
python -m ts43ecs.tools.gen_compliance_matrix
python -m ts43ecs.tools.gen_ts56_matrix
```

Compliance traceability is enforced, not aspirational: tests carry
`@pytest.mark.spec("<TS.43 §>")` markers, `ts43ecs/tools/gen_compliance_matrix.py`
generates the matrix, and a unit test fails the build if any of the 27 required
sections loses coverage.

## Protocol fidelity notes

Where the spec and the real AOSP client disagree, this server follows the
client (verified against `frameworks/libs/service_entitlement`):

- EAP relay rides on **HTTP 200** (not 401), Content-Type
  `application/vnd.gsma.eap-relay.v1.0+json`
- XML Content-Type is `text/vnd.wap.connectivity-xml`
- GET is the default method; the auth token and `vers` are **query parameters**
- The client only accepts HTTP 200 and 302 — anything else throws on-device
- Parameter names are matched case-insensitively (AOSP lowercases them)
- Known AOSP quirks are reproduced, including the `NetworkVoiceIRATCapablity`
  parameter-name typo

## Key documentation

| File | Purpose |
|---|---|
| `docs/compliance_matrix.md` | Generated TS.43 v13 § → test traceability |
| `docs/ts56_matrix.md` | Generated TS.56 test case → scenario preset mapping |
| `docs/android_testing_playbook.md` | Real-device testing guide |
| `docs/console.md` | Browser test console at `/console` |
| `docs/operations.md` | Schema migrations, JWT key rotation, TLS floor |
| `docs/conformance.md` | AOSP differential testing and the replay corpus |
| `docs/AOSP_client_protocol_spec.md` | Android client protocol, extracted from AOSP source |
| `docs/TS43_XML_schema_reference.md` | Per-AppID XML formats with exact parameter names |
| `docs/eap_aka_packet_format_reference.md` | Byte-level EAP-AKA/AKA′ packet encoding |
| `docs/SECURITY_REVIEW.md` | Security posture and review notes |
| `docs/v1/ts43-ecs-impl-plan-fable5.md` | v1.0 implementation plan (historical) |
| `docs/v1.2/v1.2_improvement_plan.md` | v1.2 improvement plan (current scope source of truth) |
| `CHANGELOG.md` | Release history (Keep a Changelog) |

## Security

See [SECURITY.md](SECURITY.md) for vulnerability reporting instructions and threat scope.

## License

Apache-2.0
