# Pysae Projects Reference

Central reference mapping Pysae repositories to their associated resources.

## Projects

### Application repositories

| Repository | K8s deployment | Description |
|-----------|----------------|-------------|
| `pysae/api` | `pysae-api` | API backend principale (Python/FastAPI) |
| `pysae/op` | `op-dashboard` | Application de supervision (Vue.js) |
| `pysae/driver` | `driver-app` | Application mobile conducteur (Vue.js) |
| `pysae/info` | `info-app` | Affichage information voyageurs (Vue.js) |
| `pysae/editor` | `editor` | Éditeur de réseau de transport (Vue.js) |
| `pysae/screen` | `screen` | Écran embarqué (Vue.js) |
| `pysae/gtfs-validator` | `gtfs-validator` | Validation des flux GTFS |
| `pysae/gtfsrt-to-siri` | `gtfsrt-to-siri` | Convertisseur GTFS-RT vers SIRI |
| `pysae/prefect` | `prefect-worker` | Workers Prefect (orchestration de flux) |
| `pysae/nav-data-processor-api` | `nav-data-processor` | API de traitement des données de navigation |
| `pysae/shift/app` | `shift-app`, `shift-api` | Plateforme Shift — monorepo (web Vue.js + API Node.js) |

To resolve the GitLab project ID for API calls on a different repo: `glab api "projects/pysae%2F<repo>/..."` (URL-encode the path). For subgroup repos: `pysae%2Fshift%2Fapp`.

### Platform & infra repositories

| Repository                   | Description                                                                         |
|------------------------------|-------------------------------------------------------------------------------------|
| `pysae/shift/specs`          | Spécifications API Shift (OpenAPI, docs de conception)                              |
| `pysae/platform-tools`       | Outillage développeur : skills Claude Code, scripts CI, relais webhook GitLab       |
| `pysae/argo-apps`            | Définitions ArgoCD app-of-apps pour tous les déploiements applicatifs               |
| `pysae/infra/argo-infra`     | Définitions ArgoCD pour les services d'infrastructure (ingress, cert-manager, etc.) |
| `pysae/infra/auth-proxy`     | Proxy d'authentification pour les services internes                                 |
| `pysae/infra/infra-common`   | Modules Terraform partagés et configuration infra (VPC, IAM, sécurité)              |
| `pysae/infra/infra-cluster`  | Configuration Terraform des clusters EKS (dev et prod)                              |
| `pysae/infra/pysae-perf`     | Suite de tests de performance (tests de charge k6)                                  |

## Slack Channels

| Repository / keyword                          | Channel          | Channel ID    |
|-----------------------------------------------|------------------|---------------|
| `api`                                         | `#tech-api`      | `C05SWB6MXE3` |
| `driver`                                      | `#tech-driver`   | `C057538V93N` |
| `op`, `info`, `screen`, `shift`, or other frontend app | `#tech-frontend` | `C04ASKNPJES` |
| *(anything else)*                             | `#tech-global`   | `C098LM0192B` |

## OpenAPI Specifications

The Pysae API exposes OpenAPI 3.1.0 specs for each API version and visibility level. Specs are available in both dev and prod environments.

Base URLs:
- **Dev**: `https://api.dev.pysae.com` (offline nights and weekends)
- **Prod**: `https://api.pysae.com`

Spec URL pattern: `<base>/api/docs/<version>/<visibility>/openapi.json`

Swagger UI pattern: `<base>/docs/<version>/<visibility>` (redirects to latest version, public visibility by default)

| Version | Visibility | Prod spec URL | Title |
|---------|------------|---------------|-------|
| `v3` | `public` | `https://api.pysae.com/api/docs/v3/public/openapi.json` | Pysae Public API v3 |
| `v3` | `internal` | `https://api.pysae.com/api/docs/v3/internal/openapi.json` | Pysae Internal API v3 |
| `v4` | `public` | `https://api.pysae.com/api/docs/v4/public/openapi.json` | Pysae Public API v4 |
| `v4` | `internal` | `https://api.pysae.com/api/docs/v4/internal/openapi.json` | Pysae Internal API v4 |
| `v5` | `public` | `https://api.pysae.com/api/docs/v5/public/openapi.json` | Pysae Public API v5 |
| `v5` | `internal` | `https://api.pysae.com/api/docs/v5/internal/openapi.json` | Pysae Internal API v5 |
| `all` | `public` | `https://api.pysae.com/api/docs/all/public/openapi.json` | Pysae Public API (all versions merged) |
| `all` | `internal` | `https://api.pysae.com/api/docs/all/internal/openapi.json` | Pysae Internal API (all versions merged) |

Notes:
- `public` specs contain endpoints accessible with standard API keys
- `internal` specs include all endpoints (public + internal/admin) — prefer `internal` when importing a spec
- `all` merges endpoints from v3, v4, and v5 into a single spec
- v5 public is currently empty (no public endpoints yet)
- Dev URLs follow the same pattern with `api.dev.pysae.com` as base
- When importing into Postman, prefer importing **each version separately** (v3, v4, v5) rather than `all` — this gives one collection per version, making navigation and testing easier

## Kubernetes Deployments

### API flavors

The API is deployed as multiple specialized deployments ("flavors"), all sharing the same image but serving different workloads.

| Deployment suffix | Description                                                |
|-------------------|------------------------------------------------------------|
| `api`             | API tornado legacy                                         |
| `api-fast`        | Endpoints de lecture haut débit (positions véhicules, SSE) |
| `api-ndp`         | NDP (Numerical Data Platform) — ingestion/export en masse  |
| `api-gtfsrt`      | Génération des flux GTFS-RT                                |
| `api-evt`         | Traitement des événements (webhooks, tâches asynchrones)   |
| `api-stats`       | Endpoints de statistiques et d'analyse                     |
| `api-simu`        | Moteur de simulation                                       |
| `api-docs`        | Documentation API (Swagger/ReDoc)                          |

Deployment names follow the pattern `<env>-<suffix>` (e.g. `dev-api-fast`, `prod-api-ndp`).

### Deployment naming conventions

| Context      | Pattern                          | Namespace | kubectl context |
|--------------|----------------------------------|-----------|-----------------|
| Dev          | `dev-<service>`                  | `dev`     | `dev`           |
| Prod         | `prod-<service>`                 | `prod`    | `prod`          |
| Review       | `review-<slug>-<service>`        | `mr`      | `dev`           |
| Infra        | `<app-name>`                     | per-app   | both            |

### Container registry

All app images are stored in AWS ECR: `596368530845.dkr.ecr.eu-west-3.amazonaws.com/apps/<service>:<tag>`

Image tag formats:
- Dev: `main-<short-sha>` (or `main-<short-sha>-dev` for driver/info)
- Prod: `v<major>-<minor>-<patch>-<short-sha>` (semver)
- Review: `<branch-slug>-<short-sha>`

## ArgoCD Deployments

### Application services

Apps that map to a GitLab repository and are deployed per environment.

| Repository            | Dev app                | Prod app                | Review app pattern                    | Namespace (dev/prod/review) |
|-----------------------|------------------------|-------------------------|---------------------------------------|-----------------------------|
| `pysae/api`           | `dev-api`              | `prod-api`              | `review-<slug>-api`                   | `dev` / `prod` / `mr`      |
| `pysae/driver`        | `dev-driver`           | `prod-driver`           | `review-<slug>-driver`                | `dev` / `prod` / `mr`      |
| `pysae/op`            | `dev-op`               | `prod-op`               | `review-<slug>-op`                    | `dev` / `prod` / `mr`      |
| `pysae/editor`        | `dev-editor`           | `prod-editor`           | `review-<slug>-editor`               | `dev` / `prod` / `mr`      |
| `pysae/info`          | `dev-info`             | `prod-info`             | `review-<slug>-info`                  | `dev` / `prod` / `mr`      |
| `pysae/screen`        | `dev-screen`           | `prod-screen`           | `review-<slug>-screen`               | `dev` / `prod` / `mr`      |
| `pysae/gtfs-validator`| `dev-gtfs-validator`   | `prod-gtfs-validator`   | `review-<slug>-gtfs-validator`        | `dev` / `prod` / `mr`      |
| `pysae/gtfsrt-to-siri`| `dev-gtfsrt-to-siri`  | `prod-gtfsrt-to-siri`   | `review-<slug>-gtfsrt-to-siri`       | `dev` / `prod` / `mr`      |
| `pysae/shift/app`     | `dev-shift-app`, `dev-shift-api` | `prod-shift-app`, `prod-shift-api` | `review-<slug>-shift-app`, `review-<slug>-shift-api` | `dev` / `prod` / `mr` |
| `pysae/platform-tools`| `dev-gitlab-webhook-relay` | —                    | `review-<slug>-gitlab-webhook-relay`  | `gitlab`                    |

Review environments deploy all apps together under namespace `mr`, with app names following the pattern `review-<slug>-<service>`. The orchestrator app is `review-<slug>-main`.

### Prefect workers

| Environment | App                  | Source                                    | Namespace |
|-------------|----------------------|-------------------------------------------|-----------|
| Dev         | `dev-prefect-worker` | `prefecthq.github.io/prefect-helm`        | `dev`     |
| Prod        | `prod-prefect-worker`| `prefecthq.github.io/prefect-helm`        | `prod`    |

### Infrastructure services

Managed via `pysae/infra/argo-infra` and `pysae/infra/auth-proxy`. Deployed on both clusters.

| App                              | Namespace                        | Source repo                        |
|----------------------------------|----------------------------------|------------------------------------|
| `nginx-ingress-controller`       | `ingress`                        | `pysae/infra/argo-infra`           |
| `cert-manager`                   | `cert-manager`                   | `pysae/infra/argo-infra`           |
| `cert-manager-issuers`           | `cert-manager`                   | `pysae/infra/argo-infra`           |
| `datadog`                        | `datadog`                        | `pysae/infra/argo-infra`           |
| `external-dns`                   | `external-dns`                   | `pysae/infra/argo-infra`           |
| `external-secrets`               | `dev`                            | `pysae/infra/argo-infra`           |
| `keda`                           | `keda`                           | `pysae/infra/argo-infra`           |
| `kyverno`                        | `kyverno`                        | `pysae/infra/argo-infra`           |
| `autoscaler`                     | `autoscaler`                     | `pysae/infra/argo-infra`           |
| `aws-node-termination-handler`   | `aws-node-termination-handler`   | `pysae/infra/argo-infra`           |
| `descheduler`                    | `descheduler`                    | `pysae/infra/argo-infra`           |
| `oauth2-proxy`                   | `oauth2-proxy`                   | `pysae/infra/argo-infra`           |
| `auth-proxy`                     | `auth-proxy`                     | `pysae/infra/auth-proxy`           |
| `metabase`                       | `metabase`                       | `pysae/infra/argo-infra`           |
| `n8n`                            | `n8n`                            | `pysae/infra/argo-infra`           |
| `uptime-kuma`                    | `uptime-kuma`                    | `pysae/infra/argo-infra`           |

### Root apps (App-of-Apps)

| App            | Cluster | Source repo                |
|----------------|---------|----------------------------|
| `apps-root`    | both    | `pysae/argo-apps`          |
| `infra-root`   | both    | `pysae/infra/argo-infra`   |
| `dev-main`     | dev     | `pysae/argo-apps`          |
| `prod-main`    | prod    | `pysae/argo-apps`          |

## Datadog

### Environment tags (`env`)

| Value | Description |
|-------|-------------|
| `prod` | Production |
| `dev` | Development (main branch deployed) |
| `review-<slug>` | Review environment (one per MR, slug = branch name) |
| `local` | Local development |
| `global` | Cross-environment (used for finops/cost monitors) |

### Service names (`service`)

Service names in Datadog map to deployment flavors, not repositories.

| Service | Repository | Description |
|---------|------------|-------------|
| `api` | `pysae/api` | API REST principale (FastAPI) |
| `api-nav-data-processor` | `pysae/api` | Flavor NDP (déploiement `api-ndp`) |
| `api-gtfs-rt` | `pysae/api` | Flavor GTFS-RT (déploiement `api-gtfsrt`) |
| `api-stats` | `pysae/api` | Flavor statistiques |
| `api-simulation` | `pysae/api` | Flavor simulation (déploiement `api-simu`) |
| `events-handler` | `pysae/api` | Flavor traitement d'événements (déploiement `api-evt`) |
| `docs` | `pysae/api` | Documentation API (déploiement `api-docs`) |
| `driver` | `pysae/driver` | Application conducteur (browser RUM) |
| `op` | `pysae/op` | Application de supervision (browser RUM) |
| `editor` | `pysae/editor` | Éditeur de réseau (browser RUM) |
| `info` | `pysae/info` | Affichage information voyageurs (browser RUM) |
| `screen` | `pysae/screen` | Écran embarqué (browser RUM) |
| `gtfsrt-to-siri` | `pysae/gtfsrt-to-siri` | Convertisseur GTFS-RT vers SIRI |
| `gtfs-validator` | `pysae/gtfs-validator` | Validateur GTFS |
| `shift-app` | `pysae/shift/app` | Frontend web Shift (browser RUM) |
| `shift-api` | `pysae/shift/app` | Backend API Shift |
| `gitlab-webhook-relay` | `pysae/platform-tools` | Relais webhook GitLab |
| `prefect` / `pysae-prefect` | — | Workers Prefect |
| `redis` | — | Instances Redis |

### Monitors

Monitors are tagged with `env` and `service`. Finops monitors use `team:finops` and `type:budget`.

## Prefect

### Work Pools

| Name | Type | Environment | Concurrency |
|------|------|-------------|-------------|
| `dev-k8s` | kubernetes | Dev | 10 |
| `prod-k8s` | kubernetes | Prod | 10 |
| `dev-process-k8s` | process | Dev | None |
| `prod-process-k8s` | process | Prod | None |
| `default-agent-pool` | prefect-agent | — | None |

### Flows

| Flow | Description |
|------|-------------|
| `Publish GTFS` | Publication des flux GTFS statiques pour les réseaux de transport |
| `Synchronize external transit data` | Synchronisation des données de transport depuis des sources externes |
| `Launch notification on long trip` | Alerte sur les courses conducteur anormalement longues |
| `Delete device stale trip` | Nettoyage des courses périmées des appareils |
| `Cleanup repos` | Nettoyage des dépôts anciens/inutilisés |
| `remove old review environments` | Suppression des environnements review expirés dans ArgoCD |
| `Migrate groups and repos` | Migration de données pour les groupes et dépôts |
| `Delete deprecated collections` | Suppression des collections MongoDB dépréciées |

### Deployments

Deployments follow the pattern `<flow>/<flow> (<env>)` and run on `<env>-k8s` work pools.

| Flow | Dev deployment | Prod deployment |
|------|---------------|-----------------|
| Publish GTFS | `dev-k8s` | `prod-k8s` |
| Synchronize external transit data | `dev-k8s` | `prod-k8s` |
| Launch notification on long trip | `dev-k8s` | `prod-k8s` |
| Cleanup repos | `dev-k8s` | `prod-k8s` |
| remove old review environments | `dev-k8s` | `prod-k8s` |
| Migrate groups and repos | — | `prod-k8s` |

## AWS

Account: `596368530845` | Region: `eu-west-3` (Paris)

### EKS Clusters

| Cluster | Purpose |
|---------|---------|
| `dev` | Development + review environments |
| `prod` | Production |
| `pysae-perf` | Performance/load testing |

### Secrets Manager

Application secrets follow the pattern `pysae/<env>/secrets`. Each secret is a JSON object containing key-value pairs.

| Secret | Description |
|--------|-------------|
| `pysae/prod/secrets` | Secrets applicatifs de production |
| `pysae/dev/secrets` | Secrets applicatifs de développement |
| `pysae/review/secrets` | Secrets des environnements review (partagés entre toutes les review) |
| `pysae/testing/secrets` | Secrets de test/CI |
| `pysae/local-dev/secrets` | Développement local (mongo-uri uniquement) |
| `pysae/local-prod/secrets` | Développement local type prod (mongo-uri uniquement) |
| `prod/sftp-bastion/ssh-private-key` | Clé SSH pour le transfert de fichiers vers le bastion SFTP (prod) |
| `DdApiKeySecret-Cr3SbiBZUGvq` | Clé API Datadog (utilisée par l'agent Datadog sur EKS) |

#### Secret keys detail

Keys present in each environment secret (`pysae/<env>/secrets`):

| Key | Env var | Description | Flavors | prod | dev | review | testing |
|-----|---------|-------------|---------|:----:|:---:|:------:|:-------:|
| `api-mongo-uri` | `MONGO_URI` | MongoDB connection string | all | x | x | x | x |
| `api-auth0-mgmt-client-id` | `AUTH0_MGMT_CLIENT_ID` | Auth0 Management API client ID | fast, stats | x | x | x | |
| `api-auth0-mgmt-client-secret` | `AUTH0_MGMT_CLIENT_SECRET` | Auth0 Management API client secret | fast, stats | x | x | x | |
| `api-firebase-json` | *(mounted as file)* | Firebase service account JSON (push notifications) | fast, ndp, gtfsrt, stats, simu | x | x | | |
| `api-prefect-api-key` | `PREFECT_API_KEY` | Prefect Cloud API key (flow triggers) | all | x | x | x | x |
| `api-recaptcha-secret` | `RECAPTCHA_SECRET` | Google reCAPTCHA secret key | fast | x | x | x | x |
| `api-twilio-auth-token` | `TWILIO_AUTH_TOKEN` | Twilio auth token (SMS notifications) | fast, ndp, stats | x | x | x | x |
| `gtfsrt-to-siri-auth0-client-secret` | — | Auth0 client secret for SIRI auth | *(gtfsrt-to-siri)* | x | x | x | |
| `gitlab-webhook-relay-trigger-token` | — | GitLab pipeline trigger token | *(webhook-relay)* | x | x | | |
| `gitlab-webhook-relay-webhook-secret` | — | GitLab webhook signature secret | *(webhook-relay)* | x | x | | |
| `prefect-worker-api-key` | — | Prefect Cloud API key (worker auth) | *(prefect-worker)* | x | x | x | x |

### ECR (Container Registry)

Base URI: `596368530845.dkr.ecr.eu-west-3.amazonaws.com`

| Repository | Description |
|------------|-------------|
| `apps/api` | Image API (partagée par tous les flavors) |
| `apps/api-fast` | Image API fast (legacy, peut encore être référencée) |
| `apps/api-static` | Assets statiques de l'API |
| `apps/nav-data-processor-api` | Image API NDP (Nav Data Processor) |
| `apps/driver` | Application conducteur |
| `apps/op` | Application de supervision |
| `apps/editor` | Éditeur de réseau |
| `apps/info` | Affichage information voyageurs |
| `apps/screen` | Écran embarqué |
| `apps/gtfs-validator` | Validateur GTFS |
| `apps/gtfsrt-to-siri` | Convertisseur GTFS-RT vers SIRI |
| `apps/shift-app` | Frontend web Shift |
| `apps/shift-api` | Backend API Shift |
| `apps/gitlab-webhook-relay` | Relais webhook GitLab |
| `infra/pysae-prefect` | Image worker Prefect |
| `infra/auth-proxy` | Proxy d'authentification |
| `infra/ci-deployer` | Image d'aide au déploiement CI |
| `infra/backup-restore` | Image de sauvegarde/restauration de base de données |

### S3 Buckets

| Bucket | Description |
|--------|-------------|
| `pysae-prefect` | Stockage Prefect (partagé) |
| `pysae-prefect-dev` | Stockage Prefect (dev) |
| `pysae-prefect-prod` | Stockage Prefect (prod) |
| `pysae-db-dump` | Dumps de base de données |
| `pysae-billing` | Données de facturation |
| `pysae-public-files` | Fichiers publics (accessibles sans authentification) |
| `pysae-public-tools` | Outils et scripts publics |
| `pysae-tf-states` | État distant Terraform |
| `pysae-orchestration-test` | Tests d'orchestration |
| `gtfs2netex-bucket` | Données de conversion GTFS vers NeTEx |
| `datadogintegration-forwarderstack--forwarderbucket-*` | Forwarder de logs Datadog (géré par CloudFormation) |

## Operational

### Dev environment wake-up / sleep

The dev environment has a cost-saving mechanism:
- **Sleep**: scales down dev deployments to 0 replicas (nightly or manual)
- **Wake-up**: scales back up (morning or manual trigger via CI job in `infra-cluster`)

### Cross-repo deploy dependencies

Some deploys require coordination across repositories:

| Change in | May require deploy of |
|-----------|----------------------|
| `pysae/api` (new endpoint) | op, driver, info (API consumers) |
| `pysae/api` (OpenAPI spec change) | op (Orval codegen) |
| `pysae/infra/gitlab-templates` | All repos (CI template consumers) |
| `pysae/argo-apps` (values change) | Target service (ArgoCD sync) |
| `pysae/gtfsrt-to-siri` (new feature) | op (UI for SIRI config) |
