# Flash

Managed LoRA post-training service: SFT, GRPO, and on-policy distillation (from a
Fireworks-hosted GLM teacher) on managed Flash GPUs.
The allocator picks the cheapest validated managed GPU class that fits the run.

## Scope

- `flash train <cfg.toml>` / control-plane `POST /runs` — submit a training job;
  one dedicated GPU per run, supervised server-side (stall watchdog, bounded
  auto-retry resuming from the last streamed checkpoint, endpoint GC).
- `flash checkpoints`, `flash deploy`, `flash chat` — serving for trained adapters,
  including deployable intermediate RL checkpoints.
- **Freesolo SDK environments.** Every run names a Freesolo environment id.
  Scaffold `environment.py` plus `dataset/train.jsonl`, upload `.` or another
  folder with `flash env push --name <name> <folder>`, then reference the
  returned id. The worker loads it through `freesolo.environments`. There are no
  built-in task environments. Single-turn and bounded multi-turn environments are
  supported.

## Layout

- `flash/catalog.py` — curated model catalog (Qwen3 dense supported tier;
  Qwen3.5/3.6 experimental tier), VRAM-fit sizing, and each model's `thinking`
  capability (opt-in reasoning mode `thinking = true`)
- `flash/schema.py`, `flash/spec.py` — TOML → `JobSpec`
- `flash/runner.py` — server-side run supervisor (durable job handle,
  retries, cost guard, endpoint GC)
- `flash/providers/` — managed GPU substrate code (pricing, GPU classes, durable
  submit/poll, preflight) behind the `base.Provider` protocol, with an
  `allocator.py` that picks the cheapest fitting managed GPU class
- `flash/engine/` — the on-GPU worker (TRL + colocated vLLM rollouts; distillation
  scores on-policy student samples against a remote GLM teacher) and the shared
  recipe; SFT targets and RL rewards route through the active environment
  (task-specific grading lives with its example, not in the engine)
- `flash/envs/` — environment machinery: registry and the adapter that loads
  Freesolo SDK environments onto the worker's interface
- `flash env setup` — scaffold a starter local Freesolo env, `dataset/train.jsonl`,
  ready-to-run configs, and a `TRAINING.md` playbook with common failure modes and
  mitigations
- `flash/serve/`, `flash/server/` — adapter serving and the FastAPI control
  plane (run operator-side via the separate `flash-server` command)
- `Dockerfile` — the control-plane image (used by the repo docker-compose)
- `tests/` — pytest suite (CPU-only; offline-by-default, no GPU/network)

## Local commands

```bash
cd flash
uv sync --extra server
uv run pytest                           # CPU tests (offline-by-default, no GPU/network)
uv run ruff check . && uv run ruff format .
uv run flash --help
uv run flash-server                      # control plane (operator-side, run once)
```

The control plane owns infrastructure credentials plus the shared `HF_TOKEN`.
The artifact repo is platform-managed and environment-scoped (runs for the same environment share
one private `Freesolo-Co/flashrun-<environment>-<hash>` repo, written by the operator `HF_TOKEN`);
Flash uploads code under content-addressed prefixes and only reuses completed snapshots. The repo is
not a user knob and not an operator-wide env var. Clients authenticate with their freesolo API key
(`flash login`).

## Release channels

Two channels are published to PyPI from the *same source*, distinguished by one line in
`flash/_channel.py` (`CHANNEL`):

| Channel | PyPI package | CLI | Default plane | Published from |
| --- | --- | --- | --- | --- |
| prod | `freesolo-flash` | `flash` | `flash.freesolo.co` | push to `main` that bumps `[project].version` (`.github/workflows/publish.yml`) |
| dev | `freesolo-flash-dev` | `flash-dev` | `flash-dev.freesolo.co` | push to `dev` whose `[tool.flash-dev].version` isn't on PyPI yet (`.github/workflows/publish-dev.yml`) |

Each environment holds exactly **one** channel: both packages ship the same import package
(`flash/`) with one baked `CHANNEL` line, so installing both into the same environment makes the
later install win for *both* CLIs. For side-by-side prod and staging, install each channel in its
own virtualenv (or via `pipx`, which isolates per tool). The dev build is produced by
`scripts/build_dev_dist.py`, which renames the package/CLI and flips `CHANNEL` to `dev` before
`uv build`. Both channels ship at the **same version**: `[project].version` and
`[tool.flash-dev].version` must match (CI enforces this via `.github/workflows/version-parity.yml`),
so cutting a release means bumping both together. Either CLI still honours an explicit
`FLASH_API_URL` / the `login --api-url` flag; the channel only sets the default.

## Serving From an API

`flash chat` is a CLI wrapper around the Flash control-plane chat endpoint. To call a
deployed adapter from your own app, deploy the finished run once and then POST chat
requests with your freesolo API key:

```bash
export FLASH_API_URL=https://flash.freesolo.co
export FREESOLO_API_KEY=fslo_...
export RUN_ID=flash-1782194170-ce1cfcff

curl -X POST "$FLASH_API_URL/v1/runs/$RUN_ID/deploy" \
  -H "Authorization: Bearer $FREESOLO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"dry_run": false}'

curl -X POST "$FLASH_API_URL/v1/runs/$RUN_ID/chat" \
  -H "Authorization: Bearer $FREESOLO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": "Write a two-sentence summary of the run."}
    ],
    "temperature": 0.0,
    "max_tokens": 256
  }'
```

The response uses the OpenAI chat-completions shape:

```json
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "..."
      }
    }
  ]
}
```

Use `choices[0].message.content` for the generated text. The run id is the adapter id
for serving. If the run is not deployed yet, `/v1/runs/<run_id>/chat` returns `409`
with a hint to deploy first.

Operators can also call the serving backend directly after the adapter is registered.
The default serving backend is `https://serve.freesolo.co`, and
operators can point Flash at another backend by setting `FREESOLO_SERVING_URL`.
Use that same base URL when calling the backend directly; pass the run id as `model`:

```bash
export FREESOLO_SERVING_URL=https://serve.freesolo.co

curl -X POST "$FREESOLO_SERVING_URL/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "flash-1782194170-ce1cfcff",
    "messages": [{"role": "user", "content": "Hello"}],
    "temperature": 0.0,
    "max_tokens": 256
  }'
```

Prefer the Flash control-plane endpoint for user apps because it enforces run ownership
and forwards per-run serving options such as thinking-mode parity.
