Metadata-Version: 2.5
Name: freesolo-flash
Version: 1.3.32
Summary: Flash — managed LoRA post-training (SFT/GRPO/OPD) for Freesolo environments, driven by the `flash` CLI
Project-URL: Homepage, https://github.com/freesolo-co/flash
Project-URL: Repository, https://github.com/freesolo-co/flash
Author: Freesolo
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: fine-tuning,freesolo,grpo,llm,lora,rl,sft
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: <3.13,>=3.11
Provides-Extra: dev
Requires-Dist: cloudpickle>=3.0; extra == 'dev'
Requires-Dist: datasets>=2.19; extra == 'dev'
Requires-Dist: fastapi<1,>=0.136; extra == 'dev'
Requires-Dist: freesolo>=0.4.2; extra == 'dev'
Requires-Dist: httpx>=0.27; extra == 'dev'
Requires-Dist: huggingface-hub>=1.2.0; extra == 'dev'
Requires-Dist: jsonschema>=4.23; extra == 'dev'
Requires-Dist: modal==1.6.2; extra == 'dev'
Requires-Dist: mypy>=1.13.0; extra == 'dev'
Requires-Dist: orjson>=3.10; extra == 'dev'
Requires-Dist: packaging<27,>=24; extra == 'dev'
Requires-Dist: pillow>=11.0.0; extra == 'dev'
Requires-Dist: pydantic-settings>=2.7; extra == 'dev'
Requires-Dist: pytest>=9.0.3; extra == 'dev'
Requires-Dist: referencing>=0.28.4; extra == 'dev'
Requires-Dist: regex>=2024.11.6; extra == 'dev'
Requires-Dist: ruff<0.18,>=0.6; extra == 'dev'
Requires-Dist: tiktoken>=0.12; extra == 'dev'
Requires-Dist: tokenizers>=0.22; extra == 'dev'
Requires-Dist: transformers<5.13,>=5.8; extra == 'dev'
Requires-Dist: uvicorn<1,>=0.52; extra == 'dev'
Provides-Extra: gpu
Requires-Dist: accelerate>=1.4; extra == 'gpu'
Requires-Dist: datasets>=2.19; extra == 'gpu'
Requires-Dist: freesolo>=0.4.2; extra == 'gpu'
Requires-Dist: httpx>=0.27; extra == 'gpu'
Requires-Dist: huggingface-hub>=1.2.0; extra == 'gpu'
Requires-Dist: peft>=0.19; extra == 'gpu'
Requires-Dist: tiktoken>=0.12; extra == 'gpu'
Requires-Dist: tokenizers>=0.22; extra == 'gpu'
Requires-Dist: torch==2.10.0; extra == 'gpu'
Requires-Dist: transformers<5.13,>=5.8; extra == 'gpu'
Requires-Dist: uv>=0.5; extra == 'gpu'
Requires-Dist: vllm==0.19.1; extra == 'gpu'
Provides-Extra: serve-runtime
Requires-Dist: fastapi<0.137,>=0.136; extra == 'serve-runtime'
Requires-Dist: huggingface-hub>=1.2.0; extra == 'serve-runtime'
Requires-Dist: pillow>=11.0.0; extra == 'serve-runtime'
Requires-Dist: safetensors<1,>=0.8; extra == 'serve-runtime'
Requires-Dist: tokenizers>=0.22; extra == 'serve-runtime'
Requires-Dist: transformers<5.13,>=5.8; extra == 'serve-runtime'
Requires-Dist: uvicorn<1,>=0.52; extra == 'serve-runtime'
Requires-Dist: vllm==0.23.0; extra == 'serve-runtime'
Provides-Extra: server
Requires-Dist: datasets>=2.19; extra == 'server'
Requires-Dist: fastapi<1,>=0.136; extra == 'server'
Requires-Dist: freesolo>=0.4.2; extra == 'server'
Requires-Dist: httpx>=0.27; extra == 'server'
Requires-Dist: huggingface-hub>=1.2.0; extra == 'server'
Requires-Dist: jinja2>=3.1.0; extra == 'server'
Requires-Dist: jsonschema>=4.23; extra == 'server'
Requires-Dist: modal==1.6.2; extra == 'server'
Requires-Dist: orjson>=3.10; extra == 'server'
Requires-Dist: pillow>=11.0.0; extra == 'server'
Requires-Dist: pydantic-settings>=2.7; extra == 'server'
Requires-Dist: referencing>=0.28.4; extra == 'server'
Requires-Dist: regex>=2024.11.6; extra == 'server'
Requires-Dist: transformers<5.13,>=5.8; extra == 'server'
Requires-Dist: uvicorn<1,>=0.52; extra == 'server'
Provides-Extra: serving
Requires-Dist: anyio<5,>=4.0; extra == 'serving'
Requires-Dist: fastapi<1,>=0.136; extra == 'serving'
Requires-Dist: httpx>=0.27.0; extra == 'serving'
Requires-Dist: modal==1.6.2; extra == 'serving'
Requires-Dist: orjson>=3.10; extra == 'serving'
Requires-Dist: orjson>=3.10.0; extra == 'serving'
Requires-Dist: pillow>=11.0.0; extra == 'serving'
Requires-Dist: pydantic-settings>=2.6.0; extra == 'serving'
Requires-Dist: pydantic-settings>=2.7; extra == 'serving'
Requires-Dist: pydantic>=2.7.0; extra == 'serving'
Requires-Dist: python-dotenv>=1.0.0; extra == 'serving'
Requires-Dist: starlette<2,>=0.46; extra == 'serving'
Description-Content-Type: text/markdown

# Flash

LoRA post-training for open-weight models: SFT, GRPO, and on-policy distillation. You
describe a run in a TOML file, Flash allocates a GPU, trains, streams checkpoints, and
serves the resulting adapter.

```bash
pip install freesolo-flash
export FREESOLO_API_KEY=fslo_...
flash login
flash train run.toml
```

The allocator picks the cheapest validated GPU class that fits the run, then supervises it
server-side: stall watchdog, bounded auto-retry resuming from the last streamed checkpoint,
and endpoint GC.

## What this repository is

Flash is the **client and control plane** for Freesolo's hosted post-training service:

| Path               | What it is                                                        |
| ------------------ | ----------------------------------------------------------------- |
| `flash/cli/`       | the `flash` CLI — no declared runtime dependencies\*              |
| `flash/server/`    | the FastAPI control plane — run submission, auth, project scoping |
| `flash/engine/`    | the GPU worker and training recipes — verl + colocated vLLM       |
| `flash/providers/` | the GPU substrate — pricing, allocation, submit/poll              |
| `flash/envs/`      | environment loading                                               |

\* `pip install freesolo-flash` pulls nothing. Commands that run an environment **locally** —
`flash env test` and `flash env eval` — do need the `freesolo` SDK, which they import at the
point of use; install it with `pip install freesolo`.

Two components stay Freesolo-operated and are **not** in this repository:

| Component                 | Where it lives                                                                                         | Self-hosted equivalent                                            |
| ------------------------- | ------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------- |
| Multi-tenant identity     | `api.freesolo.co`; verifies keys and owns projects/orgs                                                | `FLASH_STANDALONE=1` runs single-tenant on your own key           |
| Hosted Multi-LoRA serving | `serve.freesolo.co`, deployed from `flash/serving/`; `flash/serve/` owns the client and shared runtime | `flash serve deploy` provisions serving in your own Modal account |

So there are three ways to use Flash: against the **hosted service**, **self-hosted** against
your own GPU accounts, or as **training and provider code to read and modify**. The training
path is self-hostable end to end — see **[SELF_HOSTING.md](SELF_HOSTING.md)**.

## Using the hosted service

The active training catalog is `Qwen/Qwen3.5-9B`, `Qwen/Qwen3.8-27B`, and
`Qwen/Qwen3.6-35B-A3B`. The hosted serving catalog routes all three models on B200.
The retired dense Qwen3.6 27B identity is never translated into Qwen3.8. Customer-owned
Modal supports all three active models; 27B and 35B-A3B are bound to the certified serving image
digest. Customer-owned `flash serve` requires explicit `--provider modal`. Training and serving both use Modal.

`flash login` is not interactive. Export `FREESOLO_API_KEY` first so the key does not appear in process listings:

```bash
pip install freesolo-flash
export FREESOLO_API_KEY=fslo_...
flash login          # validates the key, stores it in ~/.flash/config.json
flash whoami         # confirm the identity behind it
```

Every run names an environment, which supplies the task data and the reward or SFT target.
Environments are published under a project, which scopes them to an organization:

```bash
flash projects create my-project                       # returns a project uuid
flash projects list                                    # look up existing uuids
flash env setup                                        # scaffold environment.py + dataset/train.jsonl
flash env push --project PROJECT_UUID --name my-env .  # returns an environment id
```

Every training TOML carries a required top-level `project`, validated against the
authenticated organization before Flash allocates anything:

```toml
project = "your-project-uuid"
model = "Qwen/Qwen3.5-9B"
algorithm = "sft"

[environment]
id = "your-org/your-project/my-env"

[train]
epochs = 1
max_examples = 1000
lora_rank = 32              # lora_alpha defaults to 2 x lora_rank; set it to override
```

```bash
flash train run.toml                  # submit, prints a run id
flash runs status RUN_ID               # follow it
flash models deploy RUN_ID/final             # serve the final checkpoint
flash models chat RUN_ID/final -m "hello"    # talk to it
```

Run management lives under `flash runs` (`list`, `status`, `log`, `cancel`, `checkpoint`)
and serving under `flash models` (`deploy`, `chat`, `deployments`, `undeploy`, `export`).
`flash models` lists supported base models — six curated Qwen checkpoints — and `flash gpus`
lists GPU classes with estimated $/hr.

Intermediate RL checkpoints are deployable: list them with `flash runs checkpoint RUN_ID`,
then pass `RUN_ID/step-N` as the checkpoint id. To copy a finished adapter into your own
HuggingFace repo:

```bash
export HF_TOKEN=hf_...
flash models export --adapter-id RUN_ID/final --repository your-org/your-repo
```

There are no built-in task environments — the environment you push defines the task.
Single-turn and bounded multi-turn environments are supported.

### Calling a deployed adapter from your own app

Deploy once with `flash models deploy RUN_ID/final`, then POST chat requests with your API key:

```bash
export RUN_ID=flash-1782194170-ce1cfcff
export FREESOLO_API_KEY=fslo_...

curl -X POST "https://flash.freesolo.co/v1/runs/$RUN_ID/chat" \
  -H "Authorization: Bearer $FREESOLO_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"checkpoint_id\": \"$RUN_ID/final\", \"messages\": [{\"role\": \"user\", \"content\": \"hello\"}], \"max_tokens\": 256}"
```

The response uses the OpenAI chat-completions shape; read `choices[0].message.content`. The
checkpoint id is the permanent serving identity. If that checkpoint is not deployed yet, the
endpoint returns `409` with a hint to deploy first. Prefer this control-plane endpoint over calling the serving backend
directly: it enforces run ownership and forwards per-run serving options such as
thinking-mode parity.

### Workload estimates (SFT)

`flash train`, `--dry-run`, and `--cost` read the pinned environment's packaged dataset
without importing `environment.py`. `dataset/train.jsonl` is the canonical default;
`dataset/train.json` also works, and `[environment.params] split` or `dataset_path` can
select another packaged file.

The estimate tokenizes raw `input`/`output` fields plus the statically readable training
contract. SFT rows can instead supply `prompt_messages`, a nonempty chat-message list
that is authoritative for both profiling and training. This preserves stored multi-turn
histories and screenshot blocks; image-bearing transcripts use unpacked execution.
The row still requires `input` for dataset identity. Environment-added prompts, few-shot
examples, tool schemas, filters, and transformations are **not** executed, so real training
can retain fewer rows, truncate more often, and cost more than the estimate. If no readable dataset exists, cost, dry-run, and
submit all fail before GPU allocation rather than billing a profiling run.

## Working on the code

The test suite is CPU-only and offline by default. No GPU, no network, no credentials:

```bash
uv sync --extra server --dev
uv run ruff check .
uv run ruff format --check .
uv run pytest -q
```

CI (`.github/workflows/ci.yml`) runs those on both supported interpreters (3.11 and 3.12),
plus two size gates and a serial pass for timing-sensitive tests:

```bash
uv run python scripts/check_file_size.py      # no module in flash/ over 1000 lines
uv run python scripts/check_function_size.py  # no function in flash/ over 150 lines
uv run pytest -q -m wallclock                 # asserts on real elapsed time; runs alone
```

`mypy` also runs in CI but is **advisory** — it reports existing type errors without failing
the build. Formatting is gated, so run `uv run ruff format .` before you push.

Use `flash`, `flash-cli`, or `python -m flash.cli` from a checkout.

See [CONTRIBUTING.md](CONTRIBUTING.md) for the branching model — in short, **pull requests go
into `dev`**.

### Layout

- `flash/core/catalog.py` — curated model catalog (Qwen3.5, Qwen3.8 dense, and Qwen3.6 MoE), VRAM
  sizing, and each model's `thinking` capability
- `flash/schema/`, `flash/core/spec.py` — TOML to `JobSpec`
- `flash/runner/` — server-side run supervisor (durable job handle, retries, cost guard)
- `flash/providers/` — GPU substrate behind the `base.Provider` protocol, with `allocator.py`
  picking the cheapest fitting class
- `flash/engine/` — the on-GPU worker (verl + colocated vLLM rollouts) and the shared recipe.
  SFT targets and RL rewards route through the active environment, so task-specific grading
  lives with the example, not in the engine
- `flash/envs/` — environment registry and the Freesolo SDK adapter
- `flash/serve/` — serving client, shared runtime, and customer-owned provider deployment
- `flash/serving/` — the hosted multi-LoRA Modal app behind `serve.freesolo.co`
- `flash/server/` — the FastAPI control plane (run via the separate `flash-server` command)
- `tests/` — pytest suite, CPU-only and offline

Within `flash/engine/worker/`, trainers live under `train/`, split by algorithm (`sft/`,
`rl/`, `opd/`) over a shared `core/`. Each carries a `child/` holding stdlib-only modules
**copied** into the verl subprocess rather than imported — flash and verl pin incompatible
torch/vllm versions, so neither can import the other.

## Self-hosting

Run your own control plane against your own GPU accounts, with no Freesolo backend involved.
**[SELF_HOSTING.md](SELF_HOSTING.md) is the full guide**; the short version:

```bash
pip install 'freesolo-flash[server]'   # the base install is client-only

export FLASH_STANDALONE=1
export FREESOLO_INTERNAL_KEY=$(openssl rand -hex 32)
export HF_TOKEN=hf_...
export FLASH_HF_NAMESPACE=your-hf-username   # a namespace your HF_TOKEN can write to
export MODAL_TOKEN_ID=...
export MODAL_TOKEN_SECRET=...

flash-server --host 0.0.0.0 --port 8080
```

Training and serving use **Modal**. Both Modal token variables are required. Training runs in GPU
Sandboxes with persisted attempt identities, checkpoint recovery, cancellation, and a shared
`flash-weights` Volume. GPU selection remains automatic within Modal's supported shapes.

`FLASH_STANDALONE=1` stops the plane calling out for project, environment, and billing
validation, and trusts `FREESOLO_INTERNAL_KEY` as a single-tenant operator credential.
External bearer tokens are rejected rather than accepted unverified. A standalone plane is
**single-tenant** — whoever holds that key can spend your GPU budget, so keep it off
untrusted networks. See [the security model](SELF_HOSTING.md#the-security-model).

`flash serve deploy` provisions serving in your own Modal account and requires explicit
`--provider modal`. Each deployment owns one exact base model and its compatible adapters; another
base model requires another deployment. Legacy provider deployment identities are unsupported.
It implements the same contract `flash/serve/` speaks.

Training selects the public worker image by its qualified digest. The historical `cu128` and
`cu128-sm*` tags can contain an incompatible trainer; they are build outputs, not the launch
contract. The selected image runs the pinned upstream Verl stack, with native kernel JIT when
no compatible baked cache is available:

```bash
docker pull ghcr.io/freesolo-co/flash-worker@sha256:34c84092f87b9ae43457620a8e5c0dc2e3dc30fdd5de5b4300c1eeb384e9a066
```

## Small Qwen3.5 models

Qwen3.5-0.8B and Qwen3.5-4B support text SFT and GRPO with pinned BF16 checkpoints.
Hosted serving uses L40S for 0.8B and exactly H100 for 4B (no H200 substitution),
with 32k context, four hot LoRA slots and rank up to 64. Image input and image training
are not enabled for these two entries. Customer-owned deployment profiles are available
for offline planning; live customer-owned allocation awaits separate qualification.

## Release channels

Two channels are published to PyPI from the _same source_, distinguished by one line in
`flash/_internal/channel.py`:

| Channel | PyPI package         | CLI         | Default plane           | Published on                           |
| ------- | -------------------- | ----------- | ----------------------- | -------------------------------------- |
| prod    | `freesolo-flash`     | `flash`     | `flash.freesolo.co`     | a version bump merged to `main`        |
| dev     | `freesolo-flash-dev` | `flash-dev` | `flash-dev.freesolo.co` | a push to `dev` with an unused version |

Each environment holds exactly **one** channel: both packages ship the same import package
with one baked `CHANNEL` line, so installing both makes the later install win for _both_
CLIs. For side-by-side prod and staging, use a virtualenv per channel (or `pipx`).

Within a commit, `[project].version` and `[tool.flash-dev].version` must match (CI enforces
this), so cutting a release bumps both. The **published** channels still differ, because dev
publishes on merge to `dev` while prod publishes only once `dev` is promoted to `main`.

Either CLI honours an explicit `FLASH_API_URL` or `login --api-url`; the channel only sets
the default.

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md). Security issues: [SECURITY.md](SECURITY.md) — do not
open a public issue.

## License

Apache-2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).

Checkpoint deployment states are `deploying`, `ready`, `undeploying`, `undeployed`,
and `failed`. Queueing, loading, and smoke verification happen within `deploying`.
Removal stays `undeploying` until confirmed, including retries after cleanup errors;
it never becomes `undeployed` merely because a removal request was sent. Existing
persisted states are translated when loaded, preserving registration and generation
fences. Deploy alongside the matching Freesolo deployment-state migration and backend.
