Metadata-Version: 2.4
Name: autoslm-train
Version: 0.2.0
Summary: Managed LoRA post-training: claim a key, train on managed consumer GPUs
Project-URL: Homepage, https://github.com/Eric-Mao06/autoslm
Project-URL: Repository, https://github.com/Eric-Mao06/autoslm
Project-URL: Documentation, https://github.com/Eric-Mao06/autoslm/tree/main/docs
Project-URL: Issues, https://github.com/Eric-Mao06/autoslm/issues
Project-URL: Changelog, https://github.com/Eric-Mao06/autoslm/blob/main/CHANGELOG.md
Author: AutoSLM contributors
Maintainer: AutoSLM contributors
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: fine-tuning,grpo,llm,lora,post-train,rlhf,runpod,sft
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: <3.13,>=3.11
Requires-Dist: datasets>=2.19
Requires-Dist: huggingface-hub>=0.25
Requires-Dist: runpod-flash
Provides-Extra: all
Requires-Dist: accelerate>=1.4; extra == 'all'
Requires-Dist: bitsandbytes>=0.49; extra == 'all'
Requires-Dist: fastapi; extra == 'all'
Requires-Dist: peft>=0.19; extra == 'all'
Requires-Dist: torch==2.10.0; extra == 'all'
Requires-Dist: transformers<5.11,>=5.6; extra == 'all'
Requires-Dist: trl<1.6,>=1.5; extra == 'all'
Requires-Dist: uvicorn; extra == 'all'
Requires-Dist: vllm==0.19.1; extra == 'all'
Provides-Extra: gpu
Requires-Dist: accelerate>=1.4; extra == 'gpu'
Requires-Dist: bitsandbytes>=0.49; extra == 'gpu'
Requires-Dist: peft>=0.19; extra == 'gpu'
Requires-Dist: torch==2.10.0; extra == 'gpu'
Requires-Dist: transformers<5.11,>=5.6; extra == 'gpu'
Requires-Dist: trl<1.6,>=1.5; extra == 'gpu'
Requires-Dist: vllm==0.19.1; extra == 'gpu'
Provides-Extra: server
Requires-Dist: fastapi; extra == 'server'
Requires-Dist: uvicorn; extra == 'server'
Description-Content-Type: text/markdown

# AutoSLM — managed LoRA post-training: claim a key, train

[![CI](https://github.com/Eric-Mao06/autoslm/actions/workflows/ci.yml/badge.svg)](https://github.com/Eric-Mao06/autoslm/actions/workflows/ci.yml) <!-- pragma: allowlist secret -->
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12-blue.svg)](pyproject.toml)

AutoSLM is a **managed post-train service for small language models** — SFT, GRPO,
**on-policy distillation**, and DPO with LoRA. Like Tinker or the Prime Intellect labs:
**claim a single AutoSLM key with `slm login` and train** — no GPU accounts, no provider
credentials, no Docker. Each run gets one dedicated GPU behind the scenes: RTX 4090/5090,
RTX A5000, and A100 are the validated classes, the full RunPod list (A4000 → B200) is
selectable, and `gpu.type = "cheapest"` picks the cheapest card that fits the model
using live pricing (`slm gpus`). Point a TOML at an environment, train, evaluate, deploy.

Why it's competitive for SLM single-turn post-train (all numbers measured; see
[`bench/results/`](bench/results/)):

- **Cheap.** The identical 40-step GSM8K GRPO recipe on Qwen3-4B: **$0.22 on AutoSLM**
  vs $0.53 on a leading per-token post-train API — with quality parity.
- **Fast enough, and honest about it.** ~15 s per GRPO optimizer step on a 5090
  (was 46 s before the v2 stack) vs ~11 s on the hosted API; provisioning can add
  minutes of queueing at peak (documented trade-off of dedicated consumer GPUs).
- **Durable.** Runs are supervised server-side: your laptop can sleep, crash, or change —
  `slm attach` re-follows from anywhere, checkpoints stream mid-run, and worker loss
  auto-retries on a fresh host resuming from the last checkpoint (verified by live fault
  injection — a killed GPU worker cost 2 steps and 3 minutes).
- **Current models.** Qwen3.5 dense (0.8B–9B), Qwen3 (0.6B–8B), MiniCPM5-1B curated —
  or **any HF model that fits the GPU** via `model_policy = "allow"`.
- **Weights are always downloadable.** Adapters, checkpoints, and eval generations are
  stored per-run and never locked in.

## Install

```bash
uv tool install autoslm-train   # installs the `slm` command
# or: pipx install autoslm-train / pip install autoslm-train
# from source: git clone ... && uv sync && uv run slm --help
```

> The client is pure-Python and lightweight — all GPU work happens on the managed
> service. Requires Python 3.11/3.12. No RunPod or Hugging Face account needed.

## Quickstart

```bash
slm login                                   # instantly claims your AutoSLM API key
slm models                                  # curated consumer-GPU model catalog
slm train configs/examples/gsm8k_grpo.toml  # submits + follows logs (Ctrl-C detaches)
slm status <run_id>                         # state + accrued cost
slm attach <run_id>                         # re-follow from any machine
slm eval  configs/examples/gsm8k_grpo.toml --adapter <run_id>
slm deploy <run_id> --mode dev              # scale-to-zero serving ($0 idle)
slm chat <run_id> -m "What is 17 * 23?"
slm serve-proxy <run_id> --port 8000        # local OpenAI-compatible /v1
slm undeploy <run_id>
```

`slm login --email you@example.com` tags your key; `slm whoami` shows it. Validate any
config without credentials or spend via `slm train cfg.toml --dry-run`.

A run is a single TOML:

```toml
model = "Qwen/Qwen3.5-4B"     # or any HF id with model_policy = "allow"
algorithm = "grpo"            # sft | grpo | opd | dpo

[environment]
id = "gsm8k"                  # built-in: gsm8k | math | tests_pass (or a Hub env id)

[train]
steps = 150
lora_rank = 32
seeds = [0]

[gpu]
type = "RTX 5090"
max_cost_usd = 10.0           # spend guard
max_retries = 2               # auto-resubmit on infra failures (resumes from checkpoint)
```

## Algorithms

| algorithm | what it is | when to use |
|---|---|---|
| `sft` | supervised fine-tune on environment targets | you have reference completions |
| `grpo` | verifiable-reward RL (TRL + colocated vLLM rollouts) | you have a grader, no teacher |
| `opd` | **on-policy distillation**: dense token-level teacher supervision on the student's own rollouts | a stronger open model exists for your task — often GRPO-level lift at ~1/3 the wall-clock (measured: +8.0 pts on GSM8K for a 0.6B student in 3.9 min / $0.04 of GPU) |
| `dpo` | offline preference optimization | you have prompt/chosen/rejected data |

## Models

`slm models` shows the curated tier — every entry passed a real train+eval validation on
its recommended GPU (see `bench/results/phase1/`):

| model | algos | GPU |
|---|---|---|
| Qwen/Qwen3-0.6B, Qwen3-4B-Instruct-2507, Qwen3-8B | sft, grpo | 4090 / 5090 |
| Qwen/Qwen3.5-0.8B, 2B, 4B | sft, grpo (text-only fine-tune) | 4090 / 5090 |
| Qwen/Qwen3.5-9B | sft | 5090 |
| openbmb/MiniCPM5-1B | sft, grpo | 4090 |
| Qwen/Qwen3.6-35B-A3B | sft (QLoRA) — needs a bigger worker disk, provisioned automatically (`min_disk_gb`) | 5090 |

Anything else on the Hub: set `model_policy = "allow"` — a coarse VRAM-fit check blocks
only provably-impossible configurations.

## How it works

```
slm train cfg.toml ──HTTP──> AutoSLM control plane (your claimed key)
                               └─ autoslm.orchestrator        # supervisor: durable job handle,
                                    │                         # stall watchdog, bounded auto-retry,
                                    │                         # endpoint GC — runs server-side
                                    └─ autoslm.flash.durable  # deploy -> submit -> poll (REST)
                                         └─ autoslm.engine.worker   # SFT/GRPO/OPD/DPO + eval on the GPU
```

The CLI is a thin client: configs are parsed and validated locally, then the spec is
submitted to the control plane, which owns provider credentials, supervises the GPU run,
and streams logs back. Because supervision is server-side, runs survive anything that
happens to your machine. (Operators: the control plane is in this package too — see
[self-hosting](docs/self-hosting.md).)

The worker stack is `trl ≥1.5 / vllm 0.19 / transformers 5.x / torch 2.10`. Train and
eval run in separate processes; evals are greedy with a shared grader so SFT targets, RL
rewards, and reported accuracies always agree.

## Serving

Two explicit cost/latency modes (measured numbers in `bench/results/`):

- `slm deploy <run_id> --mode dev` (default): scale-to-zero, configurable
  `--idle-timeout`. **$0 while idle; ~4 min cold start** after inactivity, ~10 s warm
  requests.
- `slm deploy <run_id> --mode always-on`: one warm worker 24/7 — no cold starts,
  continuous billing (projected $/day printed at deploy time).

`slm serve-proxy` exposes a local OpenAI-compatible `/v1` so any OpenAI SDK client works
against your deployment. `slm deployments` lists what's running; `slm undeploy` stops it.

## Environments

Built-ins implement a small task interface (`dataset`, `prompt_messages`, `sft_target`,
`reward`, `grade`): **`gsm8k`**, **`math`** (DeepScaleR → MATH-500), **`tests_pass`**
(diff-apply + test command). Prime Intellect `verifiers` environments run unchanged
(single-turn) via `slm env install owner/name`. Scaffold custom ones with `slm env init`
and publish them with `slm env push` — the managed service runs built-in and Hub
environments (local `path = ...` environments work only on a self-hosted control plane's
machine and are rejected by the managed API).

## Reliability notes (what actually goes wrong on serverless GPUs, and what we do)

- Transient "no GPU availability" and **throttled hosts**: surfaced in the run log;
  the stall watchdog re-submits on a fresh endpoint after `AUTOSLM_STALL_AFTER_S`.
- **Worker loss mid-run**: the job retries; the replacement worker resumes from
  the latest streamed checkpoint (≤ one save interval lost).
- **Client crash**: irrelevant — supervision is server-side; `slm attach <run_id>`
  re-follows the logs from anywhere.
- **RTX 5090 (sm_120)**: pypi wheels ship no sm_120 SASS, so workers are pinned to
  CUDA-13-driver hosts automatically (`AUTOSLM_MIN_CUDA` to override).
- Worker crash forensics: `error_<mode>.txt` + `console_<mode>.txt` are kept per run.

## Docs

- [Config reference](docs/config-reference.md) — TOML schema, algorithms, env knobs,
  memory recipes.
- [Environments](docs/environments.md) — authoring + verifiers interop.
- [Self-hosting the control plane](docs/self-hosting.md) — run the managed service
  yourself (operator guide).
- [Benchmarks & upgrade evidence](bench/results/) — phase-by-phase before/after reports.

## Development

```bash
uv sync --extra server
AUTOSLM_SKIP_NET=1 uv run pytest tests -q   # CPU tests (no GPU/network)
uv run ruff check . && uv run ruff format .
```

See [CONTRIBUTING.md](CONTRIBUTING.md). CI runs tests (3.11 + 3.12) + ruff on every push.

## License

[Apache-2.0](LICENSE).
