Metadata-Version: 2.4
Name: fitscout
Version: 0.1.1
Summary: Preflight training advisor: detect your hardware, size an LLM fine-tune to fit it, and emit a runnable config - before you download the model.
Author: Muhammed Şen
License-Expression: MIT
License-File: LICENSE
Keywords: fine-tuning,gpu,llm,lora,memory-estimation,qlora,training,vram
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Provides-Extra: all
Requires-Dist: accelerate>=0.30; extra == 'all'
Requires-Dist: bitsandbytes>=0.43; extra == 'all'
Requires-Dist: pynvml>=11.5; extra == 'all'
Requires-Dist: safetensors>=0.4; extra == 'all'
Requires-Dist: torch>=2.1; extra == 'all'
Requires-Dist: transformers>=4.40; extra == 'all'
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest>=7.4; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: hardware
Requires-Dist: pynvml>=11.5; extra == 'hardware'
Provides-Extra: introspect
Requires-Dist: accelerate>=0.30; extra == 'introspect'
Requires-Dist: safetensors>=0.4; extra == 'introspect'
Requires-Dist: transformers>=4.40; extra == 'introspect'
Provides-Extra: quant
Requires-Dist: bitsandbytes>=0.43; extra == 'quant'
Provides-Extra: torch
Requires-Dist: torch>=2.1; extra == 'torch'
Description-Content-Type: text/markdown

# fitscout

**A preflight training advisor for LLM fine-tuning.**

fitscout runs *before* you fine-tune. It detects your hardware and environment
(Colab / AWS / Azure / local), computes the numbers needed to make a given
model fit - full vs LoRA vs QLoRA, micro batch and gradient accumulation,
fp16 vs bf16, gradient checkpointing, 8-bit optimizer - and emits a runnable
config. All of this *before the model is downloaded*.

> fitscout does **not** train your model. It is an advisor that produces input
> for tools like the Hugging Face Trainer, Axolotl and DeepSpeed.

## Why

- **Accurate.** Memory is estimated component by component (weights, gradients,
  optimizer, activations including the output cross-entropy, overhead) with
  principled formulas, not a fitted constant. Optional calibration measures a
  real step once per GPU and tightens the estimate into a guarantee.
- **Generic by design.** fitscout never branches on a model's name or family.
  It reads only architectural fields (layers, hidden size, head counts, KV
  heads, intermediate size, MoE experts). A brand-new model needs a new config,
  never new code - enforced by a test that fails if a model name appears in the
  source.
- **Honest.** Every estimate carries a confidence level and an uncertainty
  margin, and the default recommendation is conservative: it decides on the
  *upper* end of the estimate and never knowingly returns a HIGH-OOM-risk plan.

## Install

```bash
pip install fitscout              # core advisor (dependency-light)
pip install "fitscout[all]"       # + introspection, GPU detection, torch, quant
```

The core has no required dependencies and runs anywhere. Heavy pieces are
optional extras: `introspect` (transformers/accelerate/safetensors),
`hardware` (pynvml), `torch`, `quant` (bitsandbytes).

## Quickstart

The whole public API is three functions. The recommended import alias is `fsc`
(`fs` is avoided as it is the import name of the PyFilesystem package). The flow
is: learn *what hardware* and *which model*, decide *how to train*, then turn
that into a config you can run.

```
inspect_environment() --.
                         |--> recommend() --> TrainingPlan --> export_config() --> runnable config
inspect_model(model) ---'
```

**1. Import**

```python
import fitscout as fsc
```

**2. Detect the hardware** -> returns an `Environment`

```python
env = fsc.inspect_environment()
# Environment(where='colab', gpu_name='NVIDIA T4', vram_gb=16.0,
#             bf16=False, compute_capability=7.5, num_gpus=1)
```

**3. Profile the model** (no weights downloaded, no model-name logic) -> `ModelProfile`

```python
model = fsc.inspect_model("Qwen/Qwen2.5-7B")
# ModelProfile(params=7615283200, num_layers=28, hidden_size=3584,
#              num_attention_heads=28, num_kv_heads=4, intermediate_size=18944,
#              vocab_size=152064, tie_word_embeddings=False, num_experts=1, dtype_bytes=2)
```

**4. Get a conservative plan** -> `TrainingPlan`

```python
plan = fsc.recommend(model, env, seq_len=1024)
# fits=True  method=qlora  precision=fp16
# micro_batch=2  grad_accum=8  grad_ckpt=True  opt_8bit=False
# oom_risk=MEDIUM   estimate=11.9 GiB (uncalibrated, +/-20%)
```

In words: *this 7B model fits a 16 GiB T4 with QLoRA + gradient checkpointing at
micro batch 2; estimated peak 11.9 GiB, medium risk.*

**5. Export a runnable config** -> dict (or a ready-to-run script)

```python
from fitscout.export import export_config
from fitscout.export.huggingface import render_script

cfg = export_config(plan, "Qwen/Qwen2.5-7B")   # TrainingArguments + LoRA + 4-bit
print(render_script(plan, "Qwen/Qwen2.5-7B"))   # a copy-paste training script
```

That is the whole loop: three public functions produce a plan, and one export
call turns it into something you can run. Export backends: `huggingface`,
`axolotl`, `deepspeed`.

For a readable summary in a notebook or terminal, print the plan as tables:

```python
print(fsc.format_plan(plan, env))
```

```
+-------------+---------------------------------+
| fitscout - training plan                      |
+-------------+---------------------------------+
| GPU         | Tesla T4 (16 GiB) [colab]       |
| Fits        | yes                             |
| Method      | qlora                           |
| Risk        | MEDIUM                          |
| Estimate    | 11.9 GiB (+/-20%, uncalibrated) |
+-------------+---------------------------------+
```

## Command line

```bash
fitscout Qwen/Qwen2.5-7B --vram 16 --seq-len 1024 --backend huggingface
```

```
GPU: CLI-specified (16 GiB)  where=cli
fits: True   risk: MEDIUM
method: qlora   precision: fp16
micro_batch: 2   grad_accum: 8   grad_ckpt: True   opt_8bit: False
seq_len: 1024
estimate: 11.9 GiB  (+/-20%, uncalibrated)
```

Omit `--vram` to auto-detect the GPU. Add `--calibrate` to use a cached
correction factor.

## Same model, different GPUs

A 7B model, `seq_len=1024` - fitscout adapts the plan to the card:

| GPU | VRAM | Plan |
|-----|------|------|
| H100 / A100-80 | 80 GiB | LoRA, micro batch 4 |
| A100 | 40 GiB | LoRA, micro batch 1 |
| RTX 4090 / L4 | 24 GiB | LoRA + gradient checkpointing |
| T4 (free Colab) | 16 GiB | QLoRA + gradient checkpointing |
| none | - | suggests the cheapest cloud GPU that fits |

## Calibration: estimate -> guarantee

The formula is conservative but uncalibrated estimates carry a +/-20% margin. To
tighten it, measure one real step on your GPU:

```python
from fitscout.memory.calibrate import calibrate

calibrate(model, "Qwen/Qwen2.5-7B")          # runs a tiny dry-run, caches the factor
plan = fsc.recommend(model, env, calibrate=True)   # now "calibrated", +/-10%
```

The correction is cached per `(gpu, framework)`, so it carries over to other
models on the same hardware.

## API reference

### Top level - `fitscout`

| Function | What it does |
|----------|--------------|
| `inspect_environment()` | Detect the runtime (Colab/AWS/Azure/local) and primary GPU (name, VRAM, bf16 support). Returns `Environment`. |
| `inspect_model(source, *, dtype_bytes=2, trust_remote_code=False)` | Profile a model's architecture **without downloading weights** (meta-device -> safetensors index -> analytical). Returns `ModelProfile`. |
| `recommend(model, env=None, seq_len=2048, calibrate=False)` | Pick a conservative full/LoRA/QLoRA plan that fits the hardware; never returns HIGH risk. Returns `TrainingPlan`. |
| `format_plan(plan, env=None)` | Render the plan and its memory breakdown as ASCII tables (same output as the CLI). Returns `str`. |

Data contracts: `Environment`, `ModelProfile`, `MemoryEstimate`
(`.upper_gb` gives the conservative upper bound), `TrainingPlan`.

### Memory - `fitscout.memory`

| Function | What it does |
|----------|--------------|
| `estimator.estimate_memory(profile, *, method, micro_batch, seq_len, grad_ckpt, opt_8bit, ...)` | Combine all components into a `MemoryEstimate` with confidence + margin. |
| `components.*_gb(...)` | The individual terms: `weights_gb`, `gradients_gb`, `optimizer_gb`, `block_activations_gb`, `output_activations_gb`, `dequant_buffer_gb`, `overhead_gb`. |
| `calibrate.calibrate(profile, model_ref, ...)` | Run one real step, derive `measured/estimated`, cache it per `(gpu, framework)`. |
| `calibrate.correction_factor / measure_peak_gb / CalibrationCache` | The calibration building blocks. |

### Model - `fitscout.model.introspect`

| Function | What it does |
|----------|--------------|
| `inspect_model(...)` | Same as the top-level function. |
| `extract_arch_fields(config)` | Read architectural fields from a config dict via candidate key names. |
| `analytical_param_count(fields)` | Estimate parameters from the config (last-resort fallback). |

### Environment - `fitscout.environment`

| Function | What it does |
|----------|--------------|
| `detect.detect_where(environ=None)` | Return `"colab" / "aws" / "azure" / "local"`. |
| `hardware.inspect_environment()` | Read the GPU via torch, then pynvml, else report no GPU. |

### Export - `fitscout.export`

| Function | What it does |
|----------|--------------|
| `export_config(plan, model_ref, backend="huggingface")` | Turn a plan into a backend config dict (`huggingface` / `axolotl` / `deepspeed`). |
| `available_exporters()` / `get_exporter(name)` / `register(exporter)` | Inspect and extend the exporter registry. |
| `huggingface.render_script(plan, model_ref)` | Emit a runnable training script. |
| `axolotl.render_yaml(plan, model_ref)` | Emit an Axolotl YAML config. |

### Cloud - `fitscout.cloud.catalog`

| Function | What it does |
|----------|--------------|
| `cheapest_fit(required_gb)` | Cheapest single catalog GPU that holds the requirement. |
| `rightsize(required_gb, allow_multi=True)` | Cheapest single- or multi-GPU suggestion. |

## Accuracy and honesty

fitscout is validated against real measured peak VRAM - see
[`validation/`](validation/). On a Colab T4 (QLoRA, 8-bit optimizer, gradient
checkpointing, seq_len 1024):

| Model | Estimate (GiB) | Measured (GiB) | Error |
|-------|----------------|----------------|-------|
| Qwen2.5-1.5B | 5.0 | 4.0 | +25% |
| Qwen2.5-3B | 6.0 | 5.1 | +18% |
| Qwen2.5-7B | 9.7 | 10.0 | -3% |

Estimates lean conservative (the positive direction is safe). The per-GPU
correction factor is *largely* model-independent but not perfectly so - on this
data it spans ~0.80-1.03 across model sizes - which is why a calibrated estimate
keeps a (narrower) margin rather than claiming an exact number.

## License

[MIT](LICENSE).
