Metadata-Version: 2.5
Name: pichak
Version: 0.1.0
Summary: Derive fine-tuning hyperparameters from measurements of your actual machine, and print the measurement behind every number.
Project-URL: Homepage, https://github.com/olesxg/pichak
Project-URL: Findings, https://github.com/olesxg/flap-findings
Author-email: Oleksandr Pichak <stasselust@gmail.com>
License: MIT
License-File: LICENSE
Keywords: batch-size,benchmark,fine-tuning,gpu,hyperparameters,learning-rate,llm,lora,vram
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == 'dev'
Provides-Extra: hf
Requires-Dist: safetensors>=0.4; extra == 'hf'
Requires-Dist: torch>=2.0; extra == 'hf'
Requires-Dist: transformers>=4.40; extra == 'hf'
Provides-Extra: torch
Requires-Dist: torch>=2.0; extra == 'torch'
Description-Content-Type: text/markdown

# pichak

**Fine-tuning hyperparameters derived from your machine, with the measurement
printed next to every number.**

```bash
pip install pichak
```

```python
from pichak import derive

plan = derive(
    data="train.jsonl",
    tokenizer=tok,
    step_fn=lambda b: one_real_forward_and_backward(b),
    named_weights=model.named_parameters(),
)
print(plan.report())
```

```
  seq_len            512
                     token p95 416 over 512 real rows, rounded up to 512; p99 is
                     657 and max 943, so 5% of rows lose their tail at this length

  micro_batch        3
                     a doubling ramp with a real forward+backward at each rung:
                     1=ok, 2=ok, 4=ok, 8=SPILL. 8 SPILLED to system memory — it did
                     not raise, it got 1.6x slower per sample, which is the Windows
                     failure mode, so 4 is the largest that fit, peaking at 5.05GB
                     of 5.05GB; kept 3 to leave 12% for optimizer state

  learning_rate      0.000282
                     1/32 of the scale at which one Adam step would rewrite rather
                     than perturb. ||W||_F/sqrt(numel) over 26 matrices; the
                     smallest binds and it is b.1.f.2.weight at 0.00902

  NOT MEASURED (1): target_tokens=65536
```

---

## Why not just use a good default

Because you cannot tell a good default from a bad one when it fails.

`micro_batch=4` tells you nothing about whether 4 came from a measurement on your
card, a heuristic in a blog post, or a number someone picked in 2023 for a
different model. So when it OOMs at step 40 you bisect instead of reading.

Every number `pichak` returns carries the sentence that produced it. Anything it
could **not** measure is collected under `plan.constants()` and printed together,
so a constant can never quietly pass for a derivation.

```python
plan.micro_batch          # 3
plan.why("micro_batch")   # the ramp, rung by rung, and which one failed
plan.constants()          # {'target_tokens': 65536}
```

## It catches the failure that does not raise

On Windows, a batch that exceeds VRAM does not throw. The driver pages the excess
to system memory and the step simply crawls. Measured on a GTX 1060 6GB with a
6-layer model at sequence 512:

| | micro_batch | peak | seconds/step |
|---|---|---|---|
| watching only for OOM | 14 | **18.24GB** on a 6GB card | 47.5 |
| watching per-sample time too | 3 | 5.05GB | **2.0** |

Both "work". The first is 23x slower and nothing in the logs says why. On a healthy
ramp, seconds-per-sample is flat or improves as the batch grows — bigger batches
amortise fixed costs. A sudden jump means the batch did not really fit.

## What it derives, and from what

| | measurement |
|---|---|
| `seq_len` | tokenised p95 over 512 real rows of your corpus |
| `loss_policy` | whether the rows have a separable completion to mask against |
| `micro_batch` | a doubling ramp with a real forward+backward at each rung |
| `seconds_per_step` | the wall time of that rung |
| `grad_accum` | micro-batch x seq_len against your token target |
| `lora_rank` | largest rung whose optimizer state fits — fp32 master, two Adam moments, and one step's gradients, which is the copy count people forget |
| `learning_rate` | the model's own Frobenius norms: the step size at which one Adam update would rewrite a weight rather than perturb it |

Everything is optional. Pass only a corpus and a tokenizer and you get a sequence
length; pass a `step_fn` and you get the batch. It will not invent a number it
could not measure — a plan with three derived values and an honest gap is more
useful than one with ten you cannot tell apart.

## The learning rate, since it is the surprising one

An Adam update moves a parameter by roughly `lr`, whatever the gradient's scale. So
there is a learning rate at which one step rewrites a weight matrix instead of
nudging it: where `lr * sqrt(numel(W))` reaches `||W||_F`. Every sane learning rate
is a fraction of that, and the fraction is what schedules argue about.

Measured, at the ranks behind this library:

```
rewrite/2    2.26e-3   destroyed the model twice (CE 1.74 -> 10.4, 1.11 -> 19.3)
rewrite/32   1.41e-4   the first scale that actually learned
rewrite/45             where a hand-tuned LoRA run at 2e-4 landed
```

`pichak` opens at `rewrite/32` — the top of the range that works. Opening hot costs
the model; opening cold costs time. With a penalty that asymmetric you start at the
bottom of what is known.

This is not tuned for your dataset. It is the scale at which updates are the right
*size* for these weights. It replaces "I copied 2e-4 from a post about a different
model", which is the actual alternative.

## Measurement tools, usable on their own

```bash
pichak gpu                    # virtualisation, launch latency, transfer bandwidth
pichak disk "models/*.safetensors"   # queue-depth sweep, page cache bypassed
pichak corpus train.jsonl mistralai/Mistral-Small-24B-Instruct-2501
```

```python
from pichak.measure import disk_queue_depth, transfer_bandwidth, virtualisation
```

These exist because the numbers people quote are rarely the numbers their machine
gives:

- A buffered disk benchmark reported **4079 MB/s on a drive rated 2100** — that was
  the page cache. Unbuffered, the same drive peaks at **queue depth 3** and gets
  *slower* past it.
- A rented A6000 measured **D2H 457 MB/s against H2D 1502**, and pinned memory —
  the standard fix — bought nothing. Nothing about PCIe explains that; it was a
  fabric, and it decided where hidden states could live.
- One rented machine had **84.8us kernel launches** against a normal 2-5. A
  workload issuing 400,000 launches per step would have spent nine hours on
  overhead and looked slower than a 2016 card.

## Install

```bash
pip install pichak            # the plan, the disk sweep — no dependencies
pip install pichak[torch]     # the ramp, the learning rate, the GPU measurements
pip install pichak[hf]        # + transformers, for the corpus CLI
```

Python 3.9+. The core has **no dependencies at all**; torch is only needed for the
parts that touch a GPU.

## Where the numbers come from

Every measurement quoted here was made while fine-tuning a 24B model on a 6GB
GTX 1060 and on a rented A6000 held down to the same 6GB. The raw logs — including
the six runs that failed before one worked — are at
[flap-findings](https://github.com/olesxg/flap-findings), along with the five
hypotheses that sounded right and were wrong.

## Licence

MIT. By [Oleksandr Pichak](https://github.com/olesxg).
