# Fine-tuning a code-SLM from claw-forge traces with Unsloth

This is a **reference recipe**, not a hard requirement. Every piece is
swappable. The corpus that `claw-forge export training` produces is in OpenAI
tool-use chat format by default and works with any of: Unsloth, Axolotl, trl,
or vanilla Hugging Face `transformers.Trainer` with `apply_chat_template(tools=...)`.

## 1. Export the corpus

```bash
# Single file, default settings
claw-forge export training --out corpus.jsonl

# Or split for training-time eval
claw-forge export training --split 80/10/10 --out ./corpus
```

## 2. Pick a base model

| Model | VRAM (LoRA) | Why |
|---|---|---|
| `unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit` | 24 GB | **Recommended.** Coder-specialised, native tool-use chat template, Apache 2.0. |
| `unsloth/Qwen2.5-Coder-14B-Instruct-bnb-4bit` | 40 GB | Same series, more capacity. |
| `unsloth/DeepSeek-Coder-V2-Lite-Instruct-bnb-4bit` | 24 GB | Slightly stronger raw eval scores. MoE — finickier for LoRA. |

## 3. LoRA configuration

```python
from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit",
    max_seq_length=8192,
    dtype=None,            # autodetect
    load_in_4bit=True,
)

model = FastLanguageModel.get_peft_model(
    model,
    r=16, lora_alpha=32, lora_dropout=0.0,
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",
        "gate_proj", "up_proj", "down_proj",
    ],
    use_gradient_checkpointing="unsloth",
    random_state=3407,
)
```

## 4. Load the corpus

```python
from datasets import load_dataset

ds = load_dataset("json", data_files="corpus.jsonl", split="train")

def formatter(example):
    return {
        "text": tokenizer.apply_chat_template(
            example["messages"],
            tokenize=False,
            add_generation_prompt=False,
        ),
    }

ds = ds.map(formatter, remove_columns=ds.column_names)
```

The `thinking` field on assistant messages is silently dropped by
`apply_chat_template` — most templates don't have a slot for it. If you want
to train on chain-of-thought, export with `--format raw-jsonl` and write your
own template renderer.

## 5. Train

```python
from trl import SFTTrainer, SFTConfig

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=ds,
    args=SFTConfig(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=8,
        num_train_epochs=3,
        learning_rate=2e-4,
        warmup_ratio=0.03,
        lr_scheduler_type="cosine",
        optim="adamw_8bit",
        max_seq_length=8192,
        bf16=torch.cuda.is_bf16_supported(),
        logging_steps=10,
        output_dir="./out",
        report_to="none",
    ),
)
trainer.train()
```

For ~500 trajectories at average 6K tokens each, expect single-digit hours
on a 4090. Calibrate with a 50-task dry run before committing.

## 6. Eval

Two metrics:

1. **ROUGE-L on assistant content** — reasoning-quality proxy. Cheap, fast,
   weakly correlates with correctness.
2. **Diff similarity vs `final_diff`** — the real correctness signal.
   Reconstruct the model's proposed patch from its `tool_calls` (Edit / Write
   blocks) and score with line-level Jaccard plus
   `difflib.SequenceMatcher.ratio()` against the captured `final_diff`.

```python
import difflib

def diff_similarity(predicted_diff: str, ground_truth: str) -> float:
    pred_lines = set(predicted_diff.splitlines())
    truth_lines = set(ground_truth.splitlines())
    jaccard = len(pred_lines & truth_lines) / max(1, len(pred_lines | truth_lines))
    seq = difflib.SequenceMatcher(None, predicted_diff, ground_truth).ratio()
    return 0.5 * jaccard + 0.5 * seq
```

## 7. Phase 2 — DPO / rejection sampling (out of scope here)

Once the SFT model is reasonable, the captured `final_diff` is the reward
function. Generate K candidates per prompt, score by diff similarity, build
preference pairs, run a DPO step. The export schema is forward-compatible:
re-run `claw-forge export training --include-failed` to add explicit negatives.

## Anthropic Usage Policies

This pipeline is intended for **personal/internal distillation** — building a
smaller model that imitates *your own* Claude usage on *your own* code, for
*your own* internal use. Anthropic's Usage Policies prohibit using Claude
outputs to develop models that compete with Anthropic's services. Distribution
of derived models is your responsibility.
