Metadata-Version: 2.4
Name: grillydistil
Version: 0.1.0
Summary: Temperature-scaled distillation with SA-KD — optional grilly extension
Author-email: Nicolas Cloutier <ncloutier@grillcheeseai.com>
License: MIT
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: grilly>=0.4.0
Requires-Dist: grillyinference>=0.1.0
Requires-Dist: numpy
Provides-Extra: full
Requires-Dist: transformers; extra == "full"
Requires-Dist: huggingface_hub; extra == "full"
Requires-Dist: grillycompression>=0.1.0; extra == "full"
Dynamic: license-file

# GrillyDistil

Temperature-scaled knowledge distillation with SA-KD — optional [grilly](https://github.com/grillcheese/grilly) extension.

## Features

- **SA-KD Temperature** — simulated annealing-based adaptive temperature
- **Linear Annealing** — fallback T=8 -> T=2 over first 30% of steps
- **Expanded Prompts** — 50 seed prompts per domain (200 total seeds)
- **Distillation Trainer** — full training loop with KL-divergence loss
- **Compression Synergy** — high-T phases produce 2-3x better compression ratios

## Quick Start

```bash
pip install grillydistil
```

```python
from grillydistil import SAKDTemperature, DistillationTrainer, PromptGenerator

# SA-KD Temperature
temp = SAKDTemperature(T_init=8.0, alpha=0.97)
for step in range(1000):
    loss = train_step(temperature=temp.current_temperature)
    new_T = temp.step(loss)

# Generate training prompts
gen = PromptGenerator(prompts_per_domain=500)
prompts = gen.generate()  # 2000 total prompts across 4 domains

# Full distillation
trainer = DistillationTrainer(student_model, teacher_model, tokenizer)
losses = trainer.train(dataset, epochs=3)
```

## SA-KD Algorithm

1. Start with T_init=8.0 (high temperature -> smooth teacher logits)
2. Propose T' = T + random perturbation
3. Metropolis acceptance: P = min(1, exp(-dE / T_SA))
4. SA cooling: T_SA *= 0.97 each step
5. Converges to optimal temperature for student capacity

## Requirements

- Python 3.12+
- grilly >= 0.4.0
- grillyinference >= 0.1.0
- numpy

## License

MIT
