Metadata-Version: 2.4
Name: slicktune
Version: 0.3.0
Summary: SlickML fine-tuning toolkit: composable strategies, objectives, and metrics for LLMs
Project-URL: Homepage, https://www.slickml.com
Project-URL: Repository, https://github.com/slickml/slick-tune
Project-URL: Documentation, https://github.com/slickml/slick-tune/tree/master/docs
Author-email: Amirhessam Tahmassebi <admin@slickml.com>
License: MIT
License-File: LICENSE
Keywords: fine-tuning,llm,lora,machine-learning,peft,qlora,sft
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: <3.14,>=3.10
Requires-Dist: accelerate>=0.34
Requires-Dist: click>=8.1
Requires-Dist: datasets>=3.0
Requires-Dist: peft>=0.12
Requires-Dist: rich>=13.0
Requires-Dist: sentencepiece>=0.2
Requires-Dist: torch>=2.2
Requires-Dist: transformers>=4.44
Requires-Dist: trl>=0.12
Provides-Extra: dev
Requires-Dist: assertpy>=1.1; extra == 'dev'
Requires-Dist: mypy>=2.3.0; extra == 'dev'
Requires-Dist: poethepoet>=0.32; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.3; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Requires-Dist: tox-uv>=1.25; extra == 'dev'
Requires-Dist: tox>=4.22; extra == 'dev'
Provides-Extra: docs
Requires-Dist: furo>=2022.9; extra == 'docs'
Requires-Dist: myst-parser>=0.18; extra == 'docs'
Requires-Dist: sphinx-autoapi>=1.9; extra == 'docs'
Requires-Dist: sphinx-design>=0.3; extra == 'docs'
Requires-Dist: sphinx>=5.3; extra == 'docs'
Requires-Dist: sphinxcontrib-mermaid>=0.9; extra == 'docs'
Provides-Extra: qlora
Requires-Dist: bitsandbytes>=0.43; extra == 'qlora'
Description-Content-Type: text/markdown

<div align="center">

[![build](https://github.com/slickml/slick-tune/actions/workflows/ci.yml/badge.svg)](https://github.com/slickml/slick-tune/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/slickml/slick-tune/graph/badge.svg)](https://codecov.io/gh/slickml/slick-tune)
[![downloads](https://pepy.tech/badge/slicktune)](https://pepy.tech/project/slicktune)
[![license](https://img.shields.io/github/license/slickml/slick-tune)](https://github.com/slickml/slick-tune/blob/master/LICENSE)
![pypi_version](https://img.shields.io/pypi/v/slicktune)
![python_version](https://img.shields.io/pypi/pyversions/slicktune)
[![slack_invite](https://badgen.net/badge/Join/SlickML%20Slack/purple?icon=slack)](https://www.slickml.com/slack-invite)
![twitter_url](https://img.shields.io/twitter/url?style=social&url=https%3A%2F%2Ftwitter.com%2FSlickML)

</div>

<p align="center">
  <a href="https://github.com/slickml/slick-tune">
    <img src="https://raw.githubusercontent.com/slickml/slick-tune/master/assets/design/logo.png" alt="slick-tune logo" width="420"/>
  </a>
</p>

<div align="center">
<h1 align="center">SlickTune 🧩: Composable LLM Fine-Tuning by SlickML🧞</h1>
  <p align="center">
    <a href="https://github.com/slickml/slick-tune/releases">Explore Releases</a>
    🟣
    <a href="https://github.com/slickml/slick-tune/blob/master/CONTRIBUTING.md">Become a Contributor</a>
    🟣
    <a href="https://pypi.org/project/slicktune/">PyPI</a>
    🟣
    <a href="https://www.slickml.com/slack-invite">Join our Slack</a>
    🟣
    <a href="https://twitter.com/slickml">Tweet Us</a>
  </p>
</div>


## 🧠 Philosophy

**SlickTune 🧩** is a small, composable toolkit for teaching LLMs new facts and behaviors with
[Transformers](https://huggingface.co/docs/transformers) + [PEFT](https://huggingface.co/docs/peft) + [TRL](https://huggingface.co/docs/trl).
LoRA / QLoRA are PEFT adapters; full FT updates every weight. The goal is the same SlickML spirit:
prototype fast 🏎, keep axes orthogonal, and measure whether the model actually learned *your* facts 🔎.

> **New to fine-tuning?** Start here →
> **[Fine-Tuning LLMs: A Visual Guide](docs/pages/fine_tuning_guide.md)** —
> pre-training vs prompting vs FT, Full / LoRA / DoRA / AdaLoRA / QLoRA with diagrams,
> how to choose a strategy, and how probes & holdout perplexity tell you it worked.
>


Fine-tuning is an orthogonal stack — swap any axis without rewriting the others:

```text
model  ×  strategy  ×  objective  ×  data  ×  metrics
```


## 🧩 Abstractions

```mermaid
flowchart TB
  subgraph inputs [Inputs]
    modelId[model_id]
    dataJsonl[data JSONL]
  end

  subgraph axes [Composable axes]
    strategyNode["Strategy: LoRA / DoRA / AdaLoRA / QLoRA / Full"]
    objectiveNode["Objective: SFT then DPO / GRPO"]
  end

  subgraph core [Tuner fit]
    tuner[Tuner]
    loadStep[load model and tokenizer]
    applyStep[strategy.apply]
    trainStep[TRL trainer]
    metricsStep[MetricsTracker]
  end

  subgraph outputs [Outputs]
    checkpoint[adapter or checkpoint]
    metricsFile[metrics.json]
    probeRate[probe pass rate]
  end

  modelId --> tuner
  dataJsonl --> tuner
  strategyNode --> tuner
  objectiveNode --> tuner
  tuner --> loadStep --> applyStep --> trainStep --> metricsStep
  trainStep --> checkpoint
  metricsStep --> metricsFile
  checkpoint --> probeRate
```

| Axis          | Responsibility                    | Phase 3                                                         |
| ------------- | --------------------------------- | --------------------------------------------------------------- |
| **Strategy**  | How weights change (PEFT vs full) | `LoRA` / `DoRA` / `AdaLoRA` / `QLoRA` / `Full`                  |
| **Objective** | What is optimized / data contract | `SFT` / `DPO` / `ORPO` / `KTO`                                  |
| **Data**      | Examples → chat or preferences    | train + holdout + prefs/KTO JSONL (`about_amir*.jsonl`)         |
| **Metrics**   | Comparable run stats              | `MetricsTracker` (+ holdout PPL, judge score)                   |
| **Eval**      | Holdout + judges                  | `slicktune eval`, `SubstringJudge`, `LLMJudge`                  |
| **Probe**     | Did the model learn *your* facts? | `slicktune probe`                                               |

## 📌 Quick Start

```python
from slicktune import LoRAStrategy, SFTObjective, Tuner

Tuner(
    model_id="HuggingFaceTB/SmolLM2-135M-Instruct",
    strategy=LoRAStrategy(r=8),
    objective=SFTObjective(),
    output_dir="outputs/sft_lora",
    eval_data="examples/data/about_amir.eval.jsonl",
).fit("examples/data/about_amir.jsonl")
```

### 👤 Personal “about me” loop (recommended)

1. Edit `examples/data/about_amir.jsonl` with facts about you (or keep the SlickML starter facts) ✍️.
2. Edit `examples/data/about_amir.eval.jsonl` with **held-out** paraphrases (same topics, not copied from train) for perplexity 📉.
3. Edit `examples/data/about_amir.probes.jsonl` with questions and a `must_contain` substring that should appear after training 🎯.
4. Train a strategy on a **tiny** instruct model 🧪.
5. Probe the checkpoint — pass rate shows whether fine-tuning stuck ✅.

```text
before FT  →  model guesses / hallucinates about you
after FT   →  probe answers contain your facts
```

## 🛠 Installation

Install [Python >=3.10,<3.14](https://www.python.org) and [*uv*](https://docs.astral.sh/uv/), then simply run 🏃‍♀️:

```bash
uv sync --locked --all-extras --all-groups
```

QLoRA (CUDA + bitsandbytes only) 🔥:

```bash
uv sync --extra qlora
```

Task runner is [Poe the Poet](https://poethepoet.natn.io/installation.html) (same idea as [slick-ml](https://github.com/slickml/slick-ml), with `uv` instead of Poetry). Install the CLI once 🏃‍♀️:

```bash
uv tool install poethepoet
poe greet
```

Developer workflow (`format` / `check` / `test`) lives in [CONTRIBUTING.md](CONTRIBUTING.md) 🧑‍💻🤝.

## 🚂 Train each strategy

Default demo model: `HuggingFaceTB/SmolLM2-135M-Instruct` (small enough for laptop smoke tests) 💻.

### 🟢 LoRA + SFT (default — works on Mac MPS / CPU / CUDA)

```bash
uv run slicktune train \
  --strategy lora \
  --data examples/data/about_amir.jsonl \
  --eval-data examples/data/about_amir.eval.jsonl \
  --output outputs/sft_lora \
  --epochs 20

uv run slicktune probe \
  --model-dir outputs/sft_lora \
  --probes examples/data/about_amir.probes.jsonl
```

Or: `poe train-lora` / `poe probe-lora` / `poe eval-lora` / `uv run python examples/run_sft_lora.py`

### 🟣 DoRA + SFT

```bash
uv run slicktune train \
  --strategy dora \
  --data examples/data/about_amir.jsonl \
  --eval-data examples/data/about_amir.eval.jsonl \
  --output outputs/sft_dora
# or: uv run python examples/run_sft_dora.py
```

### 🟤 AdaLoRA + SFT

```bash
uv run slicktune train \
  --strategy adalora \
  --data examples/data/about_amir.jsonl \
  --eval-data examples/data/about_amir.eval.jsonl \
  --output outputs/sft_adalora
# or: uv run python examples/run_sft_adalora.py
```

### 🟡 LoRA + DPO (preference pairs)

```bash
uv run slicktune train \
  --strategy lora \
  --objective dpo \
  --data examples/data/about_amir.prefs.jsonl \
  --output outputs/dpo_lora \
  --epochs 3

# or: poe train-dpo / uv run python examples/run_dpo_lora.py
```

### 🟢 LoRA + KTO (unpaired labels)

```bash
uv run slicktune train \
  --strategy lora \
  --objective kto \
  --data examples/data/about_amir.kto.jsonl \
  --output outputs/kto_lora \
  --epochs 3

# or: poe train-kto / uv run python examples/run_kto_lora.py
```

ORPO: `--objective orpo` with the same prefs JSONL as DPO (TRL experimental).

### 🔎 Eval harness (holdout PPL + judges)

```bash
uv run slicktune eval \
  --model-dir outputs/sft_lora \
  --eval-data examples/data/about_amir.eval.jsonl \
  --probes examples/data/about_amir.probes.jsonl \
  --judge substring
```

`--eval-data` should be a **holdout** SFT JSONL (not the training file). The shipped
`about_amir.eval.jsonl` paraphrases the same topics for holdout perplexity.

Use `--judge llm` to score generations with an LLM rubric (0–10 → normalized).
On the tiny demo model, prefer `--judge substring`: the same 135M checkpoint is a
weak judge and will under-score even when answers are correct.

### 🔵 QLoRA + SFT (CUDA required)

```bash
uv sync --extra qlora
uv run python examples/run_sft_qlora.py
```

On Apple Silicon, use LoRA instead — bitsandbytes 4-bit needs CUDA 🍎.

### 🟠 Full fine-tuning + SFT

```bash
uv run python examples/run_sft_full.py
```

Heavier on memory; prefer LoRA for iteration 💾.

## 📦 Data formats

**SFT JSONL** (any of these per line) 📝:

```json
{"messages":[{"role":"user","content":"..."},{"role":"assistant","content":"..."}]}
{"prompt":"...","response":"..."}
{"instruction":"...","input":"...","output":"..."}
```

**Probe JSONL** 🕵️:

```json
{"prompt":"Who is Amirhessam Tahmassebi?","must_contain":"SlickML"}
```

**Holdout eval JSONL** (same SFT shapes as train; keep examples out of the train file) 📉:

```json
{"messages":[{"role":"user","content":"..."},{"role":"assistant","content":"..."}]}
```

Ship example: `examples/data/about_amir.eval.jsonl`.

**Preference JSONL** (DPO / ORPO) ⚖️:

```json
{"prompt":"...","chosen":"...","rejected":"..."}
```

**KTO JSONL** (unpaired labels) ✅❌:

```json
{"prompt":"...","completion":"...","label":true}
```

## 🗺 Roadmap

| Phase   | Scope                                                         |
| ------- | ------------------------------------------------------------- |
| 0–1     | Skeleton, SFT + LoRA/QLoRA/full, metrics, personal probe loop |
| 2 (done)| DoRA / AdaLoRA, holdout PPL + substring/LLM judges            |
| 3 (now) | DPO / ORPO / KTO                                              |
| 4       | GRPO / verifiable RL                                          |
| 5       | Merge (TIES/DARE), multi-adapter                              |
| 6       | Optional PPO / multimodal                                     |

## 🧑‍💻🤝 Contributing to SlickTune 🧩

You can find the details of the development process in our [Contributing](CONTRIBUTING.md) guidelines.
We strongly believe that reading and following these guidelines will help us make the contribution process easy and effective for everyone involved 🚀🌙.

Special thanks to all of our amazing contributors 👇

<a href="https://github.com/slickml/slick-tune/graphs/contributors">
  <img src="https://contrib.rocks/image?repo=slickml/slick-tune" />
</a>

![Repobeats analytics image](https://repobeats.axiom.co/api/embed/5205f02e274ac9df0d3b8fe80be684139ac0c878.svg "Repobeats analytics image")


## ❓ 🆘 📲 Need Help?

Please join our [Slack Channel](https://www.slickml.com/slack-invite) to interact directly with the core team and our small community. This is a good place to discuss your questions and ideas or in general ask for help 👨‍👩‍👧 👫 👨‍👩‍👦.
