Metadata-Version: 2.4
Name: slmify
Version: 0.2.0
Summary: Point it at your LLM logs. See exactly where you're overpaying — fine-tune a small model, cache, or downgrade.
Project-URL: Homepage, https://github.com/small-lm/slmify
Project-URL: Repository, https://github.com/small-lm/slmify
Project-URL: Issues, https://github.com/small-lm/slmify/issues
Project-URL: Changelog, https://github.com/small-lm/slmify/blob/main/CHANGELOG.md
Author-email: SmaLLM <daniel@small-lm.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: clustering,cost,fine-tuning,llm,rightsizing,slm
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: drain3>=0.9.11
Requires-Dist: httpx>=0.27
Requires-Dist: model2vec>=0.3
Requires-Dist: numpy>=1.26
Requires-Dist: polars>=1.0
Requires-Dist: pydantic>=2.7
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Requires-Dist: scikit-learn>=1.3
Requires-Dist: typer>=0.12
Provides-Extra: all
Requires-Dist: peft>=0.11; extra == 'all'
Requires-Dist: sentence-transformers>=3.0; extra == 'all'
Requires-Dist: trl>=0.9; extra == 'all'
Requires-Dist: unsloth; extra == 'all'
Provides-Extra: st
Requires-Dist: sentence-transformers>=3.0; extra == 'st'
Provides-Extra: train
Requires-Dist: peft>=0.11; extra == 'train'
Requires-Dist: trl>=0.9; extra == 'train'
Requires-Dist: unsloth; extra == 'train'
Description-Content-Type: text/markdown

# slmify

[![CI](https://github.com/small-lm/slmify/actions/workflows/ci.yml/badge.svg)](https://github.com/small-lm/slmify/actions/workflows/ci.yml)
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](pyproject.toml)

> Point it at your LLM logs. See exactly where you're overpaying — and what to fix: fine-tune a small model, cache, or downgrade.

**LLM workload rightsizing**, as a local-first CLI. slmify ingests your production LLM logs or agent traces, groups them into workloads (workflow step → prompt template → semantic cluster), scores each workload's **SLM-fit** on a banded two-axis system (feasibility × dollar opportunity), computes **honest, range-based savings** with break-even math, recommends the cheapest safe action per workload — and exports training-ready datasets plus configs for whichever fine-tuning stack you already use.

NVIDIA's research estimates [40–70% of LLM calls in agentic systems are SLM-replaceable](https://arxiv.org/abs/2506.02153) — but sketched the conversion pipeline without implementing it. slmify measures that number on *your* traffic.

**Status: v0.2 (pre-release).** Analyzer, zero-training probe, HTML report, dataset exports, and the local trainer are all in.

## Principles (non-negotiable)

- **Local-first, zero telemetry, zero required API keys.** Your prompts never leave your machine. The default embedding model is a 30 MB static model; the tool runs offline.
- **No false precision, ever.** Bands not decimals; ranges not points; every assumption printed. Expect frequent, honest **KEEP** recommendations — a tool that tells you *not* to fine-tune 7 of 10 clusters is one you can trust.
- **Evidence over heuristics.** Every score comes with the exact metrics behind it; every formula is documented and user-overridable via `--policies`.
- **Neutral.** Reads every vendor's logs, exports to every trainer. No login, no gates, no nags.

## Two zero-friction first runs

```bash
uvx slmify              # no arguments: analyzes your local Claude Code agent
                        # traces (~/.claude/projects) — a personal verdict from
                        # data already on your laptop

uvx slmify demo         # bundled synthetic corpus + precomputed embeddings:
                        # a full report in ~30s, zero network, works on a plane
```

Then the real thing:

```bash
uvx slmify analyze ./logs.jsonl --html report.html   # format auto-detected
uvx slmify analyze ./logs.csv --map "prompt=question,response=answer,model=model_name"

# prove feasibility before spending anything: zero training, k-shot, your own endpoint
slmify probe --clusters top3 --endpoint http://localhost:11434/v1 --model qwen3:4b-instruct

slmify export c07        # train/eval JSONL + Unsloth/Axolotl/TRL/OpenAI/vLLM-router stubs
slmify train c07         # [train] extra: LoRA on the same recipe (CUDA/Unsloth)
slmify eval c07 --endpoint http://localhost:8000/v1 --model tuned --baseline-model base
slmify pricing --update  # refresh the price snapshot (only network call you didn't configure)
```

Supported formats: **Claude Code traces** (`~/.claude/projects`), **Codex CLI** (`~/.codex/sessions`) and **Gemini CLI** (`~/.gemini/tmp`) sessions, **Langfuse observation exports**, **OpenAI-style JSONL** (fine-tuning chat format, chat-completion objects, request/response pairs), and **generic JSONL/CSV** with dotted-path `--map` — which covers most homegrown logging tables and gateway DB exports. Running LiteLLM? `slmify.integrations.litellm_logger.SlmifyLogger` is a drop-in callback that logs complete request/response pairs straight into slmify's native format (SpendLogs don't store content). OTel GenAI lands in v0.3. Adapters are pluggable via the `slmify.adapters` entry-point group.

## What the report gives you

For every discovered workload:

- **SLM-fit band** (A "Prime" → D "Poor fit") from 10 published, documented metrics — output contraction, schema stability, task-type priors, template mass, semantic cohesion, reasoning depth, and more — with the top-3 evidence bullets and a confidence tag. Raw numbers live in `analysis.json`, not the headline.
- **Probe evidence on demand**: `slmify probe` runs a candidate SLM against 30 held-out samples per cluster (k-shot, zero training) through any OpenAI-compatible endpoint and adjusts the band ±1 step. Graders run cheapest-first (exact → JSON-schema → embedding similarity); an LLM judge only runs if you supply a judge endpoint, and flat judge scores automatically switch to ranking mode.
- **Dedicated-GPU break-even math**: the report tells you the sustained calls/hour where a dedicated L4/A10G/A100 beats your current spend — and says "use serverless" when you're below it.
- **Measured spend and savings ranges** — `{low, point, high}` with the assumptions listed, priced from a vendored [LiteLLM pricing snapshot](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json), monthly volumes extrapolated with a bootstrap CI over your observed days. A 3-day log honestly yields wide bars.
- **An action stack**: `FINE_TUNE_SLM`, `DOWNGRADE_TRIAL`, `EXACT_CACHE`, `SEMANTIC_CACHE`, `PROMPT_CACHE / TRIM`, `FIX_QUALITY_FIRST` (with a named remedy), or an honest `KEEP` — decided by auditable ordered rules you can override.
- **Cache economics as floor / realistic / ceiling** — exact-duplicate rate is the zero-risk floor, semantic near-duplicate mass the ceiling.
- Multi-turn conversations in flat logs are **reconstructed into episodes** (conservative prefix matching), so volumes, duplicate rates, and dataset exports don't double-count growing message arrays.

`slmify export <cluster>` writes `train.jsonl` / `eval.jsonl` (OpenAI chat format — accepted by OpenAI FT, Unsloth, Axolotl, and TRL) plus ready-to-run config stubs. The eval set is a stratified holdout that is **never** included in training data, so you can verify the fine-tune on untainted samples.

## How slmify compares

| | slmify | OpenAI Distillation | Braintrust / LangSmith | Kiln |
|---|---|---|---|---|
| Finds *what* to fix in your traffic | ✅ scored workloads | — | clusters, no scoring/savings | task-first |
| Cross-vendor (reads any logs) | ✅ | OpenAI only | their platform | n/a |
| Runs locally, data never leaves | ✅ | — | — | ✅ |
| Proves SLM feasibility pre-training | ✅ probe | — | — | — |
| $ savings with break-even math | ✅ ranges | — | — | — |
| Exports to any trainer | ✅ | — | partial | ✅ |

*As of 2026-07. These tools solve adjacent problems — several pair well with slmify rather than replacing it.*

## Install

```bash
uv tool install slmify        # or: pipx install slmify / pip install slmify
```

Core install is light: polars, numpy, scikit-learn, model2vec, drain3, pydantic, typer, rich, httpx, pyyaml. No torch, no database, no compiled clustering libs. Extras: `[st]` (sentence-transformers embedders), `[train]` (local LoRA fine-tuning, v0.2).

Offline degradation ladder: `demo` needs zero network → the first real analysis downloads the 30 MB default embedder once (then cached) → fully offline, use `--embedder hash` (TF-IDF, already installed) — degraded clusters, still useful, never a dead end.

## Privacy posture

Report exemplars are masked by default (`<EMAIL>`, `<NUM>`, `<ID>`, …) via the same template miner that names your clusters; `--show-raw` disables. Dataset exports contain raw text (training needs real data) and say so loudly. There is no telemetry of any kind.

## Benchmarks

`benchmarks/` holds reproducible runs: a ground-truth validation of the whole
grouping pipeline on the bundled labeled corpus (purity 1.00, V-measure 0.94),
and a one-command public-data benchmark (WildChat-1M / LMSYS-Chat-1M) that
measures the SLM-able share of real traffic against NVIDIA's 40–70% agentic
estimate.

## Roadmap

- **v0.3** — the standard: `--watch`/CI diff mode, OTel GenAI adapter, in-process adapter eval, an open calibration study on public data, plugin authoring docs, `--backend mlx` (Apple Silicon training).

## License

Apache-2.0.
