🏎️ Dyno — llama.cpp Auto-Tuner & Benchmark

Open-source CLI that auto-tunes and benchmarks llama.cpp / ik_llama.cpp inference on NVIDIA GPUs. Find the fastest config in seconds.

Get Started View on GitHub

⚡ Quickstart

Install Dyno and tune your first model in under a minute.

Install

pipx install llama-dyno

Detect hardware

dyno detect

Fingerprints your GPU, VRAM, driver version, CPU, RAM, and detects which llama.cpp backend is installed.

Auto-tune a model

dyno tune ~/models/mistral-7b.Q4_K_M.gguf --quick

Finds the fastest config — ngl, flash attention, KV cache quant, batch size, threads — with live progress.

Benchmark

dyno bench ~/models/mistral-7b.Q4_K_M.gguf --ngl 99 --fa

Generate a report

dyno report ~/models/mistral-7b.Q4_K_M.gguf

📖 Commands

Every command Dyno offers.

dyno detect
Fingerprint hardware — GPU model, VRAM, driver/CUDA version, CPU, RAM, and detect which llama.cpp backend is installed (with commit hash).
dyno tune <model>
Find the fastest config for your GPU + model combo. Uses a smart search: validate → coarse sweep (ngl, flash attn, KV cache) → hill-climb (batch size, threads). Supports --quick (~10 trials) and --thorough (~25 trials) modes.
dyno bench <model>
Run a specific config 3× and report median tokens/sec with variance. Flags: --ngl, --fa/--no-fa, --ctk, --ctv, --batch, --ubatch, --threads, --fmoe, --rtr, --amb.
dyno report <model>
Generate a shareable report with hardware fingerprint, model details, winning config, median scores with variance, reproducible llama-server command, and shareable markdown snippet. Output as JSON.
dyno submit <model>
Submit your results to the community results repo. Opens a PR via gh CLI, falls back to a GitHub Gist, or saves locally.
dyno search [gpu] [model]
Search local result JSON files for matching GPU and/or model keywords. Filters by case-insensitive substring match across all saved reports.
dyno compare [gpu] [model]
Compare multiple benchmark results side-by-side. Filter by GPU type and model name to see how different configs stack up.

🔬 Example Output

Dyno's Rich-powered terminal output.

System Detection

┌──────────────────────────────────────────┐
│ System Detection                        │
├──────────────────────────────────────────┤
│ Component       Value                   │
├──────────────────────────────────────────┤
│ GPU      NVIDIA GeForce RTX 4090        │
│ VRAM     24564 MiB                      │
│ Driver   545.23.08                      │
│ CUDA     12.3                          │
│ CPU      AMD Ryzen 9 7950X              │
│ Cores    16                             │
│ RAM      64228 MiB                      │
│ Backend  llama.cpp                      │
│ Commit   b1c2d3e                        │
└──────────────────────────────────────────┘

Benchmark Results

┌──────────────────────────────────────────┐
│ Benchmark Results                      │
├──────────────────────────────────────────┤
│ Metric                    Median  Var   │
├──────────────────────────────────────────┤
│ PP (prompt processing)   520.3   1.2    │
│ TG (text generation)     58.7    0.4    │
└──────────────────────────────────────────┘

✓ Generation: 58.7 tok/s | Prompt: 520.3 tok/s

Winning Config (Post-Tune)

╭──────────────────────────────────────╮
│ 🏆 Best Config                      │
├──────────────────────────────────────┤
│ ngl=99, fa=✓, ctk=f16, ctv=f16       │
│ batch=1024, ubatch=512, threads=0    │
╰──────────────────────────────────────╯