Weave QuickDistill

Quick Start:

1. Sync traces: Fetch a Weave project to import your production LLM calls
2. Select traces: Choose the traces you want to use as your ground-truth "strong model" examples
3. Run End-to-End Test: Click the button to automatically run cheaper models on your selected traces and evaluate them with an LLM judge
4. Mine Hard Examples: Find where weak models struggle most and export those examples to build better prompts

Run Weak Model Inference

Run Evaluation

⛏ Mine Hard Examples

Score all examples with a judge, visualize the distribution, then pick a threshold to export the hard ones.

⛏ Mine Hard Examples (E2E)

Export selected traces → Run weak model inference → Score with judge → Pick threshold to export hard examples

Settings

Test Judge

Select traces in the viewer, pick models to compare, and the backend runs inference on-the-fly then feeds outputs into the judge.

⚡ Run End-to-End Test

This will automatically: Export selected traces → Run weak models → Evaluate with judge

Running End-to-End Test