Metadata-Version: 2.4
Name: levi-evolve
Version: 0.1.1
Summary: Levi: Evolutionary optimization framework for algorithms and prompts
License: MIT License
        
        Copyright (c) 2025 Temoor Tanveer
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: colorama>=0.4.6
Requires-Dist: dspy>=3.1.2
Requires-Dist: litellm>=1.81.16
Requires-Dist: numpy>=2.3.5
Requires-Dist: scikit-learn>=1.4.0
Provides-Extra: demos
Requires-Dist: datasets>=3.0; extra == 'demos'
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.11.0; extra == 'dev'
Provides-Extra: example-cloudcast
Requires-Dist: networkx>=3.0; extra == 'example-cloudcast'
Provides-Extra: example-eplb
Requires-Dist: torch>=2.9.1; extra == 'example-eplb'
Provides-Extra: example-hover
Requires-Dist: bm25s>=0.3.6; extra == 'example-hover'
Requires-Dist: datasets<4; extra == 'example-hover'
Requires-Dist: pystemmer>=3.0.0; extra == 'example-hover'
Provides-Extra: example-ifbench
Requires-Dist: datasets>=3.0; extra == 'example-ifbench'
Requires-Dist: emoji>=2.0; extra == 'example-ifbench'
Requires-Dist: immutabledict>=4.0; extra == 'example-ifbench'
Requires-Dist: langdetect>=1.0.9; extra == 'example-ifbench'
Requires-Dist: litellm>=1.50; extra == 'example-ifbench'
Requires-Dist: nltk>=3.9; extra == 'example-ifbench'
Requires-Dist: packaging>=24.0; extra == 'example-ifbench'
Requires-Dist: spacy>=3.7; extra == 'example-ifbench'
Requires-Dist: syllapy>=0.7; extra == 'example-ifbench'
Provides-Extra: example-llm-sql
Requires-Dist: pandas>=2.3.3; extra == 'example-llm-sql'
Provides-Extra: examples
Requires-Dist: datasets>=3.0; extra == 'examples'
Requires-Dist: emoji>=2.0; extra == 'examples'
Requires-Dist: immutabledict>=4.0; extra == 'examples'
Requires-Dist: langdetect>=1.0.9; extra == 'examples'
Requires-Dist: litellm>=1.50; extra == 'examples'
Requires-Dist: networkx>=3.0; extra == 'examples'
Requires-Dist: nltk>=3.9; extra == 'examples'
Requires-Dist: packaging>=24.0; extra == 'examples'
Requires-Dist: pandas>=2.3.3; extra == 'examples'
Requires-Dist: spacy>=3.7; extra == 'examples'
Requires-Dist: syllapy>=0.7; extra == 'examples'
Requires-Dist: torch>=2.9.1; extra == 'examples'
Description-Content-Type: text/markdown

<p align="center">
  <img src="assets/logos/levi_logo_dark.svg#gh-dark-mode-only" width="25%" alt="LEVI" />
  <img src="assets/logos/levi_logo_light.svg#gh-light-mode-only" width="25%" alt="LEVI" />
</p>

<p align="center"><strong>AlphaEvolve Performance for a Fraction of the Cost</strong></p>

<p align="center">
  <a href="https://github.com/ttanv/levi/actions/workflows/ci.yml?query=branch%3Amain"><img src="https://github.com/ttanv/levi/actions/workflows/ci.yml/badge.svg?branch=main" alt="CI"></a>
  <a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.11%2B-blue.svg" alt="Python 3.11+"></a>
  <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-green.svg" alt="License: MIT"></a>
  <a href="https://arxiv.org/abs/2605.09764"><img src="https://img.shields.io/badge/arXiv-2605.09764-b31b1b.svg" alt="arXiv"></a>
</p>

---

LEVI is an LLM-guided evolutionary framework for **code** and **prompts**. Point it at a scoring function and a budget and LEVI evolves the artifact for you, using API models, a local server, or your Claude Code / Codex CLI subscription. **$5 with a local Qwen 30B improves on what other frameworks need $30 and Claude Opus to achieve** across a variety of problems, at a fraction of the cost.

## Why LEVI

Existing frameworks couple performance tightly to model capability. Drop to a smaller model and results degrade sharply. LEVI decouples the two by making **diversity an architectural concern** rather than a model concern, and by matching model capacity to task demand.

Cheap models handle the bulk of mutation work. A behavioral archive keeps structurally different strategies alive, preventing premature convergence. Periodic paradigm shifts from a stronger model inject genuinely new ideas. The result: you spend less and get more.


<p align="center">
  <img src="assets/plots/figure_front_page.png" width="100%" alt="LEVI vs baselines on code and prompt optimization" />
</p>
<p align="center"><em>LEVI on code optimization (exceeds every baseline's final score within 1/15th of the evaluations) and prompt optimization (outperforms GEPA at less than half the rollouts).</em></p>

## Quickstart

```bash
# Install uv first: https://docs.astral.sh/uv/getting-started/installation/
git clone https://github.com/ttanv/levi.git
cd levi
uv sync
```

Pick whichever path matches what you have access to — each is a single self-contained file under [`examples/quickstart/`](examples/quickstart/) that runs in a couple of minutes:

| You have…                                | Run                                                       | Evolves | Costs you                |
| ---------------------------------------- | --------------------------------------------------------- | ------- | ------------------------ |
| a Claude Code or Codex CLI subscription  | `uv run python examples/quickstart/quickstart_claude.py` (or `quickstart_codex.py`) | code | $0 (subscription quota)  |
| an API key (OpenAI / Anthropic / …)      | `uv run python examples/quickstart/quickstart_api.py`     | code    | ~$0.05–0.10              |
| an API key, and you want to tune prompts | `uv run python examples/quickstart/quickstart_prompts.py` | prompts | ~$0.05–0.10              |

The CLI rows use your existing Claude or Codex subscription — no API key needed. For the API rows, set `OPENAI_API_KEY` (or change `MODEL` at the top of the file to another [litellm provider](https://docs.litellm.ai/docs/providers) and set the matching key) before running.

A minimal LEVI program looks like this:

```python
import levi

result = levi.evolve_code(
    "Place 16 non-overlapping circles in the unit square. Maximize sum of radii.",
    function_signature="def run_packing() -> tuple[np.ndarray, np.ndarray, float]: ...",
    score_fn=score_fn,  # see levi.demos.circle_packing
    model="openai/gpt-4o-mini",
    budget_dollars=0.10,
)
print(result.best_score, result.best_program)
```

See [`examples/quickstart/quickstart_api.py`](examples/quickstart/quickstart_api.py) for a runnable version. Output snapshots write to `./runs/<timestamp>/` relative to your CWD; override with `output_dir="path/to/dir"`.

## Going further

- `examples/quickstart/` — the four single-file starters above (three code, one prompt). The three code starters evolve a mini n=16 circle-packing function; the prompt starter tunes a prompt for AIME math on a small Qwen.
- `examples/benchmarks/code/` — `evolve_code` at paper scale: circle packing (n=26, $15 budget, paradigm + mutation models) and the seven ADRS Leaderboard problems from the paper.
- `examples/benchmarks/prompts/` — `evolve_prompts` benchmarks comparing against GEPA: HotpotQA, HoVer, PUPA, IFBench.
- See [`examples/benchmarks/README.md`](examples/benchmarks/README.md) for datasets, keys, and per-problem setup.

## Results

LEVI holds the **highest average score (76.5)** across all seven [ADRS Leaderboard](https://ucbskyadrs.github.io/) problems, ahead of GEPA (71.9), OpenEvolve (70.6), and ShinkaEvolve (67.4). Six of the seven problems were solved on a **$4.50 budget**.

| Problem | LEVI | Best Other Framework | Saving |
|---------|------|----------------------|--------|
| Spot Single-Reg | **51.7** | GEPA 51.4 | 6.7x cheaper |
| Spot Multi-Reg | **72.4** | OpenEvolve 66.7 | 5.6x cheaper |
| LLM-SQL | **78.3** | OpenEvolve 72.5 | 4.4x cheaper |
| Cloudcast | **100.0** | GEPA 96.6 | 3.3x cheaper |
| Prism | **87.4** | GEPA / OpenEvolve / ShinkaEvolve 87.4 | 3.3x cheaper |
| EPLB | **74.6** | GEPA 70.2 | 3.3x cheaper |
| Txn Scheduling | **71.1** | OpenEvolve 70.0 | 1.5x cheaper |

<p align="center">
  <img src="assets/plots/circle_packing_best.png#gh-dark-mode-only" width="50%" alt="Circle Packing" />
  <img src="assets/plots/circle_packing_best_light.png#gh-light-mode-only" width="50%" alt="Circle Packing" />
</p>

LEVI scored **2.6359+ packing density** on the n=26 circle packing benchmark, with a local model handling the majority of mutations. See [`examples/benchmarks/code/circle_packing`](examples/benchmarks/code/circle_packing) for the full setup.

For advanced routing, pass a `levi.LM(...)` directly:

```python
local_qwen = levi.LM(
    "Qwen/Qwen3-30B-A3B-Instruct-2507",
    api_base="http://localhost:8000/v1",
    api_key="unused",
    input_cost_per_token=0.0000001,
    output_cost_per_token=0.0000004,
)
```

## How It Works

1. **Seed & score.** You provide a starting program and a scoring function. LEVI generates diverse variants to populate a behavioral archive.
2. **Evolve.** Cheap models mutate and refine solutions in parallel. The behavioral archive keeps structurally different strategies alive, preventing convergence.
3. **Paradigm shifts.** Periodically, a stronger model proposes entirely new algorithmic approaches based on the archive's best ideas.
4. **Budget stops.** LEVI tracks spend in real time and stops when your dollar, evaluation, or time cap is hit.

## Further Reading

- [LEVI: LLM-Guided Evolutionary Search Needs Better Harnesses, Not Bigger Models](https://ucbskyadrs.github.io/blog/levi/) — The full blog post on the ADRS site.

## Citation

If you use LEVI in your research, please cite:

```bibtex
@software{tanveer2026levi,
  title  = {LEVI: LLM-Guided Evolutionary Search Needs Better Harnesses, Not Bigger Models},
  author = {Tanveer, Temoor},
  url    = {https://github.com/ttanv/levi},
  year   = {2026}
}
```

Contact: ttanveer@alumni.cmu.edu
