Metadata-Version: 2.5
Name: rulesmith
Version: 0.1.2
Summary: Search for rule programs that ask a small judge for typed decisions, with DSPy and GEPA.
Project-URL: Homepage, https://github.com/bradAGI/rulesmith
Project-URL: Repository, https://github.com/bradAGI/rulesmith
Project-URL: Issues, https://github.com/bradAGI/rulesmith/issues
Author: rulesmith contributors
License-Expression: MIT
License-File: LICENSE
Keywords: decision graphs,dspy,gepa,llm,program search,rules
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: <3.15,>=3.12
Requires-Dist: dspy==3.4.0
Requires-Dist: gepa==0.1.4
Requires-Dist: httpx<1,>=0.28
Requires-Dist: pydantic<3,>=2.11
Requires-Dist: typesafe-sdk<0.7,>=0.6
Provides-Extra: bench
Requires-Dist: huggingface-hub<2,>=1; extra == 'bench'
Requires-Dist: pyarrow<24,>=17; extra == 'bench'
Requires-Dist: scikit-learn<2,>=1.5; extra == 'bench'
Provides-Extra: chess
Requires-Dist: chess<2,>=1.11; extra == 'chess'
Provides-Extra: clef
Requires-Dist: huggingface-hub<2,>=1; extra == 'clef'
Requires-Dist: pillow<13,>=11; extra == 'clef'
Requires-Dist: torch<3,>=2.11; extra == 'clef'
Requires-Dist: torchvision<1,>=0.26; extra == 'clef'
Requires-Dist: transformers[torch]<6,>=5.10.2; extra == 'clef'
Provides-Extra: doom
Requires-Dist: imageio-ffmpeg<1,>=0.6; extra == 'doom'
Requires-Dist: numpy<3,>=2; extra == 'doom'
Requires-Dist: pillow<13,>=11; extra == 'doom'
Requires-Dist: vizdoom<2,>=1.3; extra == 'doom'
Description-Content-Type: text/markdown

# rulesmith

![rulesmith: rules that ask a local model for typed decisions](https://raw.githubusercontent.com/bradAGI/rulesmith/main/image.png)

**rulesmith lets DSPy build decision graphs for you.** Give it a classification
task and labeled examples. It searches for a small program of rules: cheap tests
on the input's fields settle the cases they can, and a local or hosted judge is
asked typed questions about the rest. [GEPA](https://github.com/gepa-ai/gepa)
keeps the revisions that score better, and the result is a rules file you can
read, edit, run, and serve.

```
read has_company_logo in [0, 1]
read length of company_profile in [0, 10000]
when has_company_logo == 0 and length of company_profile == 0: fraudulent
always: ask "Is this job posting a scam?" [fraudulent: "fraudulent"] [genuine: "genuine"]
```

![Same 4B judge, same 100 examples: rulesmith beat DSPy's tuned classifier by 6.7 points on job-fraud detection with 29% of the model calls, and tied it on clothing reviews with 6%](https://raw.githubusercontent.com/bradAGI/rulesmith/main/docs/assets/claim.svg)

The [benchmarks](https://github.com/bradAGI/rulesmith/blob/main/docs/benchmarks.md) have every dataset, including where rulesmith
does not help: on one-sentence inputs it ties DSPy, and on job postings a TF-IDF
classifier still beats it.

An independent research prototype, not affiliated with TypeSafe.

## Installation

```sh
pip install rulesmith
```

Python 3.12 to 3.14. Extras for the games and benchmarks are listed in the
[quickstart](https://github.com/bradAGI/rulesmith/blob/main/docs/quickstart.md#installation).

## Quickstart

With a judge running ([Ruling](https://github.com/bradAGI/ruling) on a Mac or through vLLM,
Cloudflare's Clef-flash on an NVIDIA GPU or a Mac, TypeSafe's hosted Jev, or any
OpenAI-compatible chat API that returns log-probabilities), any OpenAI-compatible model to write the rules, and a
labeled task such as [this one](https://github.com/bradAGI/rulesmith/blob/main/examples/support-routing/task.json):

```sh
export OPENAI_API_KEY=local-no-auth   # a local model server takes any key
rulesmith optimize task.json --backend ruling \
  --reflection-model "openai/<model>" --reflection-base-url http://127.0.0.1:8080/v1 \
  --output runs/my-task
rulesmith run runs/my-task/plan.rules input.json --backend ruling
```

The [quickstart](https://github.com/bradAGI/rulesmith/blob/main/docs/quickstart.md) walks through it with every command's output.

## Beyond classification

The same search plays games. Pitted against a league of other graphs, this one learned to
hunt in Doom deathmatch and beat the champion 12-0, with no model calls at all
([how](https://github.com/bradAGI/rulesmith/blob/main/docs/doom.md)):

![A searched rules graph hunting down the Doom champion it dethroned](https://raw.githubusercontent.com/bradAGI/rulesmith/main/docs/assets/doom/duel-hunter.gif)

## Learn more

- [Quickstart](https://github.com/bradAGI/rulesmith/blob/main/docs/quickstart.md): installation, judges, and a task to a running graph.
- [Notebook](https://github.com/bradAGI/rulesmith/blob/main/notebooks/quickstart.ipynb): rules for spotting fake job postings, end to end in Colab with an OpenAI key.
- [Local notebook](https://github.com/bradAGI/rulesmith/blob/main/notebooks/local.ipynb): the same with Clef-flash judging on your Mac or NVIDIA GPU, free per call.
- [Tasks](https://github.com/bradAGI/rulesmith/blob/main/docs/tasks.md): the task file, drafting labels, structured inputs, costs, and graders.
- [Rules](https://github.com/bradAGI/rulesmith/blob/main/docs/rules.md): the language search writes, confidence, calibration, escalation, and extraction.
- [Graph format](https://github.com/bradAGI/rulesmith/blob/main/docs/graphs.md): the JSON behind each node, execution, and drawing a graph.
- [Maps](https://github.com/bradAGI/rulesmith/blob/main/docs/maps.md): many decisions as one map, searched a unit at a time.
- [Optimization](https://github.com/bradAGI/rulesmith/blob/main/docs/optimization.md): how search runs, its settings, and its limits.
- [Command line](https://github.com/bradAGI/rulesmith/blob/main/docs/cli.md): every command, including `serve` and `drift`, and the backends.
- [Python](https://github.com/bradAGI/rulesmith/blob/main/docs/python.md): running and building graphs from code.
- [Benchmarks](https://github.com/bradAGI/rulesmith/blob/main/docs/benchmarks.md): rulesmith against DSPy, TF-IDF and direct calls on five public datasets.
- [Related work](https://github.com/bradAGI/rulesmith/blob/main/docs/related-work.md): when to use rulesmith and when DSPy, and what came before.
- [Doom](https://github.com/bradAGI/rulesmith/blob/main/docs/doom.md) and [Chess](https://github.com/bradAGI/rulesmith/blob/main/docs/chess.md): graphs that play.
- [Claude Code skill](https://github.com/bradAGI/rulesmith/blob/main/skills/rulesmith/SKILL.md): copy `skills/rulesmith` into `~/.claude/skills/`.

## Development

```sh
uv sync --locked
uv run pytest -q
```

## License

[MIT](https://github.com/bradAGI/rulesmith/blob/main/LICENSE).
