Metadata-Version: 2.4
Name: decisionsmith
Version: 0.1.0
Summary: Use and fine-tune System One models (Jev, Laya) on your data, with an LLM as the teacher.
Keywords: laya,jev,system-one,classification,llm,distillation,fine-tuning
Author: Izam Mohammed
Author-email: Izam Mohammed <izamdeveloper1@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Dist: pydantic>=2.6
Requires-Dist: httpx>=0.27
Requires-Dist: agno>=3.0 ; extra == 'agno'
Requires-Dist: decisionsmith[laya,anthropic,mcp] ; extra == 'all'
Requires-Dist: anthropic>=1.0 ; extra == 'anthropic'
Requires-Dist: autogen-core>=0.4 ; extra == 'autogen'
Requires-Dist: claude-agent-sdk>=0.2 ; extra == 'claude-agent-sdk'
Requires-Dist: crewai>=1.0 ; python_full_version < '3.14' and extra == 'crewai'
Requires-Dist: dspy>=3.0 ; extra == 'dspy'
Requires-Dist: google-adk>=2.0 ; extra == 'google-adk'
Requires-Dist: haystack-ai>=2.0 ; extra == 'haystack'
Requires-Dist: instructor>=1.10 ; extra == 'instructor'
Requires-Dist: langchain-core>=1.0 ; extra == 'langchain'
Requires-Dist: langgraph>=1.0 ; extra == 'langgraph'
Requires-Dist: laya>=0.3.20,<0.4 ; extra == 'laya'
Requires-Dist: litellm>=1.70 ; extra == 'litellm'
Requires-Dist: llama-index-core>=0.12 ; extra == 'llamaindex'
Requires-Dist: llm>=0.26 ; extra == 'llm-plugin'
Requires-Dist: mcp>=2.2 ; extra == 'mcp'
Requires-Dist: openai>=1.40 ; extra == 'openai'
Requires-Dist: openai-agents>=0.20 ; extra == 'openai-agents'
Requires-Dist: outlines>=1.0 ; python_full_version < '3.14' and extra == 'outlines'
Requires-Dist: pydantic-ai-slim>=1.0 ; extra == 'pydantic-ai'
Requires-Dist: semantic-kernel>=1.0 ; extra == 'semantic-kernel'
Requires-Dist: smolagents>=1.0 ; extra == 'smolagents'
Requires-Python: >=3.10
Project-URL: Homepage, https://github.com/izam-mohammed/decisionsmith
Project-URL: Issues, https://github.com/izam-mohammed/decisionsmith/issues
Provides-Extra: agno
Provides-Extra: all
Provides-Extra: anthropic
Provides-Extra: autogen
Provides-Extra: claude-agent-sdk
Provides-Extra: crewai
Provides-Extra: dspy
Provides-Extra: google-adk
Provides-Extra: haystack
Provides-Extra: instructor
Provides-Extra: langchain
Provides-Extra: langgraph
Provides-Extra: laya
Provides-Extra: litellm
Provides-Extra: llamaindex
Provides-Extra: llm-plugin
Provides-Extra: mcp
Provides-Extra: openai
Provides-Extra: openai-agents
Provides-Extra: outlines
Provides-Extra: pydantic-ai
Provides-Extra: semantic-kernel
Provides-Extra: smolagents
Description-Content-Type: text/markdown

# decisionsmith

**Use and fine-tune System One models (Jev, Laya) on your data.**
Start with an LLM. End with a fast decision model you trained. One line in between.

```bash
uv add "decisionsmith[all]"
```

<!-- no-test: uses the Ticket class defined in the next block, which runs it -->
```python
import decisionsmith as ds

h = ds.harness(Ticket, teacher="claude-sonnet-5", student="laya")
h("You charged me twice, refund now!")
# Ticket(team='billing', wants_refund=True)
```

That's it. `Ticket` is a normal Pydantic model:

```python
from typing import Annotated, Literal

from pydantic import BaseModel, Field

import decisionsmith as ds


class Ticket(BaseModel):
    team: Annotated[
        Literal["billing", "technical", "sales"],
        ds.Options(billing="payments, refunds", technical="bugs, outages", sales="pricing"),
    ]
    wants_refund: bool = Field(description="Does the customer ask for their money back?")
```

`Literal` / `Enum` → pick one · `bool` → yes/no · `Annotated[Literal["low", "medium", "high"], ds.Scale]` → a rating.

## Why

Decision models like Laya are fast and free to run, and weak zero-shot (0.36 accuracy on typed-decisions before
fine-tuning, 0.77 after, per Laya's benchmarks). LLMs are good zero-shot, and slow and paid per call.
decisionsmith starts with the LLM, logs every answer, fine-tunes the fast model on those answers, and moves traffic
to it field by field once the numbers say it's ready.

So don't trust a System One model zero-shot on your task. Let a normal LLM make the decisions first (`mode="teacher"`
or `"shadow"`), behind the guardrails decisionsmith already has: answers validated against your schema, the text
fenced as data, a fallback when an engine fails, and a per-field confidence threshold plus a 5% audit before any
field moves to the student.

## The loop

```python
texts = open("texts.txt", encoding="utf-8").read().splitlines()  # your texts, one per line
h = ds.harness(Ticket, teacher="claude-sonnet-5", student="laya", mode="shadow")  # teacher answers, student measured
h.many(texts)  # every decision is logged (decisions.db)
print(h.status())  # per field: agreement, sure rate, accuracy when sure, what to do next
h.adapt()  # calibrate the student's confidence + pick safe thresholds (seconds, any engine)
h.finetune()  # train Laya on the logged answers; switches only if it beats the current student
```

```
team:         shadow · student agrees 94% · sure on 71% · accuracy when sure 97% (vs teacher) -> ready for cascade
wants_refund: shadow · student agrees 81% · 420 labelled -> ready to finetune: run h.finetune()
```

Then move a field when it's ready: `mode={"team": "cascade", "wants_refund": "shadow"}`.

## Four modes, per field

| mode | who answers | use when |
|---|---|---|
| `teacher` | the LLM | day 0 |
| `shadow` | the LLM; the student runs silently and is measured | measuring the student |
| `cascade` (default) | the student when it's sure, else the LLM (5% of sure answers are audited) | student proven on some fields |
| `student` | the student | student proven everywhere |

If an engine fails, the other one answers and the result says `sure=False`. Nothing crashes in the middle.

## Connect anything

| engine | string | notes |
|---|---|---|
| Any LLM | `"claude-sonnet-5"`, `"gpt-5"`, `"ollama/qwen3"` … | via LiteLLM |
| Jev (TypeSafe) | `"jev"` | hosted, needs `TYPESAFE_API_KEY`; can be adapted, not fine-tuned |
| Laya (local) | `"laya"`, `"laya:multilingual"`, `"laya:./runs/v1"` | runs on your machine |
| Laya server / laya.cpp | `"systemone:http://localhost:8000"` | any Jev-compatible `/v1/systemone` endpoint |

Any engine can be the teacher or the student.

## Fine-tune with labels you already have

```bash
decisionsmith finetune tickets.csv --schema app.py:Ticket --out runs/v1
```

A CSV needs a `text` column and one column per field. Small data (under 1,000 rows) trains only the decision
head, fast even on a laptop; bigger data trains the whole model (use a GPU, or the Kaggle/Colab notebook in
`examples/`). You get `runs/v1/`: a plain Laya checkpoint (`laya.load("runs/v1")` works), a report comparing it
with the base model on held-out data, and a go/no-go verdict with the reasons.

Or from Python, and keep the result as a versioned folder you can load anywhere:

```python
model = ds.model(["billing", "technical", "sales"])
model.train("tickets.csv")
print(model.evaluate("test.csv"))  # held-out numbers and go/no-go
m = ds.load(model.save("models/ticket"))  # models/ticket-v1
```

In production, `ds.harness(m, teacher=llm, collect=0.1)` keeps a sample of real texts, and
`ds.golden("decisions.db", teacher=llm, schema=labels)` turns them into the next labelled dataset
([golden](https://github.com/izam-mohammed/decisionsmith/blob/main/docs/golden.md), [collect](https://github.com/izam-mohammed/decisionsmith/blob/main/docs/collect.md)).

## Compare engines on your data

```bash
decisionsmith bench tickets.csv --schema app.py:Ticket --engines claude-sonnet-5,jev,laya,laya:./runs/v1
```

## From agents

`uvx --from "decisionsmith[all]" decisionsmith mcp` gives Claude Code, Codex, Cursor and other MCP clients the
harness, bench, status, finetune and label as tools. There's also a Claude Code plugin; see
[docs/agents.md](https://github.com/izam-mohammed/decisionsmith/blob/main/docs/agents.md).

## Docs

[Quickstart](https://github.com/izam-mohammed/decisionsmith/blob/main/docs/quickstart.md) · [Fine-tuning](https://github.com/izam-mohammed/decisionsmith/blob/main/docs/finetune.md) · [Guide: engines, modes, status, adapt,
bench](https://github.com/izam-mohammed/decisionsmith/blob/main/docs/guide.md) · [FAQ](https://github.com/izam-mohammed/decisionsmith/blob/main/docs/faq.md) · [Integrations](https://github.com/izam-mohammed/decisionsmith/blob/main/docs/integrations.md)

Runnable [examples](https://github.com/izam-mohammed/decisionsmith/tree/main/examples) (gallery, every one runs offline) and [notebooks](https://github.com/izam-mohammed/decisionsmith/tree/main/notebooks).

---

Built on [Laya](https://github.com/NandhaKishorM/laya) (Apache-2.0) by Nandakishor M / Convai Innovations.
Jev and TypeSafe are trademarks of their owners; decisionsmith is an independent project, not affiliated with
TypeSafe AI or Convai Innovations.
