Metadata-Version: 2.4
Name: sotto-llm
Version: 0.1.0
Summary: Structured thinking as a pluggable layer for any OpenAI-compatible API
Project-URL: Homepage, https://github.com/Madhav-Gohel/sotto
Project-URL: Repository, https://github.com/Madhav-Gohel/sotto
Project-URL: Issues, https://github.com/Madhav-Gohel/sotto/issues
Author-email: Madhav Gohel <madhavgohel2@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: agents,llm,openai,reasoning,skills,thinking,tool-use
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Provides-Extra: proxy
Requires-Dist: fastapi>=0.110; extra == 'proxy'
Requires-Dist: openai>=1.30; extra == 'proxy'
Requires-Dist: python-dotenv>=1.0; extra == 'proxy'
Requires-Dist: uvicorn>=0.27; extra == 'proxy'
Description-Content-Type: text/markdown

<p align="center">
  <picture>
    <img alt="Sotto" src="https://img.shields.io/badge/sotto-structured%20thinking-6c47ff?style=for-the-badge&labelColor=1a1a1a" width="360">
  </picture>
</p>

<h3 align="center">
Structured thinking as a pluggable layer for any OpenAI-compatible API
</h3>

<p align="center">
<em>sotto voce</em> — the passage spoken under the breath
</p>

<p align="center">
| <a href="#getting-started"><b>Getting Started</b></a> | <a href="#how-it-works"><b>How It Works</b></a> | <a href="#use-as-a-library"><b>Library</b></a> | <a href="#use-as-a-proxy"><b>Proxy</b></a> | <a href="#skills"><b>Skills</b></a> | <a href="#contributing"><b>Contributing</b></a> |
</p>

<p align="center">
<a href="https://pypi.org/project/sotto-llm/"><img src="https://img.shields.io/pypi/v/sotto-llm.svg" alt="PyPI"></a>
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.10%2B-blue.svg" alt="Python 3.10+"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green.svg" alt="MIT License"></a>
<img src="https://img.shields.io/badge/core-zero%20dependencies-brightgreen.svg" alt="Zero dependencies">
<img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg" alt="PRs welcome">
<a href="https://github.com/Madhav-Gohel/sotto"><img src="https://img.shields.io/github/stars/Madhav-Gohel/sotto?style=social" alt="GitHub stars"></a>
</p>

---

**Sotto gives any model a reasoning model's mechanism** — a private thinking phase, effort control, interleaved thinking across tool calls, and skills — without changing your model, your SDK, or your code. The core is one stdlib-only file you can drop into any project. An optional proxy adds the same layer to any client just by repointing `base_url`.

Works with **OpenAI · Azure OpenAI · vLLM · Ollama · DeepSeek · Groq · OpenRouter · LM Studio** — anything that speaks `/v1/chat/completions`.

---

## Latest Updates 🔥

- **[2026/07]** Initial public release of Sotto: single-file thinking engine, OpenAI-compatible proxy, and drop-in `SKILL.md` support.

<details>
<summary>Previous News</summary>

- Nothing here yet — you're early. ⭐ Star the repo to follow along.

</details>

---

## About

Reasoning models (o-series, DeepSeek R1, gpt-5) share a set of mechanisms: they think privately before answering, scale that thinking by an effort setting, keep reasoning between tool calls, and hide the thinking from the final output. Sotto reproduces those mechanisms as a thin, portable layer so **every** model gets them — including the ones that don't reason natively.

| Reasoning-model concept | Sotto's implementation |
|---|---|
| Thinking blocks before the answer | `<thinking>` tags parsed out, or native `reasoning_content` harvested |
| Adaptive thinking + `effort` (low/medium/high) | `ThinkingConfig.effort`, mapped to `reasoning_effort` on o-series or a depth hint in the prompt |
| Legacy `budget_tokens` | `ThinkingConfig.budget_tokens`, auto-mapped to an effort tier |
| Interleaved thinking between tool calls | The `run()` loop: think → call tool → think about result → repeat |
| Thinking preserved during tool use | Raw assistant content (tags included) kept in message history across iterations |
| Thinking hidden from the user | Stripped from `result.content`; available in `result.thinking` and `result.steps` |
| Skills | `SKILL.md` folders, keyword-matched and injected into the system prompt per request |

**Design goals**

- **Zero dependencies in the core.** `thinking.py` is stdlib only. Drop it into any project, no install.
- **No lock-in.** Any OpenAI-compatible client works. Repoint `base_url` and you're done.
- **Progressive disclosure.** Skills are injected only when a request matches, keeping context small.
- **Graceful degradation.** Weak models that skip the tags still produce a usable answer.

---

## Getting Started

### Install

```bash
pip install sotto-llm            # core: zero dependencies, stdlib only
pip install "sotto-llm[proxy]"   # + the proxy server (fastapi, uvicorn, openai, python-dotenv)
```

The core is still one self-contained stdlib-only file, so vendoring stays an option:

```bash
curl -O https://raw.githubusercontent.com/Madhav-Gohel/sotto/main/src/sotto_llm/thinking.py
```

### Try it offline (no API key)

```bash
git clone https://github.com/Madhav-Gohel/sotto.git && cd sotto
pip install -e .
python example.py   # scripted mock model runs the full think → tool → think → answer loop
```

---

## How It Works

Sotto auto-selects one of three reasoning strategies based on the model:

1. **`visible_native`** — DeepSeek R1, QwQ, vLLM/Ollama reasoning models. The server already returns `message.reasoning_content`; Sotto just harvests it.
2. **`hidden_native`** — OpenAI o-series / gpt-5 family. Sotto passes `reasoning_effort`; reasoning happens server-side.
3. **`prompted`** — everything else (gpt-4o-mini, llama, mistral…). Sotto injects a thinking protocol, the model reasons inside `<thinking>` tags, and Sotto parses them out. Unclosed tags (model cut off mid-thought) are handled.

Force a strategy with `mode="prompted" | "native" | "off"` instead of the default `"auto"`.

---

## Use as a Library

```python
from openai import OpenAI
from sotto_llm import ThinkingEngine, ThinkingConfig

client = OpenAI(base_url="http://localhost:8000/v1")   # vLLM, Ollama, Azure, anything

engine = ThinkingEngine(
    client, "gpt-4o-mini",
    config=ThinkingConfig(effort="high"),
    tools=my_openai_tools,
    tool_handlers={"get_weather": get_weather},   # python callables, run locally
    skills_dir="skills",
)

result = engine.run([{"role": "user", "content": "weather in ahmedabad?"}])

result.content              # final answer, thinking stripped
result.thinking             # list of every thinking block, in order
result.steps                # full interleaved trace: thinking / tool_call / tool_result / answer
result.as_openai_message()  # {"role": "assistant", "content": ..., "reasoning_content": ...}
```

The engine runs a full **think → act → think** agentic loop, executing your tool handlers locally and feeding results back until the model produces a final answer (capped by `max_tool_iterations`).

---

## Use as a Proxy

Any existing system — bots, agents, SDKs — gains thinking **without code changes**. Just repoint `base_url`.

```bash
pip install "sotto-llm[proxy]"
cp .env.example .env          # set UPSTREAM_BASE_URL / UPSTREAM_API_KEY
uvicorn sotto_llm.proxy:app --port 8088
```

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8088/v1", api_key="x")
resp = client.chat.completions.create(
    model="llama3.1",
    messages=[{"role": "user", "content": "..."}],
    extra_body={"thinking": {"effort": "high"}},   # optional; defaults apply without it
)

resp.choices[0].message.content            # clean answer
resp.choices[0].message.reasoning_content  # the thinking (DeepSeek/vLLM convention)
```

Reasoning comes back in `message.reasoning_content` — the same field DeepSeek and vLLM use — so existing clients keep working untouched. If the request includes `tools`, the proxy returns `tool_calls` to the caller like a normal OpenAI server (thinking still captured); the full agentic loop with local execution lives in library mode, since tools run on your side.

---

## Skills

Each skill is a folder with a `SKILL.md` in the standard skill format: `name` and `description` in the frontmatter, instructions in the body. Existing skills in that format drop in unchanged.

```markdown
---
name: code-review
description: Structured checklist for reviewing code changes... Use this skill
  whenever the user asks you to review code, look at a PR or diff, or asks
  whether some code "looks right" — even if they don't say "code review."
---

# Code Review
Review code in this order. The order matters: a fast, elegant function that
returns wrong answers is worse than a slow correct one...
```

The `description` does the triggering. Terms are extracted from the clause after "use this skill" — including quoted phrases and short domain tokens (`pr`, `ci`, `sql`) — then matched on word boundaries against the user's message. Only matching skills are injected, keeping context small (progressive disclosure).

`keywords: a, b, c` is supported as an optional extension. When present, it overrides description matching and a single hit triggers — useful when a skill needs to fire on jargon its description doesn't contain.

> **On matching quality:** this is lexical matching, not semantic. It handles clear cases well but misses paraphrases that share no vocabulary with the description. If you have an embedding endpoint, swapping `match_skills()` for cosine similarity over descriptions is a strict upgrade — the rest of the engine is unaffected.

---

## Configuration Reference

| Field | Default | Meaning |
|---|---|---|
| `enabled` | `True` | Turn the whole layer off with `False` |
| `effort` | `"medium"` | low / medium / high reasoning depth |
| `budget_tokens` | `None` | Legacy knob; `<4096` low, `<16384` medium, else high |
| `mode` | `"auto"` | auto / prompted / native / off |
| `max_tool_iterations` | `8` | Safety cap on the agentic loop |

Proxy environment variables (via `.env`):

| Variable | Default | Meaning |
|---|---|---|
| `UPSTREAM_BASE_URL` | SDK default | The OpenAI-compatible server to forward to |
| `UPSTREAM_API_KEY` | `x` | Key the upstream requires (any placeholder for local servers) |
| `SKILLS_DIR` | `skills` | Directory the engine loads `SKILL.md` files from |

---

## Limitations

- Prompted-mode thinking counts as normal output tokens (no separate billing lane like native reasoning).
- Tags are a convention, not enforcement; weak models occasionally skip them (the parser degrades gracefully — everything becomes the answer).
- The proxy is non-streaming; add SSE if you need it.
- The engine does not enforce a hard token cutoff on thinking; effort is a hint, matching how adaptive thinking treats it.

---

## Contributing

Contributions are welcome — issues, feature requests, and PRs all help.

1. Fork the repo and create a branch from `main`.
2. Keep the **core (`src/sotto_llm/thinking.py`) dependency-free** — stdlib only. New dependencies belong in the proxy extra or the examples.
3. If you touch matching or the reasoning loop, add or update a case in `example.py` so it runs offline.
4. Open a PR describing the change and the motivation.

Good first contributions: a streaming (SSE) proxy mode, an embedding-based `match_skills()` variant, and more example skills.

---

## License

MIT — see [LICENSE](LICENSE).

---

## Citation

If you use Sotto in your work, please cite it:

```bibtex
@software{sotto,
  title  = {Sotto: Structured thinking as a pluggable layer for OpenAI-compatible APIs},
  author = {Madhav Gohel},
  year   = {2026},
  url    = {https://github.com/Madhav-Gohel/sotto}
}
```

---

<p align="center">
Built for anyone who wants their model to think first. ⭐ Star the repo if it's useful.
</p>
