Metadata-Version: 2.4
Name: llmbroker
Version: 1.2.0
Project-URL: Homepage, https://andgineer.github.io/llmbroker/
Project-URL: Documentation, https://andgineer.github.io/llmbroker/
Author-email: Andrey Sorokin <andrey@sorokin.engineer>
License: Copyright (c) 2026 Andrey Sorokin (andrey@sorokin.engineer)
        
        Permission is hereby granted, free of charge, to any person obtaining a copy of
        this software and associated documentation files (the “Software”), to deal in
        the Software without restriction, including without limitation the rights to use,
        copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
        Software, and to permit persons to whom the Software is furnished to do so,
        subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
        OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
        NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
        HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
        WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
        FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
        OTHER DEALINGS IN THE SOFTWARE.
License-File: LICENSE
Keywords: one,two
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.11
Requires-Dist: httpx>=0.28.1
Requires-Dist: pyyaml>=6.0
Provides-Extra: aws
Requires-Dist: aioboto3>=13.0; extra == 'aws'
Provides-Extra: mongodb
Requires-Dist: motor>=3.6.0; extra == 'mongodb'
Provides-Extra: postgres
Requires-Dist: asyncpg>=0.30.0; extra == 'postgres'
Provides-Extra: sqlite
Requires-Dist: aiosqlite>=0.22.1; extra == 'sqlite'
Provides-Extra: vault
Requires-Dist: hvac>=2.0; extra == 'vault'
Description-Content-Type: text/markdown

[![Build Status](https://github.com/andgineer/llmbroker/workflows/CI/badge.svg)](https://github.com/andgineer/llmbroker/actions)
[![Coverage](https://raw.githubusercontent.com/andgineer/llmbroker/python-coverage-comment-action-data/badge.svg)](https://htmlpreview.github.io/?https://github.com/andgineer/llmbroker/blob/python-coverage-comment-action-data/htmlcov/index.html)
# llmbroker

Turn a crowd of free, rate-limited LLMs into one reliable model — no premium
subscription, no single point of failure. No LangChain, no heavy deps.

```bash
pip install llmbroker
llmbroker preset freetier > llms.toml   # ready-made pool of free models
llmbroker env llms.toml > .env          # which API keys to get, and where
```

```python
llms = llmbroker.Broker("llms.toml")
reply = llms.ask("Explain decorators in one sentence")
print(reply.text)   # groq rate-limited? gemini answers instead
```

Fill in whichever keys are easy — models without keys just stay inactive.

| | |
|---|---|
| **Automatic failover** | `llms.ask(...)` — next model answers when one is down |
| **Chat, tools & agents** | `llms.chat(messages, tools=...)`, `run_tool_loop(...)` |
| **Async-first** | `AsyncBroker` — same engine, for FastAPI / agents / workers |
| **Scale out** | `Broker("postgresql://…")` — sqlite / Postgres / MongoDB, calling code unchanged |
| **Self-regulating pool** | `reply.record_quality(0.3)` — weak models sink per task kind |
| **Pluggable secrets** | env vars, DB, AWS, Vault, or your own backend |
| **Multi-user mode** | per-user API keys on top of one shared pool |

[Documentation](https://andgineer.github.io/llmbroker/)

<details>
<summary>Development</summary>

Do not forget to run `. ./activate.sh`. It needs [uv](https://github.com/astral-sh/uv) installed.

Use [pre-commit](https://pre-commit.com/#install) hooks for code quality:

    pre-commit install

Install [invoke](https://docs.pyinvoke.org/en/stable/) preferably with [uv tool](https://docs.astral.sh/uv/):

    uv tool install invoke

For a list of available scripts run `invoke --list`; for details on one, `invoke <script> --help`.

The bundled `freetier` preset drifts as providers change their free tiers; refresh
it with `invoke catalog-refresh`, which prints the maintenance runbook
(`presets/freetier-refresh-prompt.md`).

Reports:

* [Allure test report](https://andgineer.github.io/llmbroker/builds/tests/)
* [Codecov](https://app.codecov.io/gh/andgineer/llmbroker/tree/main/src%2Fllmbroker)
* [Coveralls](https://coveralls.io/github/andgineer/llmbroker)

> Created with cookiecutter using [template](https://github.com/andgineer/cookiecutter-python-package)

</details>
