Metadata-Version: 2.4
Name: llmbroker
Version: 0.0.11
Project-URL: Homepage, https://andgineer.github.io/llmbroker/
Project-URL: Documentation, https://andgineer.github.io/llmbroker/
Author-email: Andrey Sorokin <andrey@sorokin.engineer>
License: Copyright (c) 2026 Andrey Sorokin (andrey@sorokin.engineer)
        
        Permission is hereby granted, free of charge, to any person obtaining a copy of
        this software and associated documentation files (the “Software”), to deal in
        the Software without restriction, including without limitation the rights to use,
        copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
        Software, and to permit persons to whom the Software is furnished to do so,
        subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
        OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
        NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
        HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
        WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
        FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
        OTHER DEALINGS IN THE SOFTWARE.
License-File: LICENSE
Keywords: one,two
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.11
Requires-Dist: aiosqlite>=0.22.1
Requires-Dist: httpx>=0.28.1
Description-Content-Type: text/markdown

[![Build Status](https://github.com/andgineer/llmbroker/workflows/CI/badge.svg)](https://github.com/andgineer/llmbroker/actions)
[![Coverage](https://raw.githubusercontent.com/andgineer/llmbroker/python-coverage-comment-action-data/badge.svg)](https://htmlpreview.github.io/?https://github.com/andgineer/llmbroker/blob/python-coverage-comment-action-data/htmlcov/index.html)
# llmbroker

Route LLM calls over a **pool of free LLMs** with automatic round-robin and
cooldown.

No LangChain, no heavy deps.

```python
llms = llmbroker.Broker("llms.toml")
print(llms.ask("Explain Python decorators in one sentence").text)
```

Put your LLMs in `llms.toml` — or grab a preset:

```bash
llmbroker preset freetier > llms.toml
```

The only setup is the API keys. Ask llmbroker which ones your pool needs — each with a note on where to get it:

```bash
llmbroker env llms.toml > .env
```

The same `Broker` scales straight to a server or cluster — add Redis or Postgres to share cooldown state across instances, the calling code stays the same.

**Why llmbroker:**

- **Automatic failover** — when one LLM is rate-limited or down, the next one in
  the pool answers instead; you get a reply, not an error, as long as any LLM is up.
- **Chat, tools & agents** — one-shot `ask`, multi-turn `chat`, and function/tool
  calling for agentic workflows.
- **Async-first** — `llmbroker.AsyncBroker` built on asyncio for FastAPI, agents, and async workers;
  `llmbroker.Broker` wraps the same engine in a blocking API for plain scripts.
- **Pluggable backends** — swap registry, state store, and telemetry independently: DB, Redis etc.
- **API keys configurable storage** — environment variables, DB, AWS, Vault, your own store.
- **Multi-user mode** -- optionally separate secrets and LLMs list for each application user.

# Documentation

[llmbroker](https://andgineer.github.io/llmbroker/)

# Developers

Do not forget to run `. ./activate.sh`.

For work it need [uv](https://github.com/astral-sh/uv) installed.

Use [pre-commit](https://pre-commit.com/#install) hooks for code quality:

    pre-commit install

## Allure test report

* [Allure report](https://andgineer.github.io/llmbroker/builds/tests/)

# Scripts
Install [invoke](https://docs.pyinvoke.org/en/stable/) preferably with [uv tool](https://docs.astral.sh/uv/):

    uv tool install invoke

For a list of available scripts run:

    invoke --list

For more information about a script run:

    invoke <script> --help

## Coverage report
* [Codecov](https://app.codecov.io/gh/andgineer/llmbroker/tree/main/src%2Fllmbroker)
* [Coveralls](https://coveralls.io/github/andgineer/llmbroker)

> Created with cookiecutter using [template](https://github.com/andgineer/cookiecutter-python-package)
