Metadata-Version: 2.4
Name: llmbroker
Version: 0.0.6
Project-URL: Homepage, https://andgineer.github.io/llmbroker/
Project-URL: Documentation, https://andgineer.github.io/llmbroker/
Author-email: Andrey Sorokin <andrey@sorokin.engineer>
License: Copyright (c) 2026 Andrey Sorokin (andrey@sorokin.engineer)
        
        Permission is hereby granted, free of charge, to any person obtaining a copy of
        this software and associated documentation files (the “Software”), to deal in
        the Software without restriction, including without limitation the rights to use,
        copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
        Software, and to permit persons to whom the Software is furnished to do so,
        subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
        OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
        NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
        HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
        WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
        FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
        OTHER DEALINGS IN THE SOFTWARE.
License-File: LICENSE
Keywords: one,two
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.11
Requires-Dist: aiosqlite>=0.22.1
Requires-Dist: httpx>=0.28.1
Description-Content-Type: text/markdown

[![Build Status](https://github.com/andgineer/llmbroker/workflows/CI/badge.svg)](https://github.com/andgineer/llmbroker/actions)
[![Coverage](https://raw.githubusercontent.com/andgineer/llmbroker/python-coverage-comment-action-data/badge.svg)](https://htmlpreview.github.io/?https://github.com/andgineer/llmbroker/blob/python-coverage-comment-action-data/htmlcov/index.html)
# llmbroker

Route LLM calls over a **pool of free LLMs** with automatic round-robin and
cooldown.

No LangChain, no heavy deps.

```python
import llmbroker

llms = llmbroker.Broker("llms.toml")
print(llms.ask("Summarize this receipt").text)
```

`llms.toml` is a plain list of `[[llms]]` entries (base\_url, model, api\_key\_ref).
Grab the [freetier preset](presets/freetier.toml) from this repo to start with a
maintained list of free LLM endpoints.

**Why llmbroker:**

- **Round-robin with automatic failover** — 429/503 cools an endpoint and tries
  the next; the caller never sees a rate-limit error unless *every* endpoint fails.
- **No heavy deps** — stdlib-only core; optional backends (`sqlite`, `redis`,
  `postgres`) are submodules you import only when you need them.
- **Dead-simple to start** — a TOML file and env vars, one constructor line, done.
- **Fully customisable** — swap registry, secrets, and telemetry backends
  independently: any DB, any secrets manager (AWS, Vault, …), any storage.
- **Cluster-ready** — add `shared_state=llmbroker.redis.SharedState(...)` to sync
  cooldown state across instances; omit for single-process.
- **Sync and async** — `llmbroker.Broker` for scripts; `llmbroker.AsyncBroker` for
  FastAPI, agents, and async workers.

# Documentation

[llmbroker](https://andgineer.github.io/llmbroker/)



# Developers

Do not forget to run `. ./activate.sh`.

For work it need [uv](https://github.com/astral-sh/uv) installed.

Use [pre-commit](https://pre-commit.com/#install) hooks for code quality:

    pre-commit install

## Allure test report

* [Allure report](https://andgineer.github.io/llmbroker/builds/tests/)

# Scripts
Install [invoke](https://docs.pyinvoke.org/en/stable/) preferably with [uv tool](https://docs.astral.sh/uv/):

    uv tool install invoke

For a list of available scripts run:

    invoke --list

For more information about a script run:

    invoke <script> --help

## Coverage report
* [Codecov](https://app.codecov.io/gh/andgineer/llmbroker/tree/main/src%2Fllmbroker)
* [Coveralls](https://coveralls.io/github/andgineer/llmbroker)

> Created with cookiecutter using [template](https://github.com/andgineer/cookiecutter-python-package)
