Metadata-Version: 2.4
Name: llm-switchboard
Version: 0.1.0
Summary: A provider-agnostic Python interface for routing generation calls across LiteLLM, OpenAI, Ollama, and Watsonx
Author-email: Osher Elhadad <Osher.Elhadad@ibm.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/skillberry-ai/llm-switchboard
Project-URL: Repository, https://github.com/skillberry-ai/llm-switchboard
Project-URL: Issues, https://github.com/skillberry-ai/llm-switchboard/issues
Keywords: llm,openai,watsonx,litellm,ollama,model-routing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.0.0
Requires-Dist: jsonschema>=4.23.0
Requires-Dist: rich>=13.0.0
Requires-Dist: litellm<1.83.10,>=1.83.9
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == "openai"
Provides-Extra: ollama
Provides-Extra: watsonx
Requires-Dist: ibm-watsonx-ai>=1.0.0; extra == "watsonx"
Provides-Extra: rits
Provides-Extra: all
Requires-Dist: openai>=1.0.0; extra == "all"
Requires-Dist: ibm-watsonx-ai>=1.0.0; extra == "all"
Provides-Extra: dev
Requires-Dist: build>=1.4.4; extra == "dev"
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black>=24.0.0; extra == "dev"
Requires-Dist: isort>=5.12.0; extra == "dev"
Requires-Dist: flake8>=7.0.0; extra == "dev"
Requires-Dist: mypy>=1.19.1; extra == "dev"
Requires-Dist: pre-commit>=3.0.0; extra == "dev"
Requires-Dist: twine>=6.2.0; extra == "dev"
Dynamic: license-file

# LLM Switchboard

LLM Switchboard is a small Python library for calling different model providers through one consistent interface. It supports synchronous and asynchronous generation, tool calls, structured output validation, provider-specific parameter mapping, and configurable logging.

## Installation

```bash
pip install llm-switchboard
```

For local development:

```bash
git clone git@github.com:skillberry-ai/llm-switchboard.git
cd llm-switchboard
pip install -e ".[dev,all]"
```

LiteLLM is installed by default. Additional provider extras are available when you need direct SDK support:

```bash
pip install "llm-switchboard[openai]"
pip install "llm-switchboard[watsonx]"
pip install "llm-switchboard[all]"
```

## Quick Start

```python
from llm_switchboard.llm import get_llm

LiteLLMClient = get_llm("litellm")
client = LiteLLMClient(model_name="openai/gpt-4o-mini")

response = client.generate("Write a short explanation of retrieval augmented generation.")
print(response)
```

You can also select a provider from environment variables:

```bash
export LLM_PROVIDER=litellm
export MODEL_NAME=openai/gpt-4o-mini
export OPENAI_API_KEY=...
```

```python
from llm_switchboard.llm import get_llm

client = get_llm("auto_from_env")()
print(client.generate("Summarize the role of a model gateway."))
```

If `LLM_PROVIDER` is not set, `auto_from_env` defaults to `litellm`. Providers that need a model name still require `MODEL_NAME`.

## Providers

### LiteLLM

LiteLLM is the default provider and works with any LiteLLM-supported model string.

```python
from llm_switchboard.llm import get_llm

client = get_llm("litellm")(model_name="openai/gpt-4o-mini")
response = client.generate(
    [{"role": "user", "content": "Return three release checklist items."}],
    temperature=0.2,
)
```

Registered names:

- `litellm`
- `litellm.output_val`

### OpenAI and Azure OpenAI

Install the direct OpenAI SDK extra when you want to use OpenAI clients without the LiteLLM adapter.

```bash
pip install "llm-switchboard[openai]"
```

```python
from llm_switchboard.llm import get_llm

client = get_llm("openai.sync")(api_key="...")
response = client.generate("Explain semantic versioning.", model="gpt-4o-mini")
```

Registered names:

- `openai.sync`
- `openai.async`
- `openai.sync.output_val`
- `openai.async.output_val`
- `azure_openai.sync`
- `azure_openai.async`
- `azure_openai.sync.output_val`
- `azure_openai.async.output_val`

### Ollama

Ollama uses the LiteLLM dependency that is installed by default.

```python
from llm_switchboard.llm import get_llm

client = get_llm("litellm.ollama")(
    model_name="llama3.1",
    api_base="http://localhost:11434",
)
print(client.generate("Give me one practical use for local models."))
```

Registered names:

- `litellm.ollama`
- `litellm.ollama.output_val`

### Watsonx

Watsonx can be used through LiteLLM or the native SDK.

```bash
pip install "llm-switchboard[watsonx]"
```

```python
from llm_switchboard.llm import get_llm

client = get_llm("litellm.watsonx")(
    model_name="meta-llama/llama-3-3-70b-instruct",
    api_key="...",
    project_id="...",
)
```

Registered names:

- `litellm.watsonx`
- `litellm.watsonx.output_val`
- `watsonx`
- `watsonx.output_val`

## Structured Output

Use an output-validation provider when you want a Pydantic model, JSON schema, or built-in type back from the response.

```python
from pydantic import BaseModel

from llm_switchboard.llm import get_llm


class Task(BaseModel):
    title: str
    priority: str


client = get_llm("litellm.output_val")(model_name="openai/gpt-4o-mini")
task = client.generate(
    "Create one task for preparing a package release.",
    output_schema=Task,
)
print(task.title, task.priority)
```

## Generation Arguments

`GenerationArgs` provides provider-agnostic parameters such as `max_tokens`, `temperature`, `top_p`, `seed`, `timeout`, `stop_sequences`, and `stream`.

```python
from llm_switchboard.llm import get_llm
from llm_switchboard.llm.types import GenerationArgs

client = get_llm("litellm")(model_name="openai/gpt-4o-mini")
args = GenerationArgs(max_tokens=120, temperature=0.3, seed=7)

response = client.generate("Draft a concise changelog entry.", generation_args=args)
```

Each provider maps supported arguments to its native SDK shape and ignores unsupported values with a warning where appropriate.

## Logging

```python
from llm_switchboard import configure_logging

configure_logging(level="INFO")
```

Environment variables:

- `LLM_SWITCHBOARD_LOG_LEVEL`
- `LLM_SWITCHBOARD_LOG_DIR`

Sensitive keys such as API keys, tokens, credentials, and authorization headers are masked before logging.

## Examples

Example scripts live in `llm_examples/`:

- `litellm_ollama_example.py`
- `azure_openai_example.py`
- `litellm_watsonx_example.py`
- `ibm_watsonx_ai_example.py`
- `litellm_rits_example.py`
- `litellm_ibm_example.py`

Run an example from the repository root:

```bash
python llm_examples/litellm_ollama_example.py
```

## Enterprise/Internal Integrations

RITS and IBM ETE LiteLLM adapters are included for internal IBM services.

RITS:

- Provider names: `litellm.rits`, `litellm.rits.output_val`
- Environment variables: `RITS_API_KEY`, `RITS_API_URL`

IBM ETE LiteLLM:

- Provider names: `litellm.ibm`, `litellm.ibm.output_val`
- Environment variables: `IBM_THIRD_PARTY_API_KEY`, `IBM_LITELLM_API_BASE`

## Development

```bash
pip install -e ".[dev,all]"
make lint
make coverage
make build
```

Release builds are created with:

```bash
python -m build
twine check dist/*
```

## License

Apache-2.0
