Metadata-Version: 2.5
Name: universal-llm-connector
Version: 1.0.0
Summary: A unified interface to all major LLM providers with system certificate and async support
Project-URL: Repository, https://github.com/rs2pydev/universal-llm-connector
Author: Tanmoy Dutta
License-Expression: MIT
License-File: LICENSE
Keywords: ai,anthropic,bedrock,connector,gemini,llm,openai
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: certifi
Requires-Dist: httpx>=0.27
Requires-Dist: pydantic>=2.0
Requires-Dist: tenacity>=8.0
Requires-Dist: truststore>=0.9
Provides-Extra: all
Requires-Dist: botocore>=1.30; extra == 'all'
Requires-Dist: httpx[socks]; extra == 'all'
Requires-Dist: huggingface-hub>=0.20; extra == 'all'
Provides-Extra: bedrock
Requires-Dist: botocore>=1.30; extra == 'bedrock'
Provides-Extra: dev
Requires-Dist: mypy>=1.5; extra == 'dev'
Requires-Dist: pre-commit>=3.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: huggingface
Requires-Dist: huggingface-hub>=0.20; extra == 'huggingface'
Provides-Extra: socks
Requires-Dist: httpx[socks]; extra == 'socks'
Description-Content-Type: text/markdown

# Universal LLM Connector

A Python SDK that provides a unified interface to all major LLM providers. Built on `httpx` for reliable HTTPS handling - system certificate support, custom base URLs, and async out of the box.

[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
[![PyPI](https://img.shields.io/pypi/v/universal-llm-connector.svg)](https://pypi.org/project/universal-llm-connector/)

---

## Problem Statement

Python developers face two recurring problems when working with LLM APIs:

**1. No unified interface across providers.** OpenAI, Anthropic, Google Gemini, Azure, AWS Bedrock, and others all have different request/response formats, auth methods, and endpoint patterns. Switching providers means rewriting integration code.

**2. SSL certificate issues in managed environments.** Python's HTTP libraries (`requests`, `urllib3`, `httpx`, `aiohttp`) use `certifi` - a static bundle of ~130 public CA certificates. If the OS trust store contains additional certificates, Python ignores them. Downloads, API calls through gateways, and pip installs from internal mirrors all fail with `SSLError: certificate verify failed`.

The existing solution (`litellm`) is built on `requests` (synchronous only, same SSL issues) and uses fragile model name decoding.

---

## Solution

`universal-llm-connector` provides:

1. A unified API for 10 LLM providers (sync + async, streaming, embeddings, tool calling, vision)
2. `corporate_fix()` - configures Python to use the OS certificate store instead of certifi's static bundle
3. `configure_huggingface()` - patches `huggingface_hub` so `transformers`, `diffusers`, and `accelerate` use system certificates

---

## Installation

Works on Windows, macOS, and Linux. Requires Python 3.10+.

```shell
pip install universal-llm-connector
```

With optional extras:

```shell
pip install universal-llm-connector[socks]       # SOCKS5 proxy support
pip install universal-llm-connector[bedrock]     # AWS Bedrock (SigV4 signing)
pip install universal-llm-connector[huggingface] # HuggingFace Hub patching
pip install universal-llm-connector[all]         # Everything
```

From source:

```shell
git clone https://github.com/rs2pydev/universal-llm-connector.git
cd universal-llm-connector
pip install -e ".[dev]"
```

---

## Quick Start

### Basic completion

```python
from universal_llm_connector import completion

response = completion(
    model="openai/gpt-4o",
    api_key="sk-...",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(response.content)
```

### Custom base URL (API gateways, self-hosted endpoints)

```python
response = completion(
    model="openai/gpt-4o",
    base_url="https://your-gateway.example.com/v1",
    api_key="your-token",
    use_system_certs=True,
    messages=[{"role": "user", "content": "Hello!"}],
)
```

### OpenAI Responses API

```python
response = completion(
    model="openai/gpt-4o",
    api="responses",
    api_key="sk-...",
    input="Explain quantum computing in one sentence.",
    instructions="Be concise.",
)
```

### Async

```python
import asyncio
from universal_llm_connector import acompletion

async def main():
    response = await acompletion(
        model="openai/gpt-4o",
        api_key="sk-...",
        messages=[{"role": "user", "content": "Hello!"}],
    )
    print(response.content)

asyncio.run(main())
```

### Streaming

```python
from universal_llm_connector import completion

for chunk in completion(
    model="openai/gpt-4o",
    api_key="sk-...",
    messages=[{"role": "user", "content": "Write a haiku."}],
    stream=True,
):
    print(chunk.content, end="", flush=True)
```

### Embeddings

```python
from universal_llm_connector import embed

response = embed(
    model="openai/text-embedding-3-small",
    input=["First sentence", "Second sentence"],
    api_key="sk-...",
)
vectors = response.embeddings
```

### Tool calling

```python
from universal_llm_connector import completion
from universal_llm_connector.models.messages import Tool, FunctionDef

weather_tool = Tool(
    function=FunctionDef(
        name="get_weather",
        description="Get weather for a city",
        parameters={
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    )
)

response = completion(
    model="openai/gpt-4o",
    api_key="sk-...",
    messages=[{"role": "user", "content": "Weather in Tokyo?"}],
    tools=[weather_tool],
)
```

### Vision (multimodal)

```python
response = completion(
    model="openai/gpt-4o",
    api_key="sk-...",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image."},
            {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
        ],
    }],
)
```

### Reusable client (connection pooling)

```python
from universal_llm_connector import UniversalClient

with UniversalClient(
    base_url="https://your-gateway.example.com/v1",
    api_key="token",
    use_system_certs=True,
) as client:
    r1 = client.completion(model="openai/gpt-4o", messages=[...])
    r2 = client.completion(model="openai/gpt-4o-mini", messages=[...])
    embedding = client.embed(model="openai/text-embedding-3-small", input="hello")
```

---

## SSL Certificate Fix

Python's `certifi` uses a static CA bundle that does not include certificates from the OS trust store. This causes SSL failures in environments where additional CAs are installed at the OS level.

### Fix for all Python HTTP libraries

```python
from universal_llm_connector import corporate_fix

corporate_fix()
```

This exports the OS certificate store to a PEM file and configures `requests`, `urllib3`, `httpx`, `aiohttp`, `pip`, and `git` to use it via environment variables and session patching.

### Fix for HuggingFace

```python
from universal_llm_connector import configure_huggingface

configure_huggingface()

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
```

### Advanced configuration

```python
corporate_fix(
    ca_bundle="/path/to/custom-ca.pem",
    proxy="http://proxy.example.com:8080",
    hf_endpoint="https://hf.example.com",
    hf_token="hf_xxxxx",
    pip_index_url="https://pypi.example.com/simple",
    verbose=True,
)
```

---

## Supported Providers

| Provider | Model format | Default base URL |
| --- | --- | --- |
| OpenAI | `openai/gpt-4o` | `https://api.openai.com/v1` |
| Anthropic | `anthropic/claude-sonnet-4-20250514` | `https://api.anthropic.com/v1` |
| Azure OpenAI | `azure/my-deployment` | (requires base_url) |
| Google Gemini | `gemini/gemini-1.5-pro` | `https://generativelanguage.googleapis.com/v1beta` |
| AWS Bedrock | `bedrock/anthropic.claude-3-sonnet` | (requires base_url) |
| Groq | `groq/llama-3.1-70b` | `https://api.groq.com/openai/v1` |
| Mistral | `mistral/mistral-large-latest` | `https://api.mistral.ai/v1` |
| GitHub Models | `github/gpt-4o` | `https://models.inference.ai.azure.com` |
| Ollama | `ollama/llama3.1` | `http://localhost:11434` |
| HuggingFace | `huggingface/meta-llama/Llama-3.1-8B` | `https://api-inference.huggingface.co` |

All providers support custom `base_url` for self-hosted or gateway endpoints.

---

## API Reference

### completion() / acompletion()

| Parameter | Type | Description |
| --- | --- | --- |
| `model` | `str` | Required. Format: `provider/model-name` |
| `messages` | `list` | Conversation messages |
| `input` | `str` or `list` | Input for OpenAI Responses API |
| `api` | `str` | `"chat"` (default) or `"responses"` |
| `stream` | `bool` | Enable streaming (default: False) |
| `base_url` | `str` | Provider or gateway URL |
| `api_key` | `str` | Authentication key |
| `timeout` | `float` | Timeout in seconds (default: 60) |
| `max_retries` | `int` | Retries on 429/5xx (default: 3) |
| `use_system_certs` | `bool` | Use OS certificate store (default: False) |
| `tools` | `list` | Tool/function definitions |

### embed() / aembed()

| Parameter | Type | Description |
| --- | --- | --- |
| `model` | `str` | Required. Format: `provider/model-name` |
| `input` | `str` or `list[str]` | Required. Text(s) to embed |
| `base_url` | `str` | Provider or gateway URL |
| `api_key` | `str` | Authentication key |

### ChatResponse

| Property | Type | Description |
| --- | --- | --- |
| `.content` | `str` | Generated text (first choice) |
| `.finish_reason` | `str` | `"stop"`, `"length"`, `"tool_calls"` |
| `.usage.total_tokens` | `int` | Total tokens |
| `.model` | `str` | Model that served the request |
| `.raw` | `dict` | Full provider response |

### EmbedResponse

| Property | Type | Description |
| --- | --- | --- |
| `.embedding` | `list[float]` | First embedding vector |
| `.embeddings` | `list[list[float]]` | All vectors (batch input) |

---

## Error Handling

```python
from universal_llm_connector.exceptions import (
    AuthenticationError,    # 401/403
    RateLimitError,         # 429, includes retry_after
    ModelNotFoundError,     # 404
    ContextLengthError,     # Input too long
    NetworkError,           # Connection/timeout
    SSLCertificateError,    # Cert verification failed
    InvalidRequestError,    # 400
)
```

---

## Development

```shell
git clone https://github.com/rs2pydev/universal-llm-connector.git
cd universal-llm-connector
pip install -e ".[dev]"

pytest                        # 81 unit tests
ruff check src/ tests/        # Lint
mypy src/                     # Type check
python -m build               # Build wheel + sdist
```

---

## Requirements

- Python 3.10+
- httpx >= 0.27
- pydantic >= 2.0
- tenacity >= 8.0
- truststore >= 0.9
- certifi

---

## License

MIT
