Metadata-Version: 2.4
Name: llmbridgekit
Version: 1.0.4
Summary: Universal Python LLM Provider Abstraction Library
Author-email: sreeyenan <sreeyenanek@gmail.com>
Project-URL: Homepage, https://github.com/sreeyenan/llmbridgekit
Project-URL: Documentation, https://github.com/sreeyenan/llmbridgekit#readme
Project-URL: Repository, https://github.com/sreeyenan/llmbridgekit
Project-URL: Issues, https://github.com/sreeyenan/llmbridgekit/issues
Keywords: llm,openai,gemini,anthropic,claude,gpt,ai,ml,nlp,chatgpt,language-model,provider-abstraction
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: pydantic<3.0,>=2.0
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == "openai"
Provides-Extra: gemini
Requires-Dist: google-genai>=0.1.0; extra == "gemini"
Provides-Extra: groq
Requires-Dist: groq>=0.4.0; extra == "groq"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.18.0; extra == "anthropic"
Provides-Extra: mistral
Requires-Dist: mistralai>=0.1.0; extra == "mistral"
Provides-Extra: cohere
Requires-Dist: cohere>=5.0.0; extra == "cohere"
Provides-Extra: local
Requires-Dist: httpx>=0.25.0; extra == "local"
Provides-Extra: yaml
Requires-Dist: pyyaml>=6.0; extra == "yaml"
Provides-Extra: validation
Requires-Dist: jsonschema>=4.19.0; extra == "validation"
Provides-Extra: http
Requires-Dist: httpx>=0.25.0; extra == "http"
Requires-Dist: requests>=2.31.0; extra == "http"
Provides-Extra: azure
Requires-Dist: llmbridgekit[openai]; extra == "azure"
Provides-Extra: ollama
Requires-Dist: llmbridgekit[local]; extra == "ollama"
Provides-Extra: lmstudio
Requires-Dist: llmbridgekit[local]; extra == "lmstudio"
Provides-Extra: openrouter
Requires-Dist: llmbridgekit[local]; extra == "openrouter"
Provides-Extra: huggingface
Requires-Dist: llmbridgekit[local]; extra == "huggingface"
Provides-Extra: together
Requires-Dist: llmbridgekit[local]; extra == "together"
Provides-Extra: cloud-providers
Requires-Dist: llmbridgekit[anthropic,cohere,gemini,groq,mistral,openai]; extra == "cloud-providers"
Provides-Extra: local-providers
Requires-Dist: llmbridgekit[lmstudio,ollama]; extra == "local-providers"
Provides-Extra: routers
Requires-Dist: llmbridgekit[huggingface,openrouter,together]; extra == "routers"
Provides-Extra: dev
Requires-Dist: pytest>=7.4.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
Requires-Dist: mypy>=1.6.0; extra == "dev"
Requires-Dist: black>=23.10.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Provides-Extra: all
Requires-Dist: llmbridgekit[cloud-providers,dev,http,local-providers,routers,validation,yaml]; extra == "all"

# LLMBridgeKit

**Version:** 1.0.0  
**Author:** [sreeyenan](https://github.com/sreeyenan)  
**License:** MIT  
**Python:** 3.10+

**Universal Python LLM Provider Abstraction Library**

A provider-agnostic Python library that lets any AI feature call multiple LLM providers through one consistent interface.

[![PyPI version](https://badge.fury.io/py/llmbridgekit.svg)](https://pypi.org/project/llmbridgekit/)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

## Features

### 🌐 Multi-Provider Support
- **Cloud Providers**: OpenAI, Azure OpenAI, Google Gemini, Anthropic Claude, Mistral AI, Cohere
- **Fast Inference**: Groq, Together AI
- **Model Routers**: OpenRouter, Hugging Face Inference Providers
- **Local Models**: Ollama, LM Studio
- **Custom**: Your own HTTP endpoint

### 🎯 Core Capabilities
- **Unified API**: Single interface for all providers
- **Text Generation**: Simple `.generate()` method
- **Chat**: Message-based `.chat()` method
- **Structured JSON**: `.generate_json()` with schema validation
- **Tool Calling**: `.generate_with_tools()` for function calling
- **Streaming**: Real-time token streaming
- **Async Support**: Full async/await support

### 🛡️ Reliability Features
- **Fallback Chains**: Automatic provider failover
- **Retry Logic**: Exponential backoff with jitter
- **Timeout Control**: Per-request and per-provider timeouts
- **Error Normalization**: Consistent error handling across providers
- **Rate Limiting**: Respect provider rate limits

### 📊 Observability
- **Usage Tracking**: Token counts and model usage
- **Cost Estimation**: Calculate API costs
- **Latency Metrics**: Track request duration
- **Audit Logging**: Request/response logging with redaction
- **Request IDs**: Distributed tracing support

### 🔒 Security
- **PII Redaction**: Mask sensitive data before sending
- **Secret Management**: Environment variable support
- **API Key Rotation**: Hot-reload credentials

## Installation

```bash
# Core installation
pip install llmbridgekit

# With specific providers
pip install llmbridgekit[openai]
pip install llmbridgekit[gemini]
pip install llmbridgekit[anthropic]

# With multiple providers
pip install llmbridgekit[openai,gemini,groq]

# Local model support
pip install llmbridgekit[local]

# Everything
pip install llmbridgekit[all]
```

## Quick Start

### Basic Text Generation

```python
from llmbridgekit import LLMClient

client = LLMClient(provider="gemini")

response = client.generate(
    prompt="Explain what a materialized view is in ClickHouse."
)

print(response.text)
print(f"Tokens: {response.usage['total_tokens']}")
```

### Chat with Messages

```python
response = client.chat(
    messages=[
        {"role": "system", "content": "You are a data analytics assistant."},
        {"role": "user", "content": "Show revenue by region."}
    ],
    provider="openai",
    model="gpt-4o"
)

print(response.text)
```

### Structured JSON Output

```python
schema = {
    "type": "object",
    "properties": {
        "intent": {
            "type": "string",
            "enum": ["query", "chart", "dashboard", "report"]
        },
        "confidence": {"type": "number", "minimum": 0, "maximum": 1},
        "entities": {
            "type": "array",
            "items": {"type": "string"}
        }
    },
    "required": ["intent", "confidence"]
}

response = client.generate_json(
    prompt="User asks: 'Show me sales by region for last quarter'",
    schema=schema
)

print(response.json_data)
# {'intent': 'query', 'confidence': 0.95, 'entities': ['sales', 'region', 'last quarter']}
```

### Streaming

```python
for chunk in client.stream(
    prompt="Write a detailed analysis of Q4 performance.",
    provider="anthropic"
):
    print(chunk.text, end="", flush=True)
```

### Fallback Chain

```python
response = client.generate(
    prompt="Generate a SQL query for revenue by category.",
    fallback_chain=[
        {"provider": "gemini", "model": "gemini-2.0-flash-exp"},
        {"provider": "groq", "model": "llama-3.3-70b-versatile"},
        {"provider": "ollama", "model": "qwen2.5:3b"}
    ]
)
```

### Tool Calling

```python
tools = [
    {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"}
            },
            "required": ["location"]
        }
    }
]

response = client.generate_with_tools(
    prompt="What's the weather in San Francisco?",
    tools=tools,
    provider="openai"
)

if response.tool_calls:
    for tool_call in response.tool_calls:
        print(f"Tool: {tool_call['name']}")
        print(f"Args: {tool_call['arguments']}")
```

## Configuration

### YAML Configuration

```yaml
# config.yaml
default_provider: gemini
default_timeout: 60
default_retries: 2

providers:
  openai:
    api_key_env: OPENAI_API_KEY
    model: gpt-4o
    temperature: 0
    max_tokens: 1000

  gemini:
    api_key_env: GEMINI_API_KEY
    model: gemini-2.0-flash-exp
    temperature: 0

  ollama:
    base_url: http://localhost:11434
    model: qwen2.5:3b
    temperature: 0

fallback_chains:
  default:
    - provider: gemini
      model: gemini-2.0-flash-exp
    - provider: groq
      model: llama-3.3-70b-versatile

tasks:
  classification:
    provider: groq
    model: llama-3.3-70b-versatile
    temperature: 0
    max_tokens: 500

  text_to_sql:
    provider: gemini
    model: gemini-2.0-flash-exp
    temperature: 0
    fallback_chain: default
```

Load configuration:

```python
from llmbridgekit import LLMClient

client = LLMClient.from_yaml("config.yaml")

# Use task-based routing
response = client.run_task(
    task="text_to_sql",
    prompt="Show me revenue by product category"
)
```

## Supported Providers

| Provider | Structured Output | Tool Calling | Streaming | Local | Status |
|----------|:-----------------:|:------------:|:---------:|:-----:|--------|
| OpenAI | ✅ | ✅ | ✅ | ❌ | Stable |
| Azure OpenAI | ✅ | ✅ | ✅ | ❌ | Stable |
| Google Gemini | ✅ | ✅ | ✅ | ❌ | Stable |
| Anthropic Claude | ✅ | ✅ | ✅ | ❌ | Stable |
| Groq | ✅ | ✅ | ✅ | ❌ | Stable |
| Mistral AI | ✅ | ✅ | ✅ | ❌ | Stable |
| Cohere | ✅ | ✅ | ✅ | ❌ | Stable |
| Ollama | ✅ | ✅ | ✅ | ✅ | Stable |
| LM Studio | ✅ | ✅ | ✅ | ✅ | Stable |
| OpenRouter | ✅ | ✅ | ✅ | ❌ | Stable |
| Hugging Face | ✅ | ✅ | ✅ | ❌ | Stable |
| Together AI | ✅ | ✅ | ✅ | ❌ | Stable |
| Custom HTTP | ✅ | ✅ | ✅ | ⚠️ | Stable |

## Use Cases

### NLQ (Natural Language Query)
```python
# Context resolution
response = client.generate_json(
    prompt=context_resolver_prompt,
    schema=context_resolution_schema,
    provider="ollama",  # Local fallback
    model="qwen2.5:3b"
)

# Text-to-SQL generation
response = client.generate(
    prompt=sql_generation_prompt,
    provider="gemini",
    fallback_chain="default"
)
```

### RAG (Retrieval Augmented Generation)
```python
# Generate embeddings
embeddings = client.embed(
    texts=["chunk1", "chunk2", "chunk3"],
    provider="openai"
)

# Generate answer with context
response = client.chat(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": f"Context: {context}\n\nQuestion: {question}"}
    ]
)
```

### Chart/Dashboard Generation
```python
response = client.generate_json(
    prompt=f"Generate chart config for: {user_request}",
    schema=chart_config_schema,
    provider="openai",
    model="gpt-4o"
)

chart_config = response.json_data
```

### Document Q&A
```python
response = client.chat(
    messages=[
        {"role": "system", "content": "Answer based on the document."},
        {"role": "user", "content": f"Document: {document}\n\nQuestion: {question}"}
    ],
    provider="anthropic",
    model="claude-opus-4"
)
```

## Advanced Features

### Async Support

```python
import asyncio
from llmbridgekit import LLMClient

async def main():
    client = LLMClient(provider="gemini")
    
    response = await client.agenerate(
        prompt="Explain async/await in Python"
    )
    
    print(response.text)

asyncio.run(main())
```

### Cost Tracking

```python
response = client.generate(
    prompt="Analyze sales data",
    provider="openai",
    model="gpt-4o"
)

print(f"Cost: ${response.cost['total_cost']:.4f}")
print(f"Input tokens: {response.usage['prompt_tokens']}")
print(f"Output tokens: {response.usage['completion_tokens']}")
```

### Redaction

```python
from llmbridgekit import LLMClient, RedactionConfig

client = LLMClient(
    provider="openai",
    redaction=RedactionConfig(
        mask_email=True,
        mask_phone=True,
        mask_api_keys=True,
        custom_patterns=[
            r'\b\d{3}-\d{2}-\d{4}\b'  # SSN
        ]
    )
)

response = client.generate(
    prompt="User email: john@example.com, phone: 555-1234"
)
# Prompt sent: "User email: [EMAIL_REDACTED], phone: [PHONE_REDACTED]"
```

### Custom Provider

```python
from llmbridgekit.providers import BaseProvider, ProviderFeatures
from llmbridgekit import provider_registry

class MyCustomProvider(BaseProvider):
    name = "my_custom"
    features = ProviderFeatures(
        chat=True,
        structured_outputs=True,
        streaming=True
    )
    
    def generate(self, prompt, config):
        # Your implementation
        response = self._call_api(prompt, config)
        return self._parse_response(response)

# Register
provider_registry.register("my_custom", MyCustomProvider)

# Use
client = LLMClient(provider="my_custom")
response = client.generate("Hello!")
```

## Testing

```bash
# Install dev dependencies
pip install llm-gateway[dev]

# Run tests
pytest

# Run with coverage
pytest --cov=llmbridgekit --cov-report=html

# Run specific provider tests
pytest tests/providers/test_openai_provider.py -v
```

## Documentation

- **[User Manual](llmbridgekit_USER_MANUAL.md)** - Complete usage guide
- **[Changelog](CHANGELOG.md)** - Version history
- **[Environment Variables](ENVIRONMENT_VARIABLES.md)** - Configuration reference
- **[Examples](examples/)** - Code examples for common use cases

## Architecture

```
Application Layer
    ↓
LLMClient (llmbridgekit/client.py)
    ↓
Task Router / Fallback Chain
    ↓
Provider Registry
    ↓
BaseProvider
    ↓
OpenAI | Gemini | Anthropic | Groq | ... | Custom HTTP
```

## License

MIT License - see LICENSE file for details.

## Support

- **Issues**: https://github.com/sreeyenan/llmbridgekit/issues
- **Discussions**: https://github.com/analytic-ai/llm-gateway/discussions
- **Documentation**: https://llm-gateway.readthedocs.io
