Metadata-Version: 2.4
Name: generic-llm-api-client
Version: 0.5.2
Summary: A unified, provider-agnostic Python client for multiple LLM APIs
Author-email: Sorin Marti <sorin.marti@gmail.com>
Maintainer-email: Sorin Marti <sorin.marti@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/RISE-UNIBAS/generic_llm_api_client
Project-URL: Documentation, https://github.com/RISE-UNIBAS/generic_llm_api_client#readme
Project-URL: Repository, https://github.com/RISE-UNIBAS/generic_llm_api_client
Project-URL: Issues, https://github.com/RISE-UNIBAS/generic_llm_api_client/issues
Keywords: llm,ai,openai,anthropic,claude,gemini,mistral,deepseek,qwen,cohere,huggingface,api-client,humanities,research,benchmarking
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: anthropic~=0.84.0
Requires-Dist: openai~=2.26.0
Requires-Dist: mistralai~=2.0.4
Requires-Dist: google-genai~=1.67.0
Requires-Dist: cohere~=5.20.7
Requires-Dist: requests~=2.32.5
Requires-Dist: geonamescache~=3.0.0
Provides-Extra: dev
Requires-Dist: pytest>=9.0.2; extra == "dev"
Requires-Dist: pytest-asyncio>=0.26.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
Requires-Dist: pytest-mock>=3.15.1; extra == "dev"
Requires-Dist: python-dotenv>=1.2.2; extra == "dev"
Requires-Dist: Pillow>=12.1.1; extra == "dev"
Requires-Dist: black>=26.3.0; extra == "dev"
Requires-Dist: mypy>=1.19.1; extra == "dev"
Requires-Dist: ruff>=0.15.5; extra == "dev"
Provides-Extra: test
Requires-Dist: pytest>=9.0.2; extra == "test"
Requires-Dist: pytest-asyncio>=0.26.0; extra == "test"
Requires-Dist: pytest-cov>=4.1.0; extra == "test"
Requires-Dist: pytest-mock>=3.15.1; extra == "test"
Requires-Dist: python-dotenv>=1.2.2; extra == "test"
Requires-Dist: Pillow>=12.1.1; extra == "test"
Dynamic: license-file

# Generic LLM API Client

A unified, provider-agnostic Python client for multiple LLM APIs. Access supported LLMs (OpenAI, Anthropic Claude, Google Gemini, Mistral, DeepSeek, Qwen, OpenRouter, and more) through a single, consistent interface.

Designed for research workflows, benchmarking, automated testing, and applications that require a consistent interface across LLM providers.

## Scope

This package is a **convenience wrapper** for working with multiple LLM providers through a unified interface. It is **not intended as a replacement** for the official provider libraries (openai, anthropic, google-genai, etc.).

### Use this package when:
- You need to query multiple LLM providers in the same project
- You are building benchmarking or comparison tools
- You want a consistent interface across providers
- You need provider-agnostic code for research workflows

### Use the official libraries when:

- You need new features as soon as they are released
- You require provider-specific advanced features
- You only work with a single provider

**Update pace:** This package is maintained by a small team and may not immediately support every new feature from upstream providers. We prioritize stability and cross-provider compatibility over immediate coverage of new features.

## Features

- **Provider-Agnostic**: Single interface for OpenAI, Anthropic, Google, Mistral, Cohere, DeepSeek, Qwen, xAI, HuggingFace and OpenRouter
- **Tool Calling (Beta)**: Let LLMs use external tools with automatic execution
- **Multimodal Support**: Text + images across all supporting providers
- **Text File Support**: Automatically include text files in prompts for document analysis
- **Automatic Image Resizing**: Reduce API costs by auto-resizing large images
- **Structured Output**: Unified Pydantic model support across providers
- **Rich Response Objects**: Detailed token usage, costs, timing, and metadata
- **Cost Accounting**: Records reasoning tokens billed outside the completion, tracks discarded fallback attempts, and preserves provider-billed costs
- **Model-Aware Requests**: Adapts parameters for models that have renamed or removed supported settings
- **Async Support**: Parallel processing for faster benchmarks
- **Built-in Retry Logic**: Automatic exponential backoff for rate limits
- **Custom Base URLs**: Integration with OpenRouter, sciCORE, and other OpenAI-compatible APIs

## Installation

```bash
pip install generic-llm-api-client
```

## Quick Start

```python
from ai_client import create_ai_client

# Create a client for any provider
client = create_ai_client('openai', api_key='sk-...')

# Send a prompt
response = client.prompt('gpt-4', 'What is 2+2?')

print(f"Response: {response.text}")
print(f"Tokens used: {response.usage.total_tokens}")
print(f"Time: {response.duration:.2f}s")
```

## Supported Providers

| Provider | ID | Multimodal | Structured Output | Tool Calling |
|----------|-----|-----------|-------------------|--------------|
| OpenAI | `openai` | Yes | Yes | Yes (Beta) |
| Anthropic Claude | `anthropic` | Yes | Yes (via tools) | Planned |
| Google Gemini | `genai` | Yes | Yes | No |
| Mistral | `mistral` | Yes | Yes | No |
| DeepSeek | `deepseek` | Yes | Yes | Via OpenAI |
| Qwen | `qwen` | Yes | Yes | Via OpenAI |
| HuggingFace | `huggingface` | Yes | Yes | Via OpenAI |
| OpenRouter | `openrouter` | Yes | Yes | Via OpenAI |
| sciCORE | `scicore` | Yes | Yes | Via OpenAI |

## Usage Examples

### Basic Text Prompt

```python
from ai_client import create_ai_client

client = create_ai_client('anthropic', api_key='sk-ant-...')
response, duration = client.prompt(
    'claude-3-5-sonnet-20241022',
    'Explain quantum computing in simple terms'
)

print(response.text)
```

### Multimodal (Text + Images)

```python
from ai_client import create_ai_client

client = create_ai_client('openai', api_key='sk-...')

response, duration = client.prompt(
    'gpt-4o',
    'Describe this image in detail',
    images=['path/to/image.jpg']
)

print(response.text)
```

### Multiple Images

```python
response, duration = client.prompt(
    'gpt-4o',
    'Compare these two images',
    images=['image1.jpg', 'image2.jpg']
)
```

### Text Files (Added in v0.2.0)

Include text files in your prompts for document analysis:

```python
from ai_client import create_ai_client

client = create_ai_client('openai', api_key='sk-...')

# Analyze a single text file
response, duration = client.prompt(
    'gpt-4o',
    'Summarize this historical document',
    files=['manuscript_transcription.txt']
)

# Analyze multiple documents
response, duration = client.prompt(
    'gpt-4o',
    'Compare these two texts and identify common themes',
    files=['document1.txt', 'document2.txt']
)

print(response.text)
```

### Automatic Image Resizing (Added in v0.2.0)

Reduce API costs by automatically resizing large images:

```python
from ai_client import create_ai_client

# Enable auto-resize (default: 2048px max dimension)
client = create_ai_client(
    'openai',
    api_key='sk-...',
    max_image_size=2048,  # Images larger than this will be resized
    image_quality=85       # JPEG quality for resized images
)

# This 4000x3000 image will be automatically resized to 2048x1536
response, duration = client.prompt(
    'gpt-4o',
    'Analyze this high-resolution historical manuscript',
    images=['huge_manuscript_scan.jpg']  # Original file is never modified
)

# Disable resizing if needed
client = create_ai_client('openai', api_key='sk-...', max_image_size=None)
```

### Combining Files and Images (Added in v0.2.0)

Compare visual and textual sources in humanities research:

```python
from ai_client import create_ai_client

client = create_ai_client('openai', api_key='sk-...')

# Compare image to text description
response, duration = client.prompt(
    'gpt-4o',
    'Does this manuscript image match the catalog description?',
    images=['manuscript_photo.jpg'],
    files=['catalog_entry.txt']
)

# Analyze multiple sources together
response, duration = client.prompt(
    'gpt-4o',
    'Which of these paintings best matches the art historical description?',
    images=['painting_a.jpg', 'painting_b.jpg', 'painting_c.jpg'],
    files=['art_history_text.txt']
)

print(response.text)
```

### Structured Output with Pydantic

```python
from pydantic import BaseModel
from ai_client import create_ai_client

class Person(BaseModel):
    name: str
    age: int
    occupation: str

client = create_ai_client('openai', api_key='sk-...')

response, duration = client.prompt(
    'gpt-4',
    'Extract: John Smith is a 35-year-old software engineer',
    response_format=Person
)

# Parse the response
import json
person_data = json.loads(response.text)
person = Person(**person_data)

print(f"{person.name}, {person.age}, {person.occupation}")
```

Claude uses a forced tool for structured output where supported. Models that reject
forced tools receive the original schema through an optional tool. If that attempt
fails, the client falls back to plain text. Text responses to optional tools are
accepted. Validate the result with your Pydantic model when schema compliance is required.

### Tool Calling (Beta)

Allow models to call external tools:

```python
from ai_client import create_ai_client

client = create_ai_client('openai', api_key='sk-...')

# Single tool usage
response = client.prompt(
    'gpt-4o',
    'What is the GeoNames ID of Zurich, Switzerland?',
    tool='GeonamesSearch'
)

print(response.text)  # "The GeoNames ID of Zurich is 2657896"

# Tool execution details
if response.tool_calls:
    print(f"Tools called: {[tc['name'] for tc in response.tool_calls]}")
if response.tool_results:
    print(f"Results: {len(response.tool_results)} tool(s) executed")
```

**Features:**
- **Provider-agnostic**: Same tool definitions work across OpenAI and Claude
- **Automatic execution**: Tools are called and results returned automatically
- **Single-round**: LLM can call tools once per request (multi-round support planned)
- **Pluggable executors**: Support for Python functions, REST APIs, and MCP servers

**Current status (proof of concept):**
- Supported: OpenAI (GPT-4o, GPT-4, etc.)
- Planned: Claude support
- Available built-in tool: GeonamesSearch
- Planned: REST API and MCP executors

See `example_tool_calling.py` for more examples.

### Async for Parallel Processing

```python
import asyncio
from ai_client import create_ai_client

async def process_batch():
    client = create_ai_client('openai', api_key='sk-...')

    # Process multiple prompts in parallel
    tasks = [
        client.prompt_async('gpt-4', f'Tell me about {topic}')
        for topic in ['Python', 'JavaScript', 'Rust']
    ]

    results = await asyncio.gather(*tasks)

    for response, duration in results:
        print(f"({duration:.2f}s) {response.text[:100]}...")

asyncio.run(process_batch())
```

### Custom Base URLs (OpenRouter, sciCORE)

```python
from ai_client import create_ai_client

# OpenRouter - access to 100+ models
client = create_ai_client(
    'openrouter',
    api_key='sk-or-...',
    base_url='https://openrouter.ai/api/v1',
    default_headers={
        "HTTP-Referer": "https://your-site.com",
        "X-Title": "Your App"
    }
)

response, _ = client.prompt('anthropic/claude-3-opus', 'Hello!')

# sciCORE (University HPC)
client = create_ai_client(
    'scicore',
    api_key='your-key',
    base_url='https://llm-api-h200.ceda.unibas.ch/v1'
)

response, _ = client.prompt('deepseek/deepseek-chat', 'Hello!')
```

### HuggingFace

Access HuggingFace through the `huggingface` provider (alias: `hf`), which covers two
deployment options.

**1. Inference Providers router (default).** One HuggingFace token gives access to the open-weight
models in the [Inference Providers catalog](https://huggingface.co/models?inference_provider=all),
executed by partner providers (Groq, Together, Cerebras, Novita, and others). No `base_url` is required;
pass a catalog model ID as the model:

```python
from ai_client import create_ai_client

client = create_ai_client('huggingface', api_key='hf_...')

response = client.prompt('deepseek-ai/DeepSeek-V3.1', 'Hello!')
print(response.text)
```

Model IDs are Hub repository IDs and accept an optional routing suffix:

| Suffix | Effect |
|--------|--------|
| *(none)* / `:fastest` | Highest-throughput provider (the default) |
| `:cheapest` | Lowest price per output token |
| `:preferred` | Your provider preference order from Hub settings |
| `:<name>` | Pin one provider, e.g. `:groq`, `:together` |

```python
# Pin the serving provider for reproducible capabilities
response = client.prompt('deepseek-ai/DeepSeek-V3.1:novita', 'Hello!')
```

Capabilities vary by provider: not every provider backing a given model
supports strict structured output or tool calling. When a provider does not, the client falls back
to JSON mode with the schema in the prompt.

To bill an organization rather than your personal account, pass HuggingFace's header through the
existing `default_headers` setting:

```python
client = create_ai_client(
    'huggingface',
    api_key='hf_...',
    default_headers={"X-HF-Bill-To": "my-org-name"}
)
```

Cost is populated only when the model id **pins a provider** (e.g.
`swiss-ai/Apertus-v1.5-8B:publicai`). A bare router id routes to whichever partner is
fastest, and multiple partners may serve the same model at different prices, so
`usage.estimated_cost_usd` stays `None`; see [PRICING.md](PRICING.md). Token counts are
tracked in both cases.

**2. Dedicated Inference Endpoints.** For any Hub model the router does not serve, deploy it to
your own endpoint and pass its URL. Note that the `model` argument is then the *endpoint name*,
not the Hub repository ID:

```python
client = create_ai_client(
    'huggingface',
    api_key='hf_...',
    base_url='https://abc123.us-east-1.aws.endpoints.huggingface.cloud/v1'
)

response = client.prompt('my-endpoint-name', 'Hello!')  # endpoint name, not repo id
```

Dedicated endpoints scale to zero after an hour of inactivity, with a 3-5 minute cold start; the
built-in retry is fixed at 3 attempts with a 60s maximum backoff, so the first call to an idle
endpoint may return an error response. Retry after the endpoint has started.

### Accessing Response Metadata

```python
response, duration = client.prompt('gpt-4', 'Hello')

# Response text
print(response.text)

# Token usage
print(f"Input tokens: {response.usage.input_tokens}")
print(f"Output tokens: {response.usage.output_tokens}")
print(f"Total tokens: {response.usage.total_tokens}")

# Reasoning a provider billed outside output_tokens (None if it reported no total).
# input + output + reasoning == total wherever a total was reported.
print(f"Reasoning tokens: {response.usage.reasoning_tokens}")

# Metadata
print(f"Model: {response.model}")
print(f"Provider: {response.provider}")
print(f"Finish reason: {response.finish_reason}")
print(f"Duration: {response.duration}s")

# Raw provider response (for detailed analysis)
raw = response.raw_response

# Convert to dict (for JSON serialization)
response_dict = response.to_dict()
```

## Configuration

### Provider-Specific Settings

```python
from ai_client import create_ai_client

# OpenAI
client = create_ai_client(
    'openai',
    api_key='sk-...',
    temperature=0.7,
    max_tokens=500,
    frequency_penalty=0.5
)

# Claude
client = create_ai_client(
    'anthropic',
    api_key='sk-ant-...',
    temperature=1.0,
    max_tokens=4096,
    top_k=40
)

# Settings can also be passed per-request
response, _ = client.prompt(
    'gpt-4',
    'Hello',
    temperature=0.9,
    max_tokens=100
)
```

### Custom System Prompts

```python
from ai_client import create_ai_client

client = create_ai_client(
    'openai',
    api_key='sk-...',
    system_prompt="You are a helpful coding assistant specialized in Python."
)

# Override for specific request
response, _ = client.prompt(
    'gpt-4',
    'Write a haiku',
    system_prompt="You are a poetic assistant."
)
```

## Use Case: Benchmarking

Evaluate multiple models within a research workflow:

```python
from ai_client import create_ai_client
import asyncio

async def benchmark_models():
    providers = [
        ('openai', 'gpt-4o-mini'),
        ('anthropic', 'claude-haiku-4-5'),
        ('genai', 'gemini-2.5-flash'),
    ]

    prompt = 'Explain quantum entanglement'

    for provider_id, model in providers:
        client = create_ai_client(provider_id, api_key=f'{provider_id}_key')

        response = await client.prompt_async(model, prompt)

        print(f"\n=== {provider_id}/{model} ===")
        print(f"Duration: {response.duration:.2f}s")
        print(f"Tokens: {response.usage.total_tokens}")
        print(f"Response: {response.text[:200]}...")

asyncio.run(benchmark_models())
```

## Error Handling

The package includes built-in retry logic with exponential backoff:

```python
from ai_client import create_ai_client, RateLimitError, APIError

client = create_ai_client('openai', api_key='sk-...')

try:
    response, duration = client.prompt('gpt-4', 'Hello')
    # Automatically retries up to 3 times on rate limit errors
except RateLimitError as e:
    print(f"Rate limited after retries: {e}")
except APIError as e:
    print(f"API error: {e}")
except Exception as e:
    print(f"Unknown error: {e}")
```

## Advanced Features

### Get Available Models

```python
from ai_client import create_ai_client

client = create_ai_client('openai', api_key='sk-...')
models = client.get_model_list()

for model_id, created_date in models:
    print(f"{model_id} (created: {created_date})")
```

### Check Multimodal Support

```python
client = create_ai_client('openai', api_key='sk-...')

if client.has_multimodal_support():
    print("This provider supports images!")
```

## Package Structure

```
ai_client/
  __init__.py           # Package exports
  base_client.py        # BaseAIClient + factory
  response.py           # LLMResponse, Usage dataclasses
  utils.py              # Retry logic, exceptions, utilities
  openai_client.py      # OpenAI implementation
  claude_client.py      # Anthropic Claude
  gemini_client.py      # Google Gemini
  mistral_client.py     # Mistral AI
  deepseek_client.py    # DeepSeek
  qwen_client.py        # Qwen
```

## Requirements

- Python >=3.9
- anthropic ~=0.71.0
- openai ~=2.6.1
- mistralai ~=1.9.11
- google-genai ~=1.46.0
- requests ~=2.32.5

## Development

```bash
# Clone the repository
git clone https://github.com/RISE-UNIBAS/generic-llm-api-client.git
cd generic-llm-api-client

# Install in development mode
pip install -e ".[dev]"

# Run tests
pytest

# Run integration tests (requires API keys)
pytest -m integration

# Format code
black ai_client tests

# Type checking
mypy ai_client/
```

## Documentation

- **[EXAMPLES.md](EXAMPLES.md)** - Comprehensive usage examples
- **[PUBLISHING.md](PUBLISHING.md)** - Guide for maintainers on publishing releases

## Contributing

Contributions are welcome. Submit proposed changes through a pull request.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Citation

If you use this package in your research, please cite:

```bibtex
@software{generic_llm_api_client,
  author = {Sorin Marti},
  title = {Generic LLM API Client: A Unified Interface for Multiple LLM Providers},
  year = {2025},
  url = {https://github.com/RISE-UNIBAS/generic-llm-api-client}
}
```

## Support

- GitHub Issues: [Report bugs or request features](https://github.com/RISE-UNIBAS/generic-llm-api-client/issues)
- Documentation: [Full documentation](https://github.com/RISE-UNIBAS/generic-llm-api-client#readme)

## Roadmap

- [ ] Tool use / function calling support
- [ ] Streaming support
- [ ] Conversation history management
- [ ] More providers (Cohere, AI21, etc.)
- [ ] Cost estimation utilities
- [ ] Prompt caching support
