Metadata-Version: 2.4
Name: litellm-td-llm-provider
Version: 0.1.1
Summary: LiteLLM provider for Treasure Data LLM Proxy
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Requires-Dist: httpx>=0.24.0
Requires-Dist: litellm>=1.0.0
Requires-Dist: python-dotenv>=1.0.1
Description-Content-Type: text/markdown

# LiteLLM TD LLM Proxy Provider

[![Python](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![PyPI](https://img.shields.io/pypi/v/litellm-td-llm-provider.svg)](https://pypi.org/project/litellm-td-llm-provider/)
[![CI status](https://github.com/treasure-data-ai-labs/litellm-td-llm-provider/actions/workflows/ci.yml/badge.svg)](https://github.com/treasure-data-ai-labs/litellm-td-llm-provider/actions)

Custom LiteLLM provider for Treasure Data's LLM Proxy API.

## Installation

### Using uv (recommended)

```bash
cd litellm-td-llm-provider
uv sync
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
```

### Using pip

```bash
cd litellm-td-llm-provider
pip install -e .
```

## Usage

### Basic Completion

```python
import litellm
from litellm_td_llm_provider import register_td_provider

# Register the TD provider
register_td_provider()

# Use with LiteLLM
response = litellm.completion(
    model="td/claude-sonnet-4-5",
    messages=[{"role": "user", "content": "Hello!"}],
    api_key="your-td-api-key",
    api_base="https://llm-proxy.us01.treasuredata.com"
)
```

### Async Completion

```python
import asyncio
import litellm
from litellm_td_llm_provider import register_td_provider

register_td_provider()

async def main():
    response = await litellm.acompletion(
        model="td/claude-sonnet-4-5",
        messages=[{"role": "user", "content": "Hello!"}],
    )
    print(response.choices[0].message.content)

asyncio.run(main())
```

### Tool Calling

```python
import litellm
from litellm_td_llm_provider import register_td_provider

register_td_provider()

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"},
                },
                "required": ["location"],
            },
        },
    }
]

response = litellm.completion(
    model="td/claude-sonnet-4-5",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
)

if response.choices[0].message.tool_calls:
    for tool_call in response.choices[0].message.tool_calls:
        print(f"Tool: {tool_call.function.name}")
        print(f"Arguments: {tool_call.function.arguments}")
```

## Configuration

### Available Models

The following Claude 4.5 models are available through TD LLM Proxy:

| Model | Model ID | Alias |
|-------|----------|-------|
| Claude Sonnet 4.5 | `claude-sonnet-4-5-20250929` | `claude-sonnet-4-5` |
| Claude Haiku 4.5 | `claude-haiku-4-5-20251001` | `claude-haiku-4-5` |
| Claude Opus 4.5 | `claude-opus-4-5-20251101` | `claude-opus-4-5` |

**Note**:
- Aliases (e.g., `claude-sonnet-4-5`) automatically point to the latest version
- Use specific versions (e.g., `claude-sonnet-4-5-20250929`) if you need to pin to a particular model snapshot

This provider uses `claude-sonnet-4-5` as the default model.

### Site Endpoints

- `us01`: https://llm-proxy.us01.treasuredata.com
- `jp01`: https://llm-proxy.treasuredata.co.jp
- `eu01`: https://llm-proxy.eu01.treasuredata.com
- `ap02`: https://llm-proxy.ap02.treasuredata.com
- `ap03`: https://llm-proxy.ap03.treasuredata.com
- `dev-us01`: https://llm-proxy-development.us01.treasuredata.com
- `dev-eu01`: https://llm-proxy-development.eu01.treasuredata.com
- `stg-us01`: https://llm-proxy-staging.us01.treasuredata.com
- `stg-jp01`: https://llm-proxy-staging.treasuredata.co.jp
- `stg-ap03`: https://llm-proxy-staging.ap03.treasuredata.com

## Environment Variables

```bash
export TD_LLM_API_KEY="your-api-key"
export TD_LLM_SITE="us01"  # Optional, defaults to us01
```

## Features

- ✅ Streaming support
- ✅ Non-streaming completion
- ✅ Async completion (`acompletion`)
- ✅ Tool calling support (OpenAI → Anthropic format conversion)
- ✅ Multi-site support
- ✅ Anthropic Messages API compatible
- ✅ OpenAI-compatible interface

## Development

### Setup

```bash
# Install dependencies (creates venv automatically)
uv sync

# Activate virtual environment
source .venv/bin/activate

# Run tests
uv run pytest
```

### Using Makefile

```bash
make install-dev  # Install dev dependencies
make format       # Format code with ruff
make lint         # Run linting
make check        # Run type checking with pyright
make test         # Run tests
make test-cov     # Run tests with coverage
make check-all    # Run all checks (format, lint, check, test)
make clean        # Clean cache and build artifacts
```

### Release Process

1. Update version in `pyproject.toml`
2. Create and push a git tag:
   ```bash
   git tag v0.1.0
   git push origin v0.1.0
   ```
3. GitHub Actions will automatically build and publish to PyPI

**Required GitHub Secret:**
- `PYPI_TOKEN`: PyPI API token for publishing

### Running Examples

```bash
# Set credentials
export TD_LLM_API_KEY="your-api-key"
export TD_LLM_SITE="us01"

# Run basic example
python examples/basic_usage.py

# Run site selection example
python examples/site_selection.py
```

## Implementation Details

This provider is based on the TypeScript implementation in `tdx-studio/electron/services/llm-proxy-client.ts` and provides:

1. **Endpoint Management**: Multi-region support with automatic site resolution
2. **Streaming**: Server-Sent Events (SSE) parsing for real-time responses
3. **Error Handling**: Proper error propagation and API error messages
4. **LiteLLM Integration**: Custom provider registration with standard LiteLLM interface

## API Compatibility

The TD LLM Proxy API is Anthropic-compatible and supports:

- `/v1/messages` endpoint
- `x-api-key` authentication
- `anthropic-version: 2023-06-01` header
- Streaming via SSE with `stream: true`
