Metadata-Version: 2.4
Name: uc-llm-provider
Version: 0.5.1
Summary: Generic LLM provider abstraction — Anthropic, OpenAI, Google, Ollama and any OpenAI-compatible local model
Author: cuber IT service
License: MIT
Keywords: abstraction,ai,anthropic,llm,local-llm,mcp,ollama,openai,provider
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.11
Requires-Dist: httpx>=0.26.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pyyaml>=6.0
Provides-Extra: all
Requires-Dist: anthropic>=0.20.0; extra == 'all'
Requires-Dist: click>=8.0.0; extra == 'all'
Requires-Dist: fastapi>=0.100.0; extra == 'all'
Requires-Dist: google-generativeai>=0.5.0; extra == 'all'
Requires-Dist: openai>=1.0.0; extra == 'all'
Requires-Dist: rich>=13.0.0; extra == 'all'
Requires-Dist: uvicorn[standard]>=0.20.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.20.0; extra == 'anthropic'
Provides-Extra: cli
Requires-Dist: click>=8.0.0; extra == 'cli'
Requires-Dist: rich>=13.0.0; extra == 'cli'
Provides-Extra: dev
Requires-Dist: click>=8.0.0; extra == 'dev'
Requires-Dist: fastapi>=0.100.0; extra == 'dev'
Requires-Dist: httpx>=0.26.0; extra == 'dev'
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: rich>=13.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Provides-Extra: google
Requires-Dist: google-generativeai>=0.5.0; extra == 'google'
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == 'openai'
Provides-Extra: server
Requires-Dist: fastapi>=0.100.0; extra == 'server'
Requires-Dist: uvicorn[standard]>=0.20.0; extra == 'server'
Description-Content-Type: text/markdown

# uc-llm-provider

Generic LLM provider abstraction for Python.  
Anthropic, OpenAI, Google — and any OpenAI-compatible local model (Ollama, LM Studio, vLLM, llama.cpp).

**Three usage modes:** Library · Server · CLI

---

## Install

```bash
pip install uc-llm-provider                  # core only
pip install "uc-llm-provider[anthropic]"     # + Anthropic SDK
pip install "uc-llm-provider[openai]"        # + OpenAI SDK
pip install "uc-llm-provider[server,cli]"    # + Server + CLI
pip install "uc-llm-provider[all]"           # everything
```

---

## Library

```python
from uc_llm_provider import get_provider, ChatRequest, ChatMessage

# Cloud
provider = get_provider({
    "name":          "anthropic",
    "provider_type": "anthropic",
    "api_key":       "sk-ant-...",
    "default_model": "claude-sonnet-4-20250514",
})

# Local (Ollama)
provider = get_provider({
    "name":          "ollama",
    "provider_type": "ollama",
    "api_base":      "http://localhost:11434",
    "default_model": "llama3.2",
})

# Any OpenAI-compatible endpoint
provider = get_provider({
    "name":          "lm-studio",
    "provider_type": "openai_compatible",
    "api_base":      "http://localhost:1234/v1",
    "default_model": "local-model",
})

# Chat
response = await provider.chat(ChatRequest(
    messages=[ChatMessage(role="user", content="Hello")],
    max_tokens=256,
))
print(response.content)
```

### Streaming

```python
async for chunk in provider.chat_stream(request):
    if chunk.type == "content_delta":
        print(chunk.content, end="", flush=True)
```

### Custom provider

```python
from uc_llm_provider import register_provider
from uc_llm_provider.providers.template import TemplateProvider

class MyProvider(TemplateProvider):
    def _get_endpoint(self, path=""):
        return "https://api.myprovider.com/v1/chat"

register_provider("myprovider", MyProvider)
```

---

## Server

```bash
uc-llm-server --port 12290 --provider anthropic --model claude-sonnet-4-20250514
uc-llm-server --port 12290 --provider ollama --api-base http://localhost:11434 --model llama3.2
uc-llm-server --port 12290 --config server.yaml
```

**Endpoints:**

| Method | Path | Description |
|---|---|---|
| GET | `/v1/health` | Provider status |
| GET | `/v1/capabilities` | Supported tiers and features |
| GET | `/v1/models` | Model list |
| GET | `/v1/models/{id}` | Model detail |
| POST | `/v1/chat` | Chat completion |
| POST | `/v1/chat/stream` | Streaming chat (SSE) |
| POST | `/v1/tokens/count` | Token count |

**Config YAML:**

```yaml
provider: anthropic
model: claude-sonnet-4-20250514
api_key: ""          # or set UC_LLM_API_KEY
log_mode: jsonl      # jsonl | none
log_dir: ./logs
host: 0.0.0.0
cors_origins: ["*"]
```

---

## CLI

```bash
uc-llm chat "Explain Playwright in one sentence"
uc-llm chat --interactive
uc-llm stream "Explain Playwright"
uc-llm models --provider ollama
uc-llm tokens "How many tokens is this?"
uc-llm health --provider anthropic
uc-llm config
```

**Global options:**

| Option | Env | Description |
|---|---|---|
| `--provider` | `UC_LLM_PROVIDER` | Provider type |
| `--model` | `UC_LLM_MODEL` | Model |
| `--api-key` | `UC_LLM_API_KEY` | API key |
| `--api-base` | `UC_LLM_API_BASE` | API base URL |
| `--log-mode` | `UC_LLM_LOG_MODE` | `jsonl` or `none` |

---

## Provider types

| `provider_type` | Provider |
|---|---|
| `anthropic` | Anthropic Claude |
| `openai` | OpenAI |
| `google` | Google Gemini |
| `ollama` | Ollama (local) |
| `openai_compatible` | Any OpenAI-compatible API |
| `lm_studio` / `lmstudio` | LM Studio |
| `vllm` | vLLM |
| `llamacpp` | llama.cpp server |
| `localai` | LocalAI |

---

## Ollama extras

```python
from uc_llm_provider import get_provider

ollama = get_provider({"provider_type": "ollama", "api_base": "http://localhost:11434"})
models = await ollama.list_local_models()
async for status in ollama.pull_model("mistral"):
    print(status)
running = await ollama.is_running()
```

---

## License

MIT · [cuber IT service](https://uc-it.de)
