Metadata-Version: 2.4
Name: ehs-llm-client
Version: 0.1.14
Summary: Unified LLM client. Currently supports Openai, Azure Openai and Google Gemini
Author-email: Andersen Huang <andersen.huang@ehsanalytics.com>
License-Expression: MIT
Description-Content-Type: text/markdown
Requires-Dist: openai>=1.97.0
Requires-Dist: google-genai
Requires-Dist: python-dotenv
Requires-Dist: python-dateutil

# llm-client

[![PyPI version](https://img.shields.io/pypi/v/ehs-llm-client)](https://pypi.org/project/ehs-llm-client/)
[![Python Version](https://img.shields.io/pypi/pyversions/ehs-llm-client)](https://www.python.org/downloads/)

`ehs-llm-client` is a unified **async Python client** for interacting with multiple LLM providers, including **OpenAI**, **Azure OpenAI**, and **Google Gemini**.  
It supports **single calls**, **structured JSON outputs**, and **batch processing**, designed for **production-ready, reusable code** in applications, scripts, or pipelines.

---

## Features

- Async support for multiple LLM providers  
- Unified interface across OpenAI, Azure, and Google Gemini  
- Structured JSON responses via schemas  
- Retry and timeout handling  
- Batch processing support  
- Environment variable configuration for API keys  
- Easy integration into existing projects or monorepos  

---

## Installation

```bash
# Install from PyPI
pip install ehs-llm-client
```

Or install editable version during development:

```bash
# From project root
pip install -e .
```

---

## Configuration

`ehs-llm-client` supports **three configuration modes**:

### 1️⃣ Config file (`.cfg`)

Example: `llmconfig.cfg`

```ini
[default_settings]
provider = openai
model = gpt-4.1-mini

[prod]
provider = openai
model = gpt-4.1
model_batch = gpt-4.1-mini
```

Usage:

```python
from ehs_llm_client import LLM

llm = LLM("prod", config_file_path="llmconfig.cfg")
```

---

### 2️⃣ Config dictionary (for tests / CI)

```python
from ehs_llm_client import LLM

llm = LLM(
    "default",
    config={
        "default": {
            "provider": "openai",
            "model": "gpt-4.1-mini"
        }
    }
)
```

---

### 3️⃣ Environment variable fallback

```bash
export LLM_PROVIDER=openai
export LLM_MODEL=gpt-4.1-mini
export OPENAI_API_KEY=sk-...
```

```python
llm = LLM("default")
```

### Azure OpenAI with GPT-6

Deploy your chosen GPT-6 model in Azure, then use its **deployment name** as `model`:

```ini
[azure]
provider = azure_openai
model = my-gpt6-deployment
base_url = https://YOUR-RESOURCE.openai.azure.com/
```

Set `AZURE_OPENAI_API_KEY` and initialize the client:

```python
llm = LLM("azure", config_file_path="llmconfig.cfg")
```

You can also configure everything with environment variables:

```powershell
$env:LLM_PROVIDER = "azure_openai"
$env:LLM_MODEL = "my-gpt6-deployment"
$env:AZURE_OPENAI_ENDPOINT = "https://YOUR-RESOURCE.openai.azure.com/"
$env:AZURE_OPENAI_API_KEY = "YOUR-AZURE-API-KEY"
```

Then use `LLM("default")`. `LLM_BASE_URL` or a configured `base_url` takes precedence
over `AZURE_OPENAI_ENDPOINT`. Both a resource URL and a full `/openai/v1/` URL are
accepted, including endpoints on `services.ai.azure.com`. The client uses Azure's
v1 API; existing `api_version` settings are retained for compatibility and ignored.
See [Azure Responses API documentation](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/responses)
for supported models and regions.

---

## Usage

### Single async call

```python
import asyncio
from ehs_llm_client import LLM

async def main():
    llm = LLM("prod", config_file_path="llmconfig.cfg")
    messages = [{"role": "user", "content": "Say hello in JSON"}]

    response, in_tokens, out_tokens = await llm.get_response_async(
        messages, schema="return json"
    )

    print(response)

asyncio.run(main())
```

One `LLM` can be used from several event loops in turn, for example a Celery task
that calls `asyncio.run()` for each group of work. Since 0.1.13, `llm.client` is
replaced with a new provider client the first time it is used from a new event
loop. Before that, calls in a later loop reused connections opened by the closed
one; they failed, and the OpenAI SDK retried them ("Retrying request to
/responses"). Assigning `llm.client` yourself still works; the next event loop
to use it adopts it.

Since 0.1.13, OpenAI and Azure OpenAI clients keep an idle connection for 60
seconds before closing it, instead of the SDK's 5. Calls that come in groups
reuse connections rather than opening new ones, which saves a TCP/TLS handshake
and an outbound (SNAT) port each time. Set `LLM_KEEPALIVE_EXPIRY` (seconds,
more than 0 and less than 240) to change it. Stay below 240: Azure drops
outbound connections idle for longer without telling either end. Other SDK
defaults are unchanged: connect timeout 5 s, read timeout 600 s, at most 1000
connections per client.

Blocked, failed or incomplete OpenAI/Azure responses raise
`ehs_llm_client.exceptions.LLMResponseError` (a `RuntimeError`) instead of returning
partial or empty text as success. Content-filter errors are not retried with the same prompt.

### Reasoning effort

The reasoning controls from version 0.1.10 are supported:

```ini
[reasoning]
provider = openai
model = gpt-6-astra
default_reasoning_effort = medium
```

Set `default_reasoning_effort` in a config section/dictionary, or set
`LLM_DEFAULT_REASONING_EFFORT` when using environment configuration. Override it
for a single call with `reasoning_effort`:

```python
response, in_tokens, out_tokens = await llm.get_response_async(
    messages, schema="return json", reasoning_effort="high"
)
```

`call_async` also accepts `reasoning_effort`. OpenAI uses the model support check
from 0.1.10. Azure forwards an explicitly configured effort because deployment
names can be arbitrary; only set it for a reasoning-capable deployment. Supported
effort values depend on the model. If you omit the setting, the API's default applies.
See [OpenAI reasoning guidance](https://developers.openai.com/api/docs/guides/reasoning).
The setting applies to single calls; batch behavior remains as in 0.1.10.

The 0.1.10 method signatures are preserved, including positional reasoning,
retry and timeout arguments. Prefer keyword arguments for clarity.

OpenAI/Azure JSON mode (`schema="return json"`) requires string message contents,
as in 0.1.10. Regular calls and schema-based output retain support for content arrays.

---

### Batch request

```python
batch_line = llm.create_batch_request(
    custom_id="test1",
    messages=[{"role": "user", "content": "Give me a JSON object"}],
    schema="return json"
)

batch_id = await llm.run_batch_process([batch_line], submission_id="batch123")
status = await llm.get_batch_status(batch_id)
print(status)
```

`create_batch_request` returns a dictionary. `run_batch_process` accepts these
dictionaries or JSONL strings that you already serialized. When `model_batch` is
omitted, OpenAI/Azure batches use `model`.

For Azure, set `model_batch` to a deployment that supports batch processing
(`GlobalBatch` or `DataZoneBatch`). Model and region availability can differ from
single-call deployments; see [Azure batch documentation](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/batch).
Azure supports submission, status checking, cancellation and result downloads
through the same client methods.

---

## API Key Management

Keep API keys **out of code**. Recommended environment variables:

| Provider | Standard env var | Existing env vars (still supported) |
|----------|------------------|-----------------------------------|
| OpenAI | `OPENAI_API_KEY` | `openaiAPIkey`, `openaiAPIKey` |
| Azure OpenAI | `AZURE_OPENAI_API_KEY` | `azure_openaiAPIkey`, `azure_openaiAPIKey` |
| Google Gemini | `GOOGLE_API_KEY` | `googleAPIkey`, `googleAPIKey` |

The 0.1.10 `APIKey` spelling takes precedence, followed by the older `APIkey`
spelling and then the standard variable. Azure credentials must come from your Azure resource.

---

## Development

```bash
# Create a virtual environment
python -m venv .venv
source .venv/bin/activate  # macOS/Linux
.venv\Scripts\activate     # Windows

# Install dependencies
pip install -e .
```

Run the regression checks (simulated HTTP responses; no API credentials needed):

```bash
python -m unittest discover -s tests -v
```

---

## License

MIT License © 2026 Your Name  
