Metadata-Version: 2.4
Name: ai-dynamo-llm
Version: 0.1.0
Summary: AI Dynamo LLM serving client — OpenAI-compatible inference client with multimodal support
Project-URL: Homepage, https://github.com/ai-dynamo/ai-dynamo-llm
Project-URL: Repository, https://github.com/ai-dynamo/ai-dynamo-llm
Author: Dynamo AI Team
License: Apache-2.0
Keywords: inference,llm,multimodal,openai,serving
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Requires-Dist: numpy>=1.24
Requires-Dist: requests>=2.28
Description-Content-Type: text/markdown

# ai-dynamo-llm

OpenAI-compatible inference client with multimodal support, designed for
NVIDIA Dynamo-style serving stacks.

## Features

- Chat completions (streaming and non-streaming) against any OpenAI-compatible endpoint
- Multimodal messages: attach images by file path or URL (base64 data-URI encoding)
- Text embeddings
- Automatic retry with backoff on 429 / 5xx / connection errors
- Token estimation with CJK-aware budgeting

## Install

```bash
pip install ai-dynamo-llm
```

## Quick start

```python
from ai_dynamo_llm import AIDynamoClient, ChatMessage

client = AIDynamoClient(api_key="sk-...", base_url="http://localhost:8000/v1")

resp = client.chat(
    model="dynamo-llm",
    messages=[
        ChatMessage.system("You are a helpful assistant."),
        ChatMessage.user("What is Dynamo?"),
    ],
)
print(resp["choices"][0]["message"]["content"])
```

## Multimodal

```python
msg = ChatMessage.multimodal(
    "Describe this image.",
    ["/tmp/screenshot.png", "https://example.com/diagram.png"],
)
resp = client.chat(model="dynamo-vlm", messages=[msg])
```

## Token budgeting

```python
from ai_dynamo_llm import estimate_tokens, truncate_to_tokens

assert estimate_tokens("hello world") <= estimate_tokens("你好世界" * 10)
prompt = truncate_to_tokens(long_doc, max_tokens=2048)
```

## License

Apache-2.0
