Metadata-Version: 2.4
Name: simple-infer
Version: 0.1.0
Summary: Minimal, hackable batch inference library for LLMs
Author: sumukshashidhar
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.12
Requires-Dist: httpx>=0.28.1
Requires-Dist: loguru>=0.7.3
Requires-Dist: openai>=1.97.0
Requires-Dist: tenacity>=9.1.2
Requires-Dist: tqdm>=4.67.1
Provides-Extra: docs
Requires-Dist: sphinx-autodoc-typehints>=3.2.0; extra == 'docs'
Requires-Dist: sphinx-rtd-theme>=3.0.2; extra == 'docs'
Requires-Dist: sphinx>=8.2.3; extra == 'docs'
Provides-Extra: test
Requires-Dist: pytest-asyncio>=1.1.0; extra == 'test'
Requires-Dist: pytest>=8.4.1; extra == 'test'
Description-Content-Type: text/markdown

# simple-infer

Minimal, hackable batch inference library for LLMs. No batch endpoints needed.

## Installation

```bash
pip install simple-infer
```

## Usage

```python
from simple_infer import infer

conversations = [
    [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"},
    ],
    [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is 2+2?"},
    ]
]

results = infer(conversations, model="gpt-4.1-nano", max_concurrent=32)
```

## Features

- **Simple**: Two main functions - `infer()` and `call_llm()`
- **Fast**: Async batch processing with configurable concurrency
- **Reliable**: Built-in retries with exponential backoff
- **Hackable**: Clean, readable code you can modify

## Development

### Setup
```bash
# Clone and install dependencies
uv sync --extra test --extra docs

# Copy environment template and add your OpenAI API key
cp .env.example .env
# Edit .env and add: OPENAI_API_KEY=your-key-here
```

### Testing
```bash
# Run tests (requires OPENAI_API_KEY in environment)
export OPENAI_API_KEY=your-key-here
uv run pytest tests/ -v

# Or use .env file
uv run pytest tests/ -v
```
