Metadata-Version: 2.4
Name: pyinferencemanager
Version: 0.4.1
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Classifier: Typing :: Typed
Requires-Dist: anthropic>=0.100.0
Requires-Dist: pytest>=7.0 ; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21 ; extra == 'dev'
Requires-Dist: maturin>=1.0 ; extra == 'dev'
Requires-Dist: ruff>=0.5.0 ; extra == 'dev'
Provides-Extra: dev
License-File: LICENSE
Summary: Intelligent AI workload orchestrator. Multi-provider routing, cost optimization, semantic caching, dynamic routing with budget enforcement. Production observability included.
Keywords: ai,orchestrator,llm,ollama,anthropic,openai,local-first,cost-optimization,caching,multi-provider,routing,inference,workload-orchestration,hardware-aware,semantic-caching,distributed-tracing,observability,budget-enforcement
Author-email: Georgi Mammen Mullassery <mullassery@gmail.com>
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://github.com/Mullassery/pyinferencemanager/releases
Project-URL: Documentation, https://github.com/Mullassery/pyinferencemanager/blob/main/README.md
Project-URL: Homepage, https://github.com/Mullassery/pyinferencemanager
Project-URL: Issues, https://github.com/Mullassery/pyinferencemanager/issues
Project-URL: Repository, https://github.com/Mullassery/pyinferencemanager.git

# PyInferenceManager

> **Intelligent AI workload orchestrator.** Multi-provider routing, cost optimization, semantic caching, dynamic routing with budget enforcement.

![Status](https://img.shields.io/badge/Status-Production--Ready-brightgreen.svg)
![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)
![Tests](https://img.shields.io/badge/Tests-261%20Passing-brightgreen.svg)
![Distribution](https://img.shields.io/badge/Distribution-Wheels--Only-blue.svg)
![License](https://img.shields.io/badge/License-Proprietary-red.svg)

---

## Product Overview

**PyInferenceManager** is a proprietary, production-grade AI workload orchestrator. Route inference requests across multiple providers (Anthropic, OpenAI, etc.) with cost optimization and intelligent fallback.

### Why Teams Choose This

**The Problem**:
- Single-provider lock-in creates vendor risk
- No visibility into inference costs
- Failures cascade without intelligent fallback
- Manual routing decisions waste engineering time

**The Solution**:
- Route to any provider with single API change
- Cost-aware routing with budget enforcement
- Semantic caching to reduce redundant calls
- Automatic fallback on provider failures
- Comprehensive cost tracking and attribution

**Result**: Reduce inference costs 30-40%, eliminate vendor lock-in, improve reliability.

---

## Installation

```bash
pip install pyinferencemanager
# or with uv
uv pip install pyinferencemanager
```

### Requirements
- Python 3.10+
- Precompiled wheels for macOS, Linux, Windows

### Distribution Model

**Proprietary-first distribution**:
- ✅ Wheels-only via PyPI (no source code)
- ✅ Production-optimized multi-provider routing
- ✅ 261 comprehensive tests
- ✅ Used in production ML systems

---

## Quick Start

```python
from pyinferencemanager import OrchestrationEngine, ProviderConfig

# Configure multiple providers
config = OrchestrationEngine.Config(
    providers=[
        ProviderConfig(name='anthropic', api_key='...'),
        ProviderConfig(name='openai', api_key='...'),
    ],
    budget=1000.00,  # Monthly budget
    cost_optimization=True,
)

engine = OrchestrationEngine(config)

# Single API for all providers
response = engine.infer(
    prompt="Analyze this dataset...",
    model='claude-3-sonnet',
    fallback_to=['gpt-4', 'claude-opus'],
    budget_tier='standard',  # or 'economy', 'premium'
)

print(f"Response: {response.text}")
print(f"Cost: ${response.cost:.4f}")
print(f"Provider used: {response.provider}")
```

---

## Features

- **Multi-Provider Routing**: Anthropic, OpenAI, and custom providers
- **Cost Optimization**: Automatic provider selection based on cost
- **Semantic Caching**: Reduce redundant API calls
- **Budget Enforcement**: Hard stop when monthly budget exceeded
- **Intelligent Fallback**: Automatic failover to alternate providers
- **Cost Attribution**: Track spending by model, user, team
- **Health Monitoring**: Provider availability and latency tracking
- **Observability**: OpenTelemetry instrumentation

---

## Quality & Testing

- **261 tests** passing
- **Production-grade** — used in real ML systems
- **Observability** — cost tracking, performance metrics
- **Reliability** — intelligent fallback and retry logic

---

## Support

For production deployments: **mullassery@gmail.com**

---

**Version**: 0.3.1  
**License**: Proprietary  
**Distribution**: Wheels-only via PyPI  
**Python**: 3.10+  

Built for production AI systems.

