Metadata-Version: 2.4
Name: inferiallm
Version: 1.3.1
Summary: InferiaLLM: distributed inference, filtration, and compute orchestration platform
Author: Inferia AI
License: Apache 2.0
Project-URL: Repository, https://github.com/InferiaAI/InferiaLLM
Project-URL: Documentation, https://docs.inferia.ai/docs
Requires-Python: <3.13,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi==0.109.0
Requires-Dist: uvicorn[standard]<0.30,>=0.27
Requires-Dist: httpx>=0.27.0
Requires-Dist: aiohttp>=3.8.5
Requires-Dist: grpcio==1.76.0
Requires-Dist: grpcio-tools==1.76.0
Requires-Dist: protobuf<7.0.0,>=6.31.1
Requires-Dist: anyio>=4.0
Requires-Dist: pydantic>=2.0
Requires-Dist: pydantic-settings>=2.0
Requires-Dist: email-validator>=2.1.0
Requires-Dist: python-jose[cryptography]
Requires-Dist: passlib[bcrypt]
Requires-Dist: bcrypt
Requires-Dist: pyotp
Requires-Dist: qrcode
Requires-Dist: pillow
Requires-Dist: asyncpg==0.31.0
Requires-Dist: psycopg2-binary==2.9.11
Requires-Dist: redis>=5.0
Requires-Dist: sqlalchemy>=2.0
Requires-Dist: alembic>=1.13
Requires-Dist: greenlet
Requires-Dist: jinja2
Requires-Dist: numpy
Requires-Dist: transformers>=4.40
Requires-Dist: tokenizers>=0.21
Requires-Dist: chromadb
Requires-Dist: llm-guard
Requires-Dist: presidio-analyzer
Requires-Dist: spacy
Requires-Dist: pypdf
Requires-Dist: python-docx
Requires-Dist: requests
Requires-Dist: groq
Requires-Dist: prometheus-client
Requires-Dist: python-dotenv
Requires-Dist: base58==2.1.1
Requires-Dist: pynacl
Requires-Dist: cachetools
Requires-Dist: python-multipart
Requires-Dist: skypilot
Requires-Dist: kubernetes==33.1.0
Provides-Extra: aws
Requires-Dist: boto3==1.42.10; extra == "aws"
Requires-Dist: skypilot[aws]; extra == "aws"
Provides-Extra: gcp
Requires-Dist: skypilot[gcp]; extra == "gcp"
Provides-Extra: azure
Requires-Dist: skypilot[azure]; extra == "azure"
Provides-Extra: lambda
Requires-Dist: skypilot[lambda]; extra == "lambda"
Provides-Extra: runpod
Requires-Dist: skypilot[runpod]; extra == "runpod"
Provides-Extra: all-providers
Requires-Dist: boto3==1.42.10; extra == "all-providers"
Requires-Dist: skypilot[aws,azure,gcp,kubernetes,lambda,runpod]; extra == "all-providers"
Provides-Extra: logstash
Requires-Dist: python-logstash>=0.4.6; extra == "logstash"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: pytest-mock; extra == "dev"
Requires-Dist: pytest-trio; extra == "dev"
Requires-Dist: pytest-twisted; extra == "dev"
Requires-Dist: pytest-tornasync; extra == "dev"
Dynamic: license-file


<div align="center">

# InferiaLLM

### The Operating System for LLMs in Production

  [![License](https://img.shields.io/badge/license-Apache--2.0-green?style=flat-square)](https://github.com/InferiaAI/InferiaLLM/blob/main/LICENSE)[![Python](https://img.shields.io/badge/python-3.10+-blue?style=flat-square)](https://www.python.org/)[![Status](https://img.shields.io/badge/status-beta-orange?style=flat-square)]()[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)

</div>

InferiaLLM acts as the authoritative execution layer between your applications and your AI infrastructure. It governs how LLMs are accessed, secured, routed, and run on compute.

---

## Installation

```bash
pip install inferiallm
```

---

## Quick Start

InferiaLLM requires a `.env` file in your current working directory to configure connections to PostgreSQL and Redis.

```bash
# 1. Initialize your environment
# Create a .env file with your DATABASE_URL and Redis settings
cp .env.sample .env

# 2. Bootstrap the platform
# This creates the database, roles, and applying schemas
inferiallm init

# 3. Launch all services
# Starts API, Orchestration, Inference, and the Dashboard in one go
inferiallm start all
```

---

## Configuration

The CLI manages configuration through environment variables. The most critical settings are:

### 1. Database & Security

| Variable | Description | Default |
| --- | --- | --- |
| `DATABASE_URL` | Primary database connection string | `postgresql://inferia:inferia@localhost:5432/inferia` |
| `PG_ADMIN_USER` | Postgres admin user (required for `init`) | `postgres` |
| `PG_ADMIN_PASSWORD` | Postgres admin password (required for `init`) | - |
| `JWT_SECRET_KEY` | Secret for signing access tokens | - |
| `INTERNAL_API_KEY` | Secret for service-to-service auth | - |
| `SECRET_ENCRYPTION_KEY` | 32-byte base64 key for encrypting credentials | - |

---

## CLI Reference

### `inferiallm init`

Bootstraps the unified database environment.
**Output:**

```text
[inferia:init] Connecting as admin to bootstrap inferia
[inferia:init] Creating role: inferia
[inferia:init] Creating database: inferia
[inferia:init] Applying schema: global_schema
[inferia:init] Bootstrap complete
```

For existing databases, apply incremental schema updates manually:

```bash
psql "$DATABASE_URL" -f db/migrations/20260212_add_inference_logs_ip.sql
```

### `inferiallm start`

Starts all InferiaLLM gateways (API, Orchestration, Inference) and the Dashboard.

You can also start specific services:

* `inferiallm start all`: Starts all core services (default).
* `inferiallm start api-gateway`: Starts only the API Gateway.
* `inferiallm start orchestration`: Starts the Orchestration stack (API, Sidecar, Worker).
* `inferiallm start inference`: Starts the Inference engine.

---

## Package Structure

The `inferia` package is a monorepo-style library that contains all backend services:

```text
package/src/inferia/
├── cli.py                  # Entry point for the CLI
├── services/
│   ├── api_gateway/        # Security & Policy Service (Port 8000)
│   ├── inference/          # Inference Proxy Service (Port 8001)
│   ├── orchestration/      # Compute Control Plane (Port 8080)
│   ├── data/               # Data Engine (Port 8003)
│   └── guardrail/          # Guardrail Engine (Port 8002)
```

---

## Core Capabilities

* **Unified Control Plane**: Orchestrate LLMs across heterogeneous compute (K8s, DePIN, VPS).
* **Policy Enforcement**: Centralized RBAC, safety guardrails, and budget controls.
* **Execution Boundary**: Authority-based routing between applications and infrastructure.

---

For full documentation, architecture diagrams, and deployment guides, visit the [main repository](https://github.com/InferiaAI/InferiaLLM).
