Metadata-Version: 2.4
Name: inferiallm
Version: 1.0.8
Summary: InferiaLLM: distributed inference, filtration, and compute orchestration platform
Author: Inferia AI
License: Apache 2.0
Project-URL: Repository, https://github.com/InferiaAI/InferiaLLM
Project-URL: Documentation, https://docs.inferia.ai/docs
Requires-Python: <3.13,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi==0.109.0
Requires-Dist: uvicorn[standard]<0.30,>=0.27
Requires-Dist: httpx>=0.27.0
Requires-Dist: aiohttp>=3.8.5
Requires-Dist: grpcio==1.76.0
Requires-Dist: grpcio-tools==1.76.0
Requires-Dist: protobuf<7.0.0,>=6.31.1
Requires-Dist: anyio>=4.0
Requires-Dist: pydantic>=2.0
Requires-Dist: pydantic-settings>=2.0
Requires-Dist: email-validator>=2.1.0
Requires-Dist: python-jose[cryptography]
Requires-Dist: passlib[bcrypt]
Requires-Dist: bcrypt
Requires-Dist: pyotp
Requires-Dist: qrcode
Requires-Dist: pillow
Requires-Dist: asyncpg==0.31.0
Requires-Dist: psycopg2-binary==2.9.11
Requires-Dist: redis>=5.0
Requires-Dist: sqlalchemy>=2.0
Requires-Dist: alembic>=1.13
Requires-Dist: greenlet
Requires-Dist: jinja2
Requires-Dist: numpy
Requires-Dist: transformers>=4.40
Requires-Dist: tokenizers>=0.21
Requires-Dist: chromadb
Requires-Dist: llm-guard
Requires-Dist: presidio-analyzer
Requires-Dist: spacy
Requires-Dist: pypdf
Requires-Dist: python-docx
Requires-Dist: requests
Requires-Dist: groq
Requires-Dist: prometheus-client
Requires-Dist: python-dotenv
Requires-Dist: base58==2.1.1
Requires-Dist: pynacl
Requires-Dist: cachetools
Requires-Dist: python-multipart
Requires-Dist: skypilot
Requires-Dist: kubernetes==33.1.0
Requires-Dist: sqlalchemy>=2.0
Requires-Dist: alembic>=1.13
Requires-Dist: greenlet
Requires-Dist: jinja2
Provides-Extra: aws
Requires-Dist: boto3==1.42.10; extra == "aws"
Requires-Dist: skypilot[aws]; extra == "aws"
Provides-Extra: gcp
Requires-Dist: skypilot[gcp]; extra == "gcp"
Provides-Extra: azure
Requires-Dist: skypilot[azure]; extra == "azure"
Provides-Extra: lambda
Requires-Dist: skypilot[lambda]; extra == "lambda"
Provides-Extra: runpod
Requires-Dist: skypilot[runpod]; extra == "runpod"
Provides-Extra: all-providers
Requires-Dist: boto3==1.42.10; extra == "all-providers"
Requires-Dist: skypilot[aws,azure,gcp,kubernetes,lambda,runpod]; extra == "all-providers"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: pytest-mock; extra == "dev"
Requires-Dist: pytest-trio; extra == "dev"
Requires-Dist: pytest-twisted; extra == "dev"
Requires-Dist: pytest-tornasync; extra == "dev"
Dynamic: license-file


<div align="center">

# InferiaLLM

### The Operating System for LLMs in Production

  [![License](https://img.shields.io/badge/license-Apache--2.0-green?style=flat-square)](https://github.com/InferiaAI/InferiaLLM/blob/main/LICENSE)[![Python](https://img.shields.io/badge/python-3.10+-blue?style=flat-square)](https://www.python.org/)[![Status](https://img.shields.io/badge/status-beta-orange?style=flat-square)]()[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)

</div>

InferiaLLM acts as the authoritative execution layer between your applications and your AI infrastructure. It governs how LLMs are accessed, secured, routed, and run on compute.


---

## Installation

```bash
pip install inferiallm
```

---

## Quick Start

InferiaLLM requires a `.env` file in your current working directory to configure connections to PostgreSQL and Redis.

```bash
# 1. Initialize your environment
# Create a .env file with your DATABASE_URL and Redis settings
cp .env.sample .env

# 2. Bootstrap the platform
# This creates the database, roles, and applying schemas
inferiallm init

# 3. Launch all services
# Starts Orchestration, Inference, and Filtration gateways in a single process
inferiallm start
```

---

## Configuration

The CLI manages configuration through environment variables. The most critical settings are:

### 1. Database & Security
| Variable | Description | Default |
| --- | --- | --- |
| `DATABASE_URL` | Primary database connection string | `postgresql://inferia:inferia@localhost:5432/inferia` |
| `PG_ADMIN_USER` | Postgres admin user (required for `init`) | `postgres` |
| `PG_ADMIN_PASSWORD` | Postgres admin password (required for `init`) | - |
| `JWT_SECRET_KEY` | Secret for signing access tokens | - |
| `INTERNAL_API_KEY` | Secret for service-to-service auth | - |
| `SECRET_ENCRYPTION_KEY` | 32-byte base64 key for encrypting credentials | - |

---

## CLI Reference

### `inferiallm init`
Bootstraps the unified database environment.
**Output:**
```text
[inferia:init] Connecting as admin to bootstrap inferia
[inferia:init] Creating role: inferia
[inferia:init] Creating database: inferia
[inferia:init] Applying schema: global_schema
[inferia:init] Bootstrap complete
```

### `inferiallm start`
Starts all InferiaLLM gateways (Orchestration, Inference, Filtration) and the Dashboard in one command.

You can also start specific services:
* `inferiallm start orchestration`: Starts the Orchestration Gateway stack.
* `inferiallm start inference`: Starts the Inference Gateway standalone.
* `inferiallm start filtration`: Starts the Filtration Gateway standalone.

---

## Package Structure

The `inferia` package is a monorepo-style library that contains all backend services:

```text
package/src/inferia/
├── cli.py                  # Entry point for the CLI
├── gateways/
│   ├── filtration_gateway  # Security & Policy Service (Port 8000)
│   ├── inference_gateway   # Inference Proxy Service (Port 8001)
│   └── orchestration_gateway # Compute Control Plane (Port 8080)
└── services/               # Shared business logic
    ├── filtration/         # Guardrails, RBAC, Audit logic
    └── orchestration/      # Compute Adapters, Scheduling logic
```

---

## Core Capabilities
* **Unified Control Plane**: Orchestrate LLMs across heterogeneous compute (K8s, DePIN, VPS).
* **Policy Enforcement**: Centralized RBAC, safety guardrails, and budget controls.
* **Execution Boundary**: Authority-based routing between applications and infrastructure.

---

For full documentation, architecture diagrams, and deployment guides, visit the [main repository](https://github.com/InferiaAI/InferiaLLM).
