Metadata-Version: 2.4
Name: inferiallm
Version: 1.0.5
Summary: InferiaLLM: distributed inference, filtration, and compute orchestration platform
Author: Inferia AI
License: Apache-2.0
Project-URL: Repository, https://github.com/InferiaAI/InferiaLLM
Project-URL: Documentation, https://docs.inferia.ai/docs
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi==0.109.0
Requires-Dist: uvicorn[standard]<0.30,>=0.27
Requires-Dist: httpx>=0.27.0
Requires-Dist: aiohttp>=3.8.5
Requires-Dist: grpcio==1.76.0
Requires-Dist: grpcio-tools==1.76.0
Requires-Dist: protobuf<7.0.0,>=6.31.1
Requires-Dist: anyio>=4.0
Requires-Dist: pydantic>=2.0
Requires-Dist: pydantic-settings>=2.0
Requires-Dist: email-validator>=2.1.0
Requires-Dist: python-jose[cryptography]
Requires-Dist: passlib[bcrypt]
Requires-Dist: bcrypt
Requires-Dist: pyotp
Requires-Dist: qrcode
Requires-Dist: pillow
Requires-Dist: asyncpg==0.31.0
Requires-Dist: psycopg2-binary==2.9.11
Requires-Dist: redis>=5.0
Requires-Dist: sqlalchemy>=2.0
Requires-Dist: greenlet
Requires-Dist: prometheus-client
Requires-Dist: python-dotenv
Requires-Dist: base58==2.1.1
Requires-Dist: pynacl
Requires-Dist: cachetools
Requires-Dist: python-multipart
Requires-Dist: skypilot
Requires-Dist: kubernetes==33.1.0
Provides-Extra: aws
Requires-Dist: boto3==1.42.10; extra == "aws"
Provides-Extra: ml
Requires-Dist: numpy; extra == "ml"
Requires-Dist: transformers>=4.40; extra == "ml"
Requires-Dist: tokenizers>=0.21; extra == "ml"
Requires-Dist: chromadb; extra == "ml"
Requires-Dist: llm-guard; extra == "ml"
Requires-Dist: presidio-analyzer; extra == "ml"
Requires-Dist: spacy; extra == "ml"
Requires-Dist: pypdf; extra == "ml"
Requires-Dist: python-docx; extra == "ml"
Requires-Dist: requests; extra == "ml"
Requires-Dist: groq; extra == "ml"
Provides-Extra: db
Requires-Dist: sqlalchemy>=2.0; extra == "db"
Requires-Dist: alembic>=1.13; extra == "db"
Requires-Dist: greenlet; extra == "db"
Requires-Dist: jinja2; extra == "db"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: pytest-mock; extra == "dev"
Requires-Dist: pytest-trio; extra == "dev"
Requires-Dist: pytest-twisted; extra == "dev"
Requires-Dist: pytest-tornasync; extra == "dev"
Dynamic: license-file


<div align="center">

# InferiaLLM CLI

### The Operating System for LLMs in Production

  [![License](https://img.shields.io/badge/license-Apache--2.0-green?style=flat-square)](https://github.com/InferiaAI/InferiaLLM/blob/main/LICENSE)[![Python](https://img.shields.io/badge/python-3.10+-blue?style=flat-square)](https://www.python.org/)[![Status](https://img.shields.io/badge/status-beta-orange?style=flat-square)]()[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)

</div>

InferiaLLM provides a unified CLI to manage the platform's control plane, initialize infrastructure, and orchestrate gateways (Orchestration, Inference, and Filtration).

---

## Installation

```bash
pip install inferiallm
```

---

## Quick Start

InferiaLLM requires a `.env` file in your current working directory to configure connections to PostgreSQL and Redis.

```bash
# 1. Initialize your environment
# Create a .env file with your DATABASE_URL and Redis settings
cp .env.sample .env

# 2. Bootstrap the platform
# This creates the database, roles, and applying schemas
inferiallm init

# 3. Launch all services
# Starts Orchestration, Inference, and Filtration gateways in a single process
inferiallm api-start
```

---

## Configuration

The CLI manages configuration through environment variables. The most critical settings are:

### 1. Database & Security
| Variable | Description | Default |
| --- | --- | --- |
| `DATABASE_URL` | Primary database connection string | `postgresql://inferia:inferia@localhost:5432/inferia` |
| `PG_ADMIN_USER` | Postgres admin user (required for `init`) | `postgres` |
| `PG_ADMIN_PASSWORD` | Postgres admin password (required for `init`) | - |
| `JWT_SECRET_KEY` | Secret for signing access tokens | - |
| `INTERNAL_API_KEY` | Secret for service-to-service auth | - |
| `SECRET_ENCRYPTION_KEY` | 32-byte base64 key for encrypting credentials | - |

### 2. Provider Specifics (Optional)
Required for provisioning compute from external providers:
* `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`
* `AKASH_MNEMONIC`
* `NOSANA_WALLET_PRIVATE_KEY`

---

## CLI Reference

### `inferiallm init`
Bootstraps the unified database environment.
**Output:**
```text
[inferia:init] Connecting as admin to bootstrap inferia
[inferia:init] Creating role: inferia
[inferia:init] Creating database: inferia
[inferia:init] Applying schema: global_schema
[inferia:init] Bootstrap complete
```

### `inferiallm api-start`
Starts all InferiaLLM gateways (Orchestration, Inference, Filtration) and the Dashboard in one command.

### `inferiallm orchestration-gateway`
Starts the Orchestration Gateway standalone (manages compute and routing).

### `inferiallm inference-gateway`
Starts the Inference Gateway standalone (handles data-plane ingress).

### `inferiallm filtration-gateway`
Starts the Filtration Gateway standalone (enforces RBAC, quotas, and guardrails).

---

## Core Capabilities
* **Unified Control Plane**: Orchestrate LLMs across heterogeneous compute (K8s, DePIN, VPS).
* **Policy Enforcement**: Centralized RBAC, safety guardrails, and budget controls.
* **Execution Boundary**: Authority-based routing between applications and infrastructure.

---

For full documentation, architecture diagrams, and deployment guides, visit the [main repository](https://github.com/InferiaAI/InferiaLLM).
