Metadata-Version: 2.4
Name: infinite-context-gateway
Version: 0.1.2
Summary: A hybrid AI context gateway combining Headroom cloud compression and In-Place Local Test-Time Training (TTT).
Author-email: Dev Team <dev@example.com>
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: httpx
Provides-Extra: cloud
Requires-Dist: headroom-ai; extra == "cloud"
Provides-Extra: local
Requires-Dist: torch; extra == "local"
Requires-Dist: transformers; extra == "local"
Requires-Dist: peft; extra == "local"
Requires-Dist: accelerate; extra == "local"
Requires-Dist: bitsandbytes; extra == "local"
Provides-Extra: all
Requires-Dist: headroom-ai; extra == "all"
Requires-Dist: torch; extra == "all"
Requires-Dist: transformers; extra == "all"
Requires-Dist: peft; extra == "all"
Requires-Dist: accelerate; extra == "all"
Requires-Dist: bitsandbytes; extra == "all"

# Infinite Context Gateway

**Infinite Context Gateway** is a production-ready Python library designed to solve the "context window limit" and high API token costs associated with analyzing massive documents. 

Whether you are building RAG pipelines, autonomous agents, or large-scale document analysis tools, this library allows you to bypass standard memory limitations by routing your data through two highly optimized, infinite-context engines:

1. **Cloud Engine**: Uses the open-source `headroom-ai` algorithm to compress massive documents locally for *free* (shrinking token counts by up to 90%), before routing the smaller payload to commercial cloud LLMs (OpenAI, Anthropic, Groq). **Use case**: Cost-effective, high-speed document analysis using state-of-the-art proprietary models.
2. **Local Engine**: Bypasses the KV-Cache entirely using **In-Place Test-Time Training (TTT)**. It injects a PEFT DoRA adapter to bake massive documents directly into the neural weights of local open-source models (like Qwen) running on your own GPU. **Use case**: 100% offline, privacy-preserving document reasoning with zero API costs.

---

## Installation

You can install the package directly from PyPI or GitHub. Choose the extras based on which engine you want to use.

**For Cloud Engine (Lightweight, CPU only):**
```bash
pip install "infinite-context-gateway[cloud]"
```

**For Local Engine (Requires NVIDIA GPU & CUDA):**
```bash
pip install "infinite-context-gateway[local]"
```

---

## Usage

### Option 1: Cloud Engine (Local Compression + Cloud LLM)

When you use the Cloud Engine, the gateway uses the open-source `headroom-ai` library to instantly compress your massive documents **locally on your computer for free**. 

It then sends the shrunken document to a cloud provider like OpenAI, Groq, or Anthropic to answer the question. Because the document was compressed locally, you save massive amounts of money on API token costs!

**Note:** The `api_key` you provide here is for the *Cloud Provider* (like OpenAI or Groq), NOT for Headroom!

```python
from infinite_context import ContextGateway

# Example: Using Groq's super-fast Llama 3 API
gateway = ContextGateway(
    engine="cloud",
    model_id="llama3-70b-8192",          # Target model on Groq
    api_key="your_groq_api_key_here",    # The API key for the Cloud LLM provider
    base_url="https://api.groq.com/openai/v1/chat/completions",
    compression_ratio=0.8                # Compress the document by 80% locally
)

# Pass in a massive document and ask a question!
massive_document = "..."
response = gateway.chat("What is the failover protocol?", massive_document)
print(response)
```

---

### Option 2: Local Engine (Test-Time Training)

When you use the Local Engine, the gateway bypasses the standard KV-Cache memory limits entirely. 

Instead of putting the document into the prompt, it injects a fast PEFT DoRA adapter into a local model (like Qwen 0.5B) and **trains the document directly into the neural weights** over a few seconds. 

This requires absolutely zero internet connection and no API keys.

```python
from infinite_context import ContextGateway

# Loads the model in 4-bit precision into your GPU
gateway = ContextGateway(
    engine="local",
    model_id="Qwen/Qwen2.5-0.5B-Instruct",
    load_in_4bit=True
)

response = gateway.chat("What is the failover protocol?", massive_document)
print(response)
```

#### Checkpoint Persistence
Because the local engine alters the neural weights, you can save the trained "Fast Weights" to your hard drive. You can then reload them days later and ask questions about the massive document *without ever reading the document again*.

```python
# 1. Train the model on the document
gateway.chat("Summarise.", massive_document, keep_state=True)

# 2. Save the brain state
gateway.save_state("./memory_checkpoint")

# 3. Reload it later (zero retraining time)
new_gateway = ContextGateway(engine="local", model_id="Qwen/Qwen2.5-0.5B-Instruct")
new_gateway.load_state("./memory_checkpoint")

# Ask a question without passing the context!
answer = new_gateway.chat("What was the protocol?", context="")
```
