Metadata-Version: 2.4
Name: offline-intelligence
Version: 0.1.7
Summary: Private On-Device Inference Engine — run LLMs locally with zero configuration
Home-page: https://github.com/OfflineIntelligence/offline-intelligence
Author: Offline Intelligence
Author-email: intelligencedevelopment.io@gmail.com
License: Apache-2.0
Project-URL: Bug Tracker, https://github.com/OfflineIntelligence/offline-intelligence/issues
Project-URL: Source, https://github.com/OfflineIntelligence/offline-intelligence
Keywords: llm,ai,inference,offline,local,on-device,private
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Rust
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Intended Audience :: Developers
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: project-url
Dynamic: requires-python
Dynamic: summary

# Offline Intelligence

Private On-Device Inference Engine — run LLMs locally with zero configuration.

## Installation

```bash
pip install offline-intelligence
```

## Quick Start

```python
from offline_intelligence import OfflineIntelligence

# Create SDK — auto-detects your hardware (NVIDIA, AMD, Intel, Apple Silicon)
sdk = OfflineIntelligence()

# Download inference engine (first run only, ~200MB, stored in AppData)
sdk.ensure_ready()

# Load a GGUF model and start inference
sdk.load_model("path/to/model.gguf")

# Chat
response = sdk.chat("What is the capital of France?")
print(response["content"])

# Cleanup
sdk.stop()
sdk.close()
```

## Context Manager

```python
from offline_intelligence import OfflineIntelligence

with OfflineIntelligence() as sdk:
    sdk.ensure_ready()
    sdk.load_model("model.gguf")
    
    # Simple string input
    response = sdk.chat("Hello!")
    print(response["content"])
    
    # Full message format
    response = sdk.chat([
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing in one sentence."}
    ])
    print(response["content"])
```

## Configuration

```python
sdk = OfflineIntelligence({
    "ctx_size": 4096,       # Context window size
    "gpu_layers": 28,       # GPU layers to offload (0 = CPU only)
    "threads": 8,           # CPU threads
    "batch_size": 512,      # Batch size for prompt processing
    "env_file": ".env",     # Optional .env file (does NOT pollute your environment)
})
```

## How It Works

1. **`OfflineIntelligence()`** — Detects your GPU hardware, creates data directories. Instant.
2. **`ensure_ready()`** — Downloads the best inference engine for your hardware. First run only.
3. **`load_model(path)`** — Spawns a local inference server on a free port. Manages all DLLs automatically.
4. **`chat(messages)`** — Sends requests to the local server. All inference stays on your machine.

No data leaves your device. No API keys. No internet required after initial setup.

## Status

```python
from offline_intelligence import SdkStatus

sdk = OfflineIntelligence()
print(sdk.status)  # SdkStatus.NOT_STARTED

sdk.ensure_ready()
print(sdk.status)  # SdkStatus.DEGRADED (engine ready, no model)

sdk.load_model("model.gguf")
print(sdk.status)  # SdkStatus.READY
```

## Platform Support

| Platform | GPU Support |
|----------|------------|
| Windows x64 | NVIDIA (CUDA), AMD (HIP), Intel (SYCL), Vulkan, CPU |
| macOS ARM64 | Apple Metal |
| macOS x64 | CPU |
| Linux x64 | NVIDIA (CUDA), AMD (ROCm), Vulkan, CPU |

## License

Apache 2.0
