Metadata-Version: 2.4
Name: everyai-core
Version: 1.0.1
Summary: Connect to every free AI provider with zero setup
Author: Rohan Jangir
License: MIT
Project-URL: Homepage, https://github.com/rohan-jangir/everyai-core
Keywords: ai,llm,groq,openrouter,huggingface,cerebras,mistral,cloudflare,nvidia,fallback,caching,rate-limiting,telemetry
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx>=0.24.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Provides-Extra: local
Requires-Dist: transformers>=4.30.0; extra == "local"
Requires-Dist: torch>=2.0.0; extra == "local"
Dynamic: license-file

# EveryAI Core 🚀

[![PyPI Version](https://img.shields.io/badge/pypi-v1.0.0-blue.svg)](https://pypi.org/project/everyai-core/)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![Python Version](https://img.shields.io/badge/python-%3E%3D%203.10-blue.svg)](pyproject.toml)
[![Security Status](https://img.shields.io/badge/security-hardened-success.svg)](#security--packaging-safety)

Connect to every free and premium AI provider with zero setup, client-side rate-limit governance, local query caching, automatic fallback failover, and a live web telemetry dashboard.

---

## 🌟 Key Features

*   **Unified Developer SDK**: Query Groq, OpenRouter, Nvidia NIM, Mistral, Cerebras, Cloudflare, and Hugging Face (both cloud REST APIs and local execution) with an identical, clean developer syntax.
*   **Smart Fallback Orchestration**: Chain alternative providers or different keys to handle rate limits and service failures smoothly.
*   **Local Response Caching**: SQLite-backed query cache saves tokens and speeds up repetitive workflows.
*   **Rate Limit Governor**: Client-side RPM (Requests Per Minute) and TPM (Tokens Per Minute) tracking queues and paces outbound requests automatically.
*   **Telemetry & Web Dashboard**: Run `everyai dashboard` to visualize token counts, response times, and query statistics.
*   **Local Offline Execution**: Load and run open-weights models locally via PyTorch and Transformers using the same API interface.

---

## 🔌 Supported Providers

| Provider | Environment Variable | Default / Preset Models | Key Features |
| :--- | :--- | :--- | :--- |
| **Groq** | `GROQ_API_KEY` | `llama3-8b-8192`, `llama3-70b-8192`, `mixtral-8x7b-32768` | Ultra-fast LPU inference, native token usage logging |
| **OpenRouter** | `OPENROUTER_API_KEY` | `google/gemini-2.5-flash`, `meta-llama/llama-3-8b-instruct:free` | Aggregated model catalog, free tier routes |
| **Hugging Face (Cloud)** | `HF_TOKEN` / `HUGGINGFACE_API_KEY` | `meta-llama/Meta-Llama-3-8B-Instruct` | Serverless API execution of open-weights models |
| **Hugging Face (Local)** | *(None required)* | `gpt2`, `facebook/opt-125m`, any local repo path | Runs models locally on CPU/GPU via PyTorch |
| **Cerebras** | `CEREBRAS_API_KEY` | `llama3.1-8b`, `llama3.1-70b` | High-speed wafer-scale engine inference |
| **Mistral** | `MISTRAL_API_KEY` | `mistral-large-latest`, `open-mixtral-8x22b` | European flagship models, native streaming |
| **Cloudflare** | `CLOUDFLARE_API_TOKEN`, `CLOUDFLARE_ACCOUNT_ID` | `@cf/meta/llama-3-8b-instruct`, `@cf/mistral/mistral-7b` | Workers AI serverless endpoints |
| **Nvidia NIM** | `NVIDIA_API_KEY` | `meta/llama-3.1-405b-instruct` | GPU-optimized microservice inference |

---

## 📦 Installation

Install the core library via `pip`:

```bash
pip install everyai-core
```

To enable local model inference capabilities (requires PyTorch and Transformers):

```bash
pip install everyai-core[local]
```

---

## ⚡ Quick Start

### 1. Set Your Environment Variables
EveryAI will automatically search your environment for keys. Add them to your shell or `.env` file:

```bash
export GROQ_API_KEY="gsk_..."
export OPENROUTER_API_KEY="sk-or-..."
export CEREBRAS_API_KEY="cbs_..."
```

### 2. Basic Non-Streaming Request
```python
from everyai_core import EveryAI

# Initialize client (uses environment keys by default)
client = EveryAI()

# Run a completion
response = client.chat(
    provider="groq",
    model="llama3-8b-8192",
    messages=[
        {"role": "user", "content": "Explain quantum computing in one sentence."}
    ],
    temperature=0.7
)

print(f"[{response.provider.upper()}] {response.choices[0].message['content']}")
print(f"Tokens Used: {response.usage.total_tokens}")
```

---

## 📘 Detailed Module & Feature Guide

### 1. Provider-Specific Direct Clients
You can access provider SDK wrappers directly through client attributes for full autocomplete and static typing support:

```python
from everyai_core import EveryAI

client = EveryAI(api_keys={"groq": "your_custom_key_here"})

# Access Groq direct instance
response = client.groq.chat(
    model="llama3-8b-8192",
    messages=[{"role": "user", "content": "Hi!"}]
)
```

---

### 2. Streaming Response Tokens
For real-time responses, pass `stream=True` to receive a generator yielding chunk responses:

```python
from everyai_core import EveryAI

client = EveryAI()

chunks = client.chat(
    provider="openrouter",
    model="google/gemini-2.5-flash",
    messages=[{"role": "user", "content": "Write a short poem about coding."}],
    stream=True
)

for chunk in chunks:
    content = chunk.choices[0].message.get("content", "")
    print(content, end="", flush=True)
```

---

### 3. Smart Fallback Failovers & Autopilot Modes
To combat rate limits, API downtime, or key exhausts, configure fallback lists. If the first provider fails, EveryAI automatically routes the prompt to the next option.

#### Manual Fallback Chain
```python
response = client.chat(
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    fallback_chain=[
        {"provider": "groq", "model": "llama3-70b-8192"},
        {"provider": "cerebras", "model": "llama3.1-70b"},
        {"provider": "openrouter", "model": "meta-llama/llama-3-8b-instruct:free"}
    ]
)
print(f"Resolved via: {response.provider}")
```

#### Autopilot Presets
Use the built-in preset strings under the `mode` parameter:
*   `"fastest"`: Groq -> OpenRouter -> Cloudflare
*   `"smartest"`: Groq Llama-70b -> OpenRouter Gemini-Flash -> Nvidia NIM
*   `"balanced"`: Mix of fast and smart models

```python
response = client.chat(
    messages=[{"role": "user", "content": "Compose a script."}],
    mode="fastest"
)
```

---

### 4. Client-Side Throttling (Rate Governor)
Pace outbound API requests client-side to prevent hitting provider rate limit caps.

```python
from everyai_core import EveryAI

# Configure 10 requests and 5,000 tokens maximum per minute
client = EveryAI(
    max_requests_per_minute=10,
    max_tokens_per_minute=5000
)

for i in range(12):
    response = client.chat(
        provider="groq",
        model="llama3-8b-8192",
        messages=[{"role": "user", "content": f"Ping number {i}"}]
    )
    # The governor will block and throttle execution if limits are exceeded
```

---

### 5. Local Query Cache
Enable local request caching backed by SQLite. Identical prompt inputs with matching model settings are resolved instantly without hitting endpoints, conserving tokens and avoiding rate limits.

```python
from everyai_core import EveryAI

# Enable caching globally
client = EveryAI(cache=True, cache_path="./my_cache.db")

# First call: hits Groq API (takes ~500ms)
res1 = client.chat(
    provider="groq",
    model="llama3-8b-8192",
    messages=[{"role": "user", "content": "Calculate 25 * 40."}]
)

# Second call: resolved instantly from local SQLite cache (~1ms)
res2 = client.chat(
    provider="groq",
    model="llama3-8b-8192",
    messages=[{"role": "user", "content": "Calculate 25 * 40."}]
)
```

---

### 6. Local Offline Models
Run open-weights models on your local machine using PyTorch and Hugging Face Transformers.

```python
from everyai_core import EveryAI

client = EveryAI()

# Run OPT-125m locally on CPU/GPU
response = client.chat(
    provider="huggingface",
    model="facebook/opt-125m",
    messages=[{"role": "user", "content": "The weather today is"}],
    local=True  # Enables local transformers inference
)
print(response.choices[0].message["content"])
```

---

### 7. Telemetry & Web Dashboard

All token counts, call latencies, success rates, and rate limit errors are recorded in a local SQLite database.

#### Query Telemetry via Code:
```python
from everyai_core import EveryAI

client = EveryAI()
# Retrieve telemetry summary
summary = client.tracker.get_summary()

print(f"Total Requests: {summary['total_calls']}")
print(f"Cache Hits: {summary['cache_hits_total']}")
print(f"Tokens Saved: {summary['tokens_saved_total']}")
```

#### Launch the Web Telemetry Dashboard:
Boot the built-in HTTP server to view real-time statistics, charts, and detailed transaction logs in your web browser:

```bash
everyai dashboard --port 8080
```

Alternatively, print a quick tabular report of statistics directly in your terminal:
```bash
everyai stats
```

---

## 🛠️ Error Handling

EveryAI automatically maps raw HTTP status codes from all providers to clean, standardized exceptions:

```python
from everyai_core import EveryAI
from everyai_core.exceptions import RateLimitError, AuthenticationError, ModelNotFoundError

client = EveryAI()

try:
    client.chat(
        provider="groq",
        model="non-existent-model",
        messages=[{"role": "user", "content": "Hello"}]
    )
except ModelNotFoundError as e:
    print(f"Requested model was not found: {e}")
except RateLimitError:
    print("Rate limit reached. Pacing requests...")
except AuthenticationError:
    print("Invalid API credentials. Check your keys.")
```

---

## 🔒 Security & Packaging Safety

EveryAI is designed with production safety and security in mind:
1.  **No Leaked Credentials**: API keys are handled entirely in-memory and are never written to telemetry databases, logs, or disk.
2.  **No Prompt Logging**: Telemetry logs record only metadata (token count, provider, model name, status, execution duration). Your sensitive prompts and responses are **never** logged to the telemetry database.
3.  **Strict Package Bundling**: The library's `MANIFEST.in` explicitly excludes local developer `.env` files, temporary verification scripts, cache SQLite database stores, and unit tests. You can deploy it to public directories with confidence.

---

## 📄 License

This project is licensed under the MIT License. See [LICENSE](LICENSE) for details.
