Metadata-Version: 2.4
Name: ollabridge
Version: 0.1.3
Summary: Turn your PC into a private, OpenAI-compatible LLM provider in ~60 seconds
Project-URL: Homepage, https://github.com/ruslanmv/ollabridge
Project-URL: Documentation, https://github.com/ruslanmv/ollabridge#readme
Project-URL: Repository, https://github.com/ruslanmv/ollabridge
Project-URL: Issues, https://github.com/ruslanmv/ollabridge/issues
Project-URL: Changelog, https://github.com/ruslanmv/ollabridge/releases
Author-email: Ruslan Magana Vsevolodovna <contact@ruslanmv.com>
Maintainer-email: Ruslan Magana Vsevolodovna <contact@ruslanmv.com>
License: 
                                         Apache License
                                   Version 2.0, January 2004
                                http://www.apache.org/licenses/
        
           TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
        
           1. Definitions.
        
              "License" shall mean the terms and conditions for use, reproduction,
              and distribution as defined by Sections 1 through 9 of this document.
        
              "Licensor" shall mean the copyright owner or entity authorized by
              the copyright owner that is granting the License.
        
              "Legal Entity" shall mean the union of the acting entity and all
              other entities that control, are controlled by, or are under common
              control with that entity. For the purposes of this definition,
              "control" means (i) the power, direct or indirect, to cause the
              direction or management of such entity, whether by contract or
              otherwise, or (ii) ownership of fifty percent (50%) or more of the
              outstanding shares, or (iii) beneficial ownership of such entity.
        
              "You" (or "Your") shall mean an individual or Legal Entity
              exercising permissions granted by this License.
        
              "Source" form shall mean the preferred form for making modifications,
              including but not limited to software source code, documentation
              source, and configuration files.
        
              "Object" form shall mean any form resulting from mechanical
              transformation or translation of a Source form, including but
              not limited to compiled object code, generated documentation,
              and conversions to other media types.
        
              "Work" shall mean the work of authorship, whether in Source or
              Object form, made available under the License, as indicated by a
              copyright notice that is included in or attached to the work
              (an example is provided in the Appendix below).
        
              "Derivative Works" shall mean any work, whether in Source or Object
              form, that is based on (or derived from) the Work and for which the
              editorial revisions, annotations, elaborations, or other modifications
              represent, as a whole, an original work of authorship. For the purposes
              of this License, Derivative Works shall not include works that remain
              separable from, or merely link (or bind by name) to the interfaces of,
              the Work and Derivative Works thereof.
        
              "Contribution" shall mean any work of authorship, including
              the original version of the Work and any modifications or additions
              to that Work or Derivative Works thereof, that is intentionally
              submitted to Licensor for inclusion in the Work by the copyright owner
              or by an individual or Legal Entity authorized to submit on behalf of
              the copyright owner. For the purposes of this definition, "submitted"
              means any form of electronic, verbal, or written communication sent
              to the Licensor or its representatives, including but not limited to
              communication on electronic mailing lists, source code control systems,
              and issue tracking systems that are managed by, or on behalf of, the
              Licensor for the purpose of discussing and improving the Work, but
              excluding communication that is conspicuously marked or otherwise
              designated in writing by the copyright owner as "Not a Contribution."
        
              "Contributor" shall mean Licensor and any individual or Legal Entity
              on behalf of whom a Contribution has been received by Licensor and
              subsequently incorporated within the Work.
        
           2. Grant of Copyright License. Subject to the terms and conditions of
              this License, each Contributor hereby grants to You a perpetual,
              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
              copyright license to reproduce, prepare Derivative Works of,
              publicly display, publicly perform, sublicense, and distribute the
              Work and such Derivative Works in Source or Object form.
        
           3. Grant of Patent License. Subject to the terms and conditions of
              this License, each Contributor hereby grants to You a perpetual,
              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
              (except as stated in this section) patent license to make, have made,
              use, offer to sell, sell, import, and otherwise transfer the Work,
              where such license applies only to those patent claims licensable
              by such Contributor that are necessarily infringed by their
              Contribution(s) alone or by combination of their Contribution(s)
              with the Work to which such Contribution(s) was submitted. If You
              institute patent litigation against any entity (including a
              cross-claim or counterclaim in a lawsuit) alleging that the Work
              or a Contribution incorporated within the Work constitutes direct
              or contributory patent infringement, then any patent licenses
              granted to You under this License for that Work shall terminate
              as of the date such litigation is filed.
        
           4. Redistribution. You may reproduce and distribute copies of the
              Work or Derivative Works thereof in any medium, with or without
              modifications, and in Source or Object form, provided that You
              meet the following conditions:
        
              (a) You must give any other recipients of the Work or
                  Derivative Works a copy of this License; and
        
              (b) You must cause any modified files to carry prominent notices
                  stating that You changed the files; and
        
              (c) You must retain, in the Source form of any Derivative Works
                  that You distribute, all copyright, patent, trademark, and
                  attribution notices from the Source form of the Work,
                  excluding those notices that do not pertain to any part of
                  the Derivative Works; and
        
              (d) If the Work includes a "NOTICE" text file as part of its
                  distribution, then any Derivative Works that You distribute must
                  include a readable copy of the attribution notices contained
                  within such NOTICE file, excluding those notices that do not
                  pertain to any part of the Derivative Works, in at least one
                  of the following places: within a NOTICE text file distributed
                  as part of the Derivative Works; within the Source form or
                  documentation, if provided along with the Derivative Works; or,
                  within a display generated by the Derivative Works, if and
                  wherever such third-party notices normally appear. The contents
                  of the NOTICE file are for informational purposes only and
                  do not modify the License. You may add Your own attribution
                  notices within Derivative Works that You distribute, alongside
                  or as an addendum to the NOTICE text from the Work, provided
                  that such additional attribution notices cannot be construed
                  as modifying the License.
        
              You may add Your own copyright statement to Your modifications and
              may provide additional or different license terms and conditions
              for use, reproduction, or distribution of Your modifications, or
              for any such Derivative Works as a whole, provided Your use,
              reproduction, and distribution of the Work otherwise complies with
              the conditions stated in this License.
        
           5. Submission of Contributions. Unless You explicitly state otherwise,
              any Contribution intentionally submitted for inclusion in the Work
              by You to the Licensor shall be under the terms and conditions of
              this License, without any additional terms or conditions.
              Notwithstanding the above, nothing herein shall supersede or modify
              the terms of any separate license agreement you may have executed
              with Licensor regarding such Contributions.
        
           6. Trademarks. This License does not grant permission to use the trade
              names, trademarks, service marks, or product names of the Licensor,
              except as required for reasonable and customary use in describing the
              origin of the Work and reproducing the content of the NOTICE file.
        
           7. Disclaimer of Warranty. Unless required by applicable law or
              agreed to in writing, Licensor provides the Work (and each
              Contributor provides its Contributions) on an "AS IS" BASIS,
              WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
              implied, including, without limitation, any warranties or conditions
              of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
              PARTICULAR PURPOSE. You are solely responsible for determining the
              appropriateness of using or redistributing the Work and assume any
              risks associated with Your exercise of permissions under this License.
        
           8. Limitation of Liability. In no event and under no legal theory,
              whether in tort (including negligence), contract, or otherwise,
              unless required by applicable law (such as deliberate and grossly
              negligent acts) or agreed to in writing, shall any Contributor be
              liable to You for damages, including any direct, indirect, special,
              incidental, or consequential damages of any character arising as a
              result of this License or out of the use or inability to use the
              Work (including but not limited to damages for loss of goodwill,
              work stoppage, computer failure or malfunction, or any and all
              other commercial damages or losses), even if such Contributor
              has been advised of the possibility of such damages.
        
           9. Accepting Warranty or Additional Liability. While redistributing
              the Work or Derivative Works thereof, You may choose to offer,
              and charge a fee for, acceptance of support, warranty, indemnity,
              or other liability obligations and/or rights consistent with this
              License. However, in accepting such obligations, You may act only
              on Your own behalf and on Your sole responsibility, not on behalf
              of any other Contributor, and only if You agree to indemnify,
              defend, and hold each Contributor harmless for any liability
              incurred by, or claims asserted against, such Contributor by reason
              of your accepting any such warranty or additional liability.
        
           END OF TERMS AND CONDITIONS
        
           APPENDIX: How to apply the Apache License to your work.
        
              To apply the Apache License to your work, attach the following
              boilerplate notice, with the fields enclosed by brackets "[]"
              replaced with your own identifying information. (Don't include
              the brackets!)  The text should be enclosed in the appropriate
              comment syntax for the file format. We also recommend that a
              file or class name and description of purpose be included on the
              same "printed page" as the copyright notice for easier
              identification within third-party archives.
        
           Copyright [yyyy] [name of copyright owner]
        
           Licensed under the Apache License, Version 2.0 (the "License");
           you may not use this file except in compliance with the License.
           You may obtain a copy of the License at
        
               http://www.apache.org/licenses/LICENSE-2.0
        
           Unless required by applicable law or agreed to in writing, software
           distributed under the License is distributed on an "AS IS" BASIS,
           WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
           See the License for the specific language governing permissions and
           limitations under the License.
License-File: LICENSE
Keywords: ai,api,chatgpt,deepseek,embeddings,gateway,langchain,llama,llm,local-first,machine-learning,ollama,openai,self-hosted
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: FastAPI
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Internet :: WWW/HTTP :: HTTP Servers
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Requires-Dist: anyio
Requires-Dist: fastapi>=0.110
Requires-Dist: httpx>=0.26
Requires-Dist: itsdangerous>=2.1
Requires-Dist: pydantic-settings>=2.2
Requires-Dist: pydantic>=2.6
Requires-Dist: python-dotenv>=1.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.7
Requires-Dist: slowapi
Requires-Dist: sqlalchemy
Requires-Dist: sqlmodel
Requires-Dist: tenacity>=8.2
Requires-Dist: typer>=0.12
Requires-Dist: uvicorn[standard]>=0.27
Requires-Dist: websockets>=12.0
Provides-Extra: dev
Requires-Dist: black>=23.7; extra == 'dev'
Requires-Dist: mypy>=1.4; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21; extra == 'dev'
Requires-Dist: pytest-cov>=4.1; extra == 'dev'
Requires-Dist: pytest>=7.4; extra == 'dev'
Requires-Dist: ruff>=0.0.280; extra == 'dev'
Provides-Extra: notebook
Requires-Dist: ipykernel>=7.1.0; extra == 'notebook'
Requires-Dist: openai>=2.14.0; extra == 'notebook'
Requires-Dist: python-dotenv>=1.2.1; extra == 'notebook'
Requires-Dist: requests>=2.32.5; extra == 'notebook'
Description-Content-Type: text/markdown

<div align="center">

<img src="assets/logo.svg" alt="OllaBridge Logo" width="500"/>

# OllaBridge ⚡️

**Your single gateway to ALL your LLMs — local, remote, anywhere.**

[![PyPI version](https://badge.fury.io/py/ollabridge.svg)](https://badge.fury.io/py/ollabridge)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

[Quick Start](#-60-second-start) • [Why OllaBridge](#-why-ollabridge) • [Distributed Compute](#-add-any-gpu-in-60-seconds) • [Examples](#-use-it-anywhere) • [Demo Client](#-try-the-interactive-demo-client) • [MCP Mode](#-ai-agents-love-ollabridge)

</div>

---

## 🎯 What is OllaBridge?

> **One gateway. All your LLMs. Everywhere.**

OllaBridge is your **single, OpenAI-compatible API** for every LLM you run — on your laptop, workstation, free GPU servers, cloud instances, anywhere.

**The Problem:** You have models running everywhere (laptop, cloud GPU, friend's gaming PC), and every app needs different configs.

**OllaBridge Solution:** Apps connect to ONE place. OllaBridge routes to the right compute automatically.

```mermaid
graph TB
    A[Your Apps] -->|OpenAI SDK| B[OllaBridge<br/>Control Plane]

    B -->|Auto Routes| C[Local Laptop<br/>llama3.1]
    B -->|Auto Routes| D[Free GPU Cloud<br/>deepseek-r1]
    B -->|Auto Routes| E[Remote Workstation<br/>mixtral]

    C -.->|Dials Out| B
    D -.->|Dials Out| B
    E -.->|Dials Out| B

    style B fill:#6366f1,stroke:#4f46e5,stroke-width:3px,color:#fff
    style A fill:#10b981,stroke:#059669,stroke-width:2px,color:#fff
    style C fill:#8b5cf6,stroke:#7c3aed,stroke-width:2px,color:#fff
    style D fill:#ec4899,stroke:#db2777,stroke-width:2px,color:#fff
    style E fill:#f59e0b,stroke:#d97706,stroke-width:2px,color:#fff
```

**Key Innovation:** Compute nodes **dial out** to your gateway. No port forwarding, no VPN, no config hell.

---

## 🚀 Why OllaBridge?

### 🎯 **Single Source of Truth**
- ✅ **One URL for everything** — Your apps never change code
- ✅ **Zero config** — Add new GPUs without touching your app
- ✅ **Smart routing** — OllaBridge picks the best node automatically
- ✅ **OpenAI compatible** — Works with any SDK, framework, or tool

### 🛡️ **Enterprise-Grade Security**
- ✅ **API key authentication** — Protect your LLMs
- ✅ **Rate limiting** — Control usage per key
- ✅ **Request logging** — Full audit trail
- ✅ **Encrypted connections** — TLS for remote nodes

### 🌍 **Works Everywhere**
- ✅ **Free GPU clouds** — Colab, Kaggle, Lightning AI (no port forwarding needed!)
- ✅ **Ephemeral instances** — Nodes dial out, IPs don't matter
- ✅ **Behind firewalls** — Your laptop can join from coffee shop WiFi
- ✅ **Mixed environments** — Combine local + cloud seamlessly

### 🤖 **AI Agent Ready**
- ✅ **MCP server** — Agents can control your infrastructure
- ✅ **Tool exposure** — Manage nodes, routes, health via tools
- ✅ **Self-healing** — Auto-install, auto-configure, auto-recover

---

## ⚡ 60-Second Start

### Step 1: Install

```bash
pip install ollabridge
```

### Step 2: Start Your Gateway

```bash
ollabridge start
```

**That's it!** You'll see:

```
✅ Ollama installed (if needed)
✅ Model downloaded (if needed)
✅ Gateway online at http://localhost:11435

╭─────────────────── 🚀 Gateway Ready ────────────────────╮
│                                                          │
│ ✅ OllaBridge is Online                                  │
│                                                          │
│ Model:        deepseek-r1                                │
│ Local API:    http://localhost:11435/v1                 │
│ Key:          sk-ollabridge-xY9kL2mN8pQ4rT6vW1zA        │
│                                                          │
│ Node join token:  eyJ0eXAi...                           │
│ Example node command:                                    │
│   ollabridge-node join --control http://localhost:11435 │
│                        --token eyJ0eXAi...              │
│                                                          │
╰──────────────────────────────────────────────────────────╯
```

### Step 3: Use It!

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11435/v1",
    api_key="sk-ollabridge-xY9kL2mN8pQ4rT6vW1zA"
)

response = client.chat.completions.create(
    model="deepseek-r1",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)
```

**Done!** You're running private LLMs with the OpenAI API.

---

## 🌍 Add Any GPU in 60 Seconds

Have a free GPU on Colab? A remote workstation? Add it instantly:

### On Your Remote GPU/Machine:

```bash
# Install
pip install ollabridge

# Join your gateway (copy the command from gateway startup)
ollabridge-node join \
  --control http://YOUR_GATEWAY_IP:11435 \
  --token eyJ0eXAi...
```

**That's it!** The remote GPU:
- ✅ Auto-installs Ollama if needed
- ✅ Auto-downloads models if needed
- ✅ **Dials out** to your gateway (no port forwarding!)
- ✅ Shows up as available compute

### Your Apps See It Automatically

```python
# Same code, now uses both local + remote GPU!
client = OpenAI(base_url="http://localhost:11435/v1", ...)
response = client.chat.completions.create(...)  # Auto-routed
```

**OllaBridge routes requests** across all your nodes automatically.

---

## 🎯 Real-World Scenarios

### Scenario 1: "I have a gaming PC at home"

```bash
# On your gaming PC:
ollabridge-node join --control https://your-gateway.com --token ...

# Now your laptop can use your gaming PC's GPU
# Even if you're at a coffee shop!
```

### Scenario 2: "I want to use free Colab GPUs"

```python
# In Colab notebook:
!pip install ollabridge
!ollabridge-node join --control https://your-gateway.com --token ...

# Now your production app can use free Colab compute
# Colab session ends? Start a new one. Zero config changes.
```

### Scenario 3: "I have multiple cloud GPUs"

```bash
# Each GPU instance:
ollabridge-node join --control https://gateway.company.com --token ...

# Your team shares one API URL
# OllaBridge load-balances across all GPUs
```

---

## 💻 Use It Anywhere

### Python (OpenAI SDK)

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11435/v1",
    api_key="your-key-here"
)

# Chat
response = client.chat.completions.create(
    model="deepseek-r1",
    messages=[{"role": "user", "content": "Explain quantum computing"}]
)

# Embeddings
embeddings = client.embeddings.create(
    model="nomic-embed-text",
    input="Hello, world!"
)
```

### Node.js / TypeScript

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:11435/v1",
  apiKey: process.env.OLLABRIDGE_KEY
});

const completion = await client.chat.completions.create({
  model: "deepseek-r1",
  messages: [{ role: "user", content: "Hello!" }]
});
```

### LangChain

```python
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="http://localhost:11435/v1",
    api_key="your-key-here",
    model="deepseek-r1"
)

response = llm.invoke("What is the meaning of life?")
```

### cURL

```bash
curl -X POST http://localhost:11435/v1/chat/completions \
  -H "Authorization: Bearer your-key-here" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-r1",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

**Works with ANY OpenAI-compatible tool or library.**

---

## 🎨 Try the Interactive Demo Client

Want to see OllaBridge in action? Check out our **2-click demo client** in the `example/` folder!

### ⚡ Quick Start

```bash
# 1. Install and start OllaBridge
cd example
./install-ollabridge.sh  # Mac/Linux
# or
.\install-ollabridge.ps1  # Windows

# 2. Run the demo client
make run

# 3. Open http://localhost:3000 in your browser
```

### ✨ Features

- 🎯 **Beautiful UI** — Modern, responsive web interface
- 🔌 **Real Integration** — Actual API calls to OllaBridge endpoints
- 📊 **Live Metrics** — Request stats, latency, uptime tracking
- 🔑 **Auth Demo** — See how API key authentication works
- 📝 **Best Practices** — Production-ready code examples

### 📚 Perfect for Learning

The example client shows you:
- ✅ How to connect to OllaBridge from a browser
- ✅ How to handle CORS properly
- ✅ How to implement authentication with API keys
- ✅ How to load models dynamically
- ✅ How to send chat requests and handle responses

**[View Full Documentation →](example/README.md)**

---

## 🤖 AI Agents Love OllaBridge

OllaBridge has a **Model Context Protocol (MCP) server** built-in.

Agents can:
- ✅ Create enrollment tokens
- ✅ List connected compute nodes
- ✅ Check gateway health
- ✅ Manage your LLM infrastructure via tools

### Start MCP Server

```bash
ollabridge-mcp
```

### Example: Agent Workflow

```python
# Agent can call these tools:
await session.call_tool("ollabridge.enroll.create", {})
# → Returns enrollment token

await session.call_tool("ollabridge.runtimes.list", {})
# → Shows all connected nodes

await session.call_tool("ollabridge.gateway.health", {})
# → Checks gateway status
```

**Use Case:** "Hey Claude, add my workstation's GPU to our LLM gateway"

→ Agent creates token, gives you the command, you run it. Done.

---

## 🔐 Security & Configuration

### Authentication

OllaBridge auto-generates a secure API key on first run (saved in `.env`):

```env
API_KEYS=sk-ollabridge-xY9kL2mN8pQ4rT6vW1zA
```

Use it in your apps:

```python
# Option 1: Bearer token
headers = {"Authorization": "Bearer sk-ollabridge-..."}

# Option 2: Custom header
headers = {"X-API-Key": "sk-ollabridge-..."}
```

### Configuration (`.env`)

```env
# API Keys (comma-separated for multiple)
API_KEYS=sk-ollabridge-abc123,sk-ollabridge-def456

# Server
HOST=0.0.0.0
PORT=11435

# Default models
DEFAULT_MODEL=deepseek-r1
DEFAULT_EMBED_MODEL=nomic-embed-text

# Rate limiting
RATE_LIMIT=60/minute

# Security
ENROLLMENT_SECRET=your-secret-here
ENROLLMENT_TTL_SECONDS=3600

# Database (optional)
DATABASE_URL=postgresql://user:pass@localhost/ollabridge
```

### Enrollment Tokens

Create short-lived tokens for nodes to join:

```bash
ollabridge enroll-create --ttl 3600
```

Tokens expire automatically for security.

---

## 📡 API Reference

### Core Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/health` | GET | Gateway health + node count |
| `/v1/chat/completions` | POST | OpenAI-compatible chat |
| `/v1/embeddings` | POST | Generate embeddings |
| `/v1/models` | GET | List available models (aggregated from nodes) |

### Admin Endpoints (require API key)

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/admin/recent` | GET | Recent request logs |
| `/admin/runtimes` | GET | List connected nodes |
| `/admin/enroll` | POST | Create enrollment token |

### Example: Check Connected Nodes

```bash
curl -H "X-API-Key: your-key" http://localhost:11435/admin/runtimes
```

**Response:**
```json
{
  "runtimes": [
    {
      "node_id": "local",
      "connector": "local_ollama",
      "healthy": true,
      "tags": ["local"],
      "models": ["deepseek-r1", "llama3.1"]
    },
    {
      "node_id": "colab-gpu-1",
      "connector": "relay_link",
      "healthy": true,
      "tags": ["gpu", "free"],
      "models": ["mixtral", "codellama"]
    }
  ]
}
```

---

## 🏗️ Architecture Deep Dive

### How It Works

1. **Control Plane (Gateway)**: Your apps connect here
2. **Nodes**: Any machine with GPUs/CPUs running models
3. **Relay Link**: Nodes dial OUT to gateway (WebSocket)
4. **Router**: Picks the best node for each request

### Why "Dial Out" Matters

**Traditional (broken):**
```
App → Gateway → Try to reach GPU
                ❌ Blocked by firewall
                ❌ NAT issues
                ❌ No public IP
```

**OllaBridge (works everywhere):**
```
App → Gateway ← GPU dials in
               ✅ Works from anywhere
               ✅ No port forwarding
               ✅ Ephemeral IPs OK
```

### Connector Types

- **RelayLink**: Node dials out via WebSocket (default, works everywhere)
- **DirectEndpoint**: HTTP to stable node (best performance)
- **LocalOllama**: Built-in local runtime (zero config)

OllaBridge picks the right one automatically.

---

## 📈 Scaling

### Add More Workers

```bash
ollabridge start --workers 4
```

### Use PostgreSQL

```bash
pip install psycopg2-binary
export DATABASE_URL=postgresql://user:pass@localhost/ollabridge
ollabridge start --workers 8
```

### Add More Nodes

```bash
# Just keep adding nodes!
ollabridge-node join --control ... --token ...
```

OllaBridge automatically load-balances across all healthy nodes.

---

## 🌍 Public Access (Optional)

### Quick Demo (Ngrok)

```bash
ollabridge start --share
```

### Production (Cloudflare Tunnel)

```bash
# Terminal 1: Start gateway
ollabridge start

# Terminal 2: Expose it
cloudflared tunnel --url http://localhost:11435
```

Now your gateway has a public `https://` URL!

**Security:** Always use API keys for public gateways.

---

## 🎓 Beginner's Guide

### "I've never used LLMs before"

1. Install: `pip install ollabridge`
2. Start: `ollabridge start`
3. Copy the API key from the output
4. Use this code:

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11435/v1",
    api_key="PASTE_KEY_HERE"
)

response = client.chat.completions.create(
    model="deepseek-r1",
    messages=[{"role": "user", "content": "Explain Python in simple terms"}]
)

print(response.choices[0].message.content)
```

**That's it!** You're running AI models on your computer.

### "I want to add my gaming PC's GPU"

1. On your main computer (gateway):
   ```bash
   ollabridge start
   # Copy the "Node join token" and gateway URL
   ```

2. On your gaming PC:
   ```bash
   pip install ollabridge
   ollabridge-node join --control http://GATEWAY_IP:11435 --token TOKEN_HERE
   ```

3. Done! Your apps can now use your gaming PC's power.

### "I want to use free Colab GPUs"

1. Start your gateway at home:
   ```bash
   ollabridge start --share
   # Note the public URL (https://xxx.ngrok.io)
   ```

2. In Colab notebook:
   ```python
   !pip install ollabridge
   !ollabridge-node join --control https://xxx.ngrok.io --token YOUR_TOKEN
   ```

3. Now your apps use FREE Colab GPUs!

**Pro tip:** When Colab disconnects, just restart and run step 2 again. Zero config changes needed.

---

## 🛠️ CLI Commands Reference

OllaBridge includes powerful CLI commands for diagnostics, testing, and management.

### Diagnostic Commands

#### `ollabridge doctor`
Diagnose your OllaBridge setup (Ollama, gateway, auth, CORS):

```bash
ollabridge doctor
```

**Output:**
```
                     OllaBridge Doctor
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Check               ┃ Result                          ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Ollama /api/tags    │ ✅ OK                           │
│ OllaBridge /health  │ ✅ OK                           │
│ API_KEYS configured │ ✅ yes                          │
│ CORS_ORIGINS        │ http://localhost:5173,...       │
│ Auth usage          │ Use Authorization: Bearer <key> │
└─────────────────────┴─────────────────────────────────┘
```

**Use case:** Troubleshooting connection issues, verifying setup before deployment.

#### `ollabridge models`
List available models (requires API key):

```bash
ollabridge models --api-key sk-ollabridge-xY9kL2mN8pQ4rT6vW1zA
```

**Output:**
```
deepseek-r1
llama3.1
mixtral
```

**Use case:** Verify which models are available across all nodes.

#### `ollabridge test-chat`
Send a test chat completion (requires API key):

```bash
# Simple test
ollabridge test-chat "Hello, how are you?" --api-key sk-ollabridge-...

# Specify model
ollabridge test-chat "Explain quantum computing" \
  --model deepseek-r1 \
  --api-key sk-ollabridge-...
```

**Output:**
```
╭─────────── Assistant ───────────╮
│ Hello! I'm doing well, thank    │
│ you for asking. How can I help  │
│ you today?                       │
╰──────────────────────────────────╯
```

**Use case:** Verify end-to-end connectivity, test API keys, validate model responses.

### Gateway Management

#### `ollabridge start`
Start the gateway (standard mode):

```bash
ollabridge start
```

#### `ollabridge start --lan`
Start with LAN URLs displayed (for classroom/shared networks):

```bash
ollabridge start --lan
```

**Output includes:**
```
🌐 LAN Access
LAN API base:    http://192.168.1.50:11435/v1
LAN Health:      http://192.168.1.50:11435/health

Example (with API key):
curl -H 'Authorization: Bearer <API_KEY>' http://192.168.1.50:11435/v1/models
```

**Use case:** Sharing your gateway with other devices on your network (Quest headsets, phones, other laptops).


#### `ollabridge start --share`
Expose a public URL (via ngrok):

```bash
ollabridge start --share
```

**Use case:** Remote access, connecting nodes from anywhere.

#### `ollabridge enroll-create`
Create enrollment tokens for nodes:

```bash
ollabridge enroll-create --ttl 3600
```

### Quick Tasks

| Task | Command |
|------|---------|
| **List models (API)** | `ollabridge models --api-key <key>` |
| **Test connectivity** | `ollabridge test-chat "test" --api-key <key>` |
| **Check health** | `curl http://localhost:11435/health` |
| **Diagnose setup** | `ollabridge doctor` |
| **See nodes** | `curl -H "X-API-Key: <key>" http://localhost:11435/admin/runtimes` |
| **View logs** | `curl -H "X-API-Key: <key>" http://localhost:11435/admin/recent` |
| **Create token** | `ollabridge enroll-create` |


### For Developers

OllaBridge requires an API key to authenticate requests. If no `.env` file is provided, or the `.env` file does not contain `API_KEYS`, OllaBridge will automatically generate a **temporary, per-run secret API key** (`sk-ollabridge-...`), print it to the screen, and use it only for the current run so you can start developing immediately. **This key is not written to disk by default**, which prevents accidental persistence of credentials and improves security. If you explicitly want OllaBridge to persist the generated API key, you must opt in by starting the gateway with:

```bash
ollabridge start --write-env
```

In this case, OllaBridge will write the generated key to `.env`. For production deployments, it is strongly recommended to set `API_KEYS` using **environment variables or a secure secret manager**, rather than relying on a `.env` file. This design provides safe defaults while avoiding unintentionally storing sensitive information.


---

## ☁️ Optional: OllaBridge Cloud

OllaBridge Local can **optionally** connect to **OllaBridge Cloud** for multi-user, multi-device deployments.

### Cloud Features

- 🔐 **Secure device pairing** with user approval
- 👥 **Multi-user support** with device ownership
- 🌍 **No port forwarding needed** (devices dial out to Cloud)
- 📱 **Multi-device per user** (PC + Quest + phone, etc.)
- 🔄 **Streaming support** for real-time responses

### Pairing Your Device with Cloud

```bash
# 1. Pair this device with OllaBridge Cloud
ollabridge-node cloud-pair --cloud https://your-cloud-url.com

# Shows pairing code - approve via web UI

# 2. Connect to Cloud (uses saved credentials)
ollabridge-node cloud-connect
```

**How it works:**
1. `cloud-pair` gets a pairing code from Cloud
2. You approve the code via Cloud's web UI
3. Device credentials saved to `~/.ollabridge/cloud_device.json`
4. `cloud-connect` connects your device to Cloud relay
5. Cloud routes requests to your device securely

### Local Mode (Default) vs Cloud Mode

| Feature | Local Mode | Cloud Mode |
|---------|------------|------------|
| **Setup** | `ollabridge-node join --control <gateway> --token <token>` | `ollabridge-node cloud-pair --cloud <url>` |
| **Authentication** | Enrollment token | Device pairing + approval |
| **Users** | Single self-hosted | Multi-user cloud accounts |
| **Devices** | Manual node management | Per-user device ownership |
| **Streaming** | Not yet | ✅ Supported |
| **Port forwarding** | Not needed (outbound) | Not needed (outbound) |

**Both modes work together!** Run local gateway + nodes for self-hosting, and optionally pair devices with Cloud for multi-user scenarios.

---

## 🏠 HomePilot Integration

OllaBridge includes a built-in **HomePilot connector** that exposes [HomePilot](https://github.com/ruslanmv/HomePilot) personas as standard OpenAI models. Any app that speaks OpenAI — including [3D Avatar Chatbot](https://github.com/ruslanmv/3D-Avatar-Chatbot) — can chat with persistent AI personas that have personality, long-term memory, and MCP tool access.

### Enable HomePilot

```env
# .env
HOMEPILOT_ENABLED=true
HOMEPILOT_BASE_URL=http://localhost:8000
HOMEPILOT_API_KEY=your-api-key
```

### How It Works

```mermaid
graph LR
    A[Your App] -->|OpenAI SDK| B[OllaBridge]
    B -->|"model=deepseek-r1"| C[Local Ollama]
    B -->|"model=persona:proj-123"| D[HomePilot]
    B -->|"model=personality:therapist"| D

    style B fill:#6366f1,stroke:#4f46e5,stroke-width:3px,color:#fff
    style D fill:#10b981,stroke:#059669,stroke-width:2px,color:#fff
```

**Smart routing**: Models starting with `persona:` or `personality:` are automatically sent to HomePilot. Everything else goes to Ollama or other connected nodes.

### Chat with a Persona

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11435/v1",
    api_key="sk-ollabridge-YOUR-KEY"
)

# Chat with a HomePilot persona — same OpenAI API
response = client.chat.completions.create(
    model="persona:my-therapist",
    messages=[{"role": "user", "content": "I've been feeling stressed."}]
)

print(response.choices[0].message.content)
```

### Discover Available Personas

```bash
# List all models (Ollama + HomePilot personas)
curl -H "Authorization: Bearer sk-ollabridge-..." \
  http://localhost:11435/v1/models
```

Returns both local Ollama models and HomePilot personas in a single list:

```json
{
  "data": [
    {"id": "deepseek-r1", "owned_by": "ollama"},
    {"id": "personality:therapist", "owned_by": "homepilot"},
    {"id": "persona:proj-abc123", "owned_by": "homepilot"}
  ]
}
```

### What Personas Bring

Each HomePilot persona includes capabilities beyond a plain LLM:

| Feature | Description |
|---|---|
| **Personality** | Rich system prompt with psychology, voice style, behavior |
| **Long-Term Memory** | Per-persona persistent memory across sessions |
| **MCP Tools** | Gmail, Calendar, GitHub, Slack, web search, and more |
| **Knowledge Base** | RAG over uploaded documents |
| **Image Generation** | ComfyUI workflows (FLUX, SDXL) |

All transparent to the client — you get a standard OpenAI-format response.

### 3D Avatar Chatbot + HomePilot

The [3D Avatar Chatbot](https://github.com/ruslanmv/3D-Avatar-Chatbot) has a built-in OllaBridge provider. Select it in Settings, fetch models, and your 3D avatar speaks with HomePilot persona personality and memory:

```
3D Avatar Chatbot → OllaBridge Gateway → HomePilot Persona → LLM + Memory + Tools
```

### Full Architecture

```
┌──────────────────────────────────────────┐
│           OllaBridge Gateway             │
│                                          │
│  Registry                                │
│  ├── local_ollama   → Ollama (:11434)    │
│  ├── relay_link     → Remote GPUs        │
│  └── homepilot      → HomePilot (:8000)  │
│                                          │
│  Router                                  │
│  ├── "persona:*"    → homepilot nodes    │
│  ├── "personality:*"→ homepilot nodes    │
│  └── other models   → best available     │
└──────────────────────────────────────────┘
```

For detailed persona system documentation, see [HomePilot's OLLABRIDGE.md](https://github.com/ruslanmv/HomePilot/blob/main/docs/OLLABRIDGE.md).

---

## 🖥️ Dashboard UI

OllaBridge ships with a built-in **Broadcast Tower Dashboard** — a real-time command center that visualizes your entire LLM routing infrastructure at a glance.

<p align="center">
  <img src="assets/dashboard-tower.svg" alt="OllaBridge Dashboard – Broadcast Tower" width="720" />
</p>

### Quick Start

```bash
# Install frontend dependencies
make ui-install

# Development (hot-reload, proxied to OllaBridge API)
make ui-dev

# Production build (served at /ui when gateway is running)
make ui-build
```

Once built, access the dashboard at **`http://localhost:11435/ui`** when OllaBridge is running.

### What You See

| Element | Description |
|---------|-------------|
| **Central Orb** | The LLM Core — pulses when online, shows active model name |
| **Source Chips** | Upstream inputs (Source Inputs) and connected LLM models |
| **Signal Beams** | Animated SVG paths showing data flowing from tower to consumers |
| **Consumer Nodes** | Runtime endpoints — each card shows node status in real time |
| **Status HUD** | Top-right overlay with live health, mode, model count, runtime count |

### Tech Stack

- **React 19** + TypeScript + Vite 8
- **TanStack Query** for real-time API polling (auto-refresh every 10–30s)
- **Framer Motion** for orb pulse, beam particles, and hover animations
- **Tailwind CSS v4** with glassmorphism design tokens (navy/cyan/violet palette)

---

## 🗺️ Roadmap

- [x] ✅ Control Plane + Node architecture
- [x] ✅ Outbound-only node enrollment (no port forwarding)
- [x] ✅ MCP server for AI agent control
- [x] ✅ Multi-node load balancing
- [x] ✅ Diagnostic CLI commands (doctor, models, test-chat)
- [x] ✅ Enhanced CORS handling for browser clients
- [x] ✅ LAN mode for classroom/shared network deployments
- [x] ✅ Cloud compatibility (optional device pairing)
- [x] ✅ Streaming support for chat completions (Cloud mode)
- [x] ✅ HomePilot persona integration (smart persona routing)
- [ ] 🚧 Tag-based routing (send "coding" requests to GPU nodes)
- [ ] 🚧 Model-specific routing rules
- [x] ✅ Web UI dashboard (Broadcast Tower visualization)
- [ ] 🚧 Prometheus metrics
- [ ] 🚧 Support for more runtimes (vLLM, llama.cpp, LM Studio)

---

## 🤝 Contributing

We welcome contributions! Areas we'd love help:

- 🔌 More runtime adapters (vLLM, llama.cpp, etc.)
- 🎨 Web UI for management
- 📊 Better monitoring/metrics
- 🔒 Security enhancements
- 📖 Documentation improvements

**How to contribute:**

1. Fork the repo
2. Create a branch (`git checkout -b feature/amazing`)
3. Make your changes
4. Add tests
5. Submit a PR

---

## 📄 License

Apache License 2.0 - see [LICENSE](LICENSE)

---

## 🙏 Built With

- [FastAPI](https://fastapi.tiangolo.com/) — Modern async web framework
- [Ollama](https://ollama.ai/) — Run LLMs locally
- [WebSockets](https://websockets.readthedocs.io/) — Real-time node connections
- [SQLModel](https://sqlmodel.tiangolo.com/) — Database with Python types

---

## 💬 Support

- 📖 [Documentation](docs/)
- 🐛 [Report Bug](https://github.com/ruslanmv/ollabridge/issues)
- 💡 [Request Feature](https://github.com/ruslanmv/ollabridge/issues)
- 💬 [Discussions](https://github.com/ruslanmv/ollabridge/discussions)

---

## 🌟 Star History

If OllaBridge helped you, give it a star! ⭐

---

## Compatible with HomePilot

<p align="center">
  <a href="https://github.com/ruslanmv/HomePilot">
    <img src="assets/homepilot-logo.svg" alt="HomePilot" width="300" />
  </a>
</p>

OllaBridge is the recommended gateway for [HomePilot](https://github.com/ruslanmv/HomePilot) personas. Route `persona:*` and `personality:*` models to HomePilot while serving local LLMs via Ollama — all through a single OpenAI-compatible endpoint.

<p align="center">
  <img src="assets/ollabridge-architecture.svg" alt="OllaBridge Architecture" width="800" />
</p>

Connect [3D Avatar Chatbot](https://github.com/ruslanmv/3D-Avatar-Chatbot) for an immersive VR persona experience with lip sync, gestures, and voice.

<p align="center">
  <img src="assets/3d-avatar-pipeline.svg" alt="3D Avatar + HomePilot Pipeline" width="800" />
</p>

---

<div align="center">

**Made with ❤️ for the local-first AI community**

**Stop paying cloud tokens. Use your own compute.**

</div>
