Metadata-Version: 2.5
Name: pocket-o-llama
Version: 1.0.0
Summary: Lightweight, hardware-independent local GGUF parser and Ollama substitute server with Hugging Face Hub integration.
Project-URL: Homepage, https://huggingface.co/spaces
Project-URL: Repository, https://github.com/Hrutu34/Pocket-O-Llama
Author-email: Hrutu Surve <survehrutu@gmail.com>
License: AGPL-3.0-or-later
License-File: LICENSE
Classifier: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Requires-Dist: fastapi>=0.110.0
Requires-Dist: gguf>=0.19.0
Requires-Dist: huggingface-hub>=0.20.0
Requires-Dist: numpy>=1.22.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: uvicorn>=0.28.0
Description-Content-Type: text/markdown

# Pocket-O-Llama

<p align="center">
  <img src="https://raw.githubusercontent.com/gist/Hrutu34/6e153934d5546da220337225f2b93ff3/raw/72fb73757933709ec2eb97df2b7ac3063da87928/Logo.svg" alt="Pocket-O-Llama Logo" width="350" height="220">
</p>

<p align="center">
  <strong>Your local, high-precision GGUF model server.</strong><br>
  <em>Designed to run smoothly on low-spec hardware without throttling your system.<br>Empowering developers to run AI locally, efficiently, and privately.</em>
  
</p>

<p align="center">
  <a href="https://pypi.org/project/pocket-o-llama/"><img src="https://img.shields.io/pypi/v/pocket-o-llama.svg" alt="PyPI version"></a>
  <a href="https://www.python.org/"><img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="Python Version"></a>
  <a href="https://github.com/Hrutu34/Pocket-O-Llama/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License"></a>
</p>

---

## Why Pocket-O-Llama?

Running local LLMs shouldn't require a supercomputer or a massive, bloated installation. Pocket-O-Llama is a lightweight, hardware-agnostic alternative to heavier engines. It is designed from the ground up to maximize the performance of highly quantized models (like those from **Unsloth AI**) on everyday laptops and desktops.

* **Native Hugging Face Hub Integration:** Pass any GGUF repository ID (e.g., `unsloth/Llama-3.2-1B-Instruct-GGUF`), and the server will auto-resolve, download, and cache the optimal quantization for you.
* **True Zero-Copy Streaming:** Real-time token streaming using `llama_cpp` for instant Time-To-First-Token.
* **Interactive Web Dashboard:** Comes with a beautiful, built-in dark-mode UI with Markdown rendering, model switching, and real-time conversation history.
* **OpenAI Drop-In Replacement:** Fully schema-compatible with `/v1/chat/completions`, meaning it instantly works with your existing LangChain, LlamaIndex, or AutoGen scripts.
* **Zero Port Conflicts:** Runs independently on custom ports (default `11435`) so it never clashes with other local services.

---

## Installation

Install Pocket-O-Llama directly via `pip`. It is recommended to use a virtual environment.

```bash
pip install pocket-o-llama
```

or

Clone the repository and install it locally via pip:

```bash
cd Pocket-O-Llama
python -m pip install .

```

---

## Usage

Launch your local GGUF model server directly from the command line by passing the path to your model file:

```powershell
# Example 1: Auto-download and run an Unsloth model from Hugging Face
pocket-chat --model unsloth/Llama-3.2-1B-Instruct-GGUF

# Example 2: Run a local .gguf file with custom thread allocation
pocket-chat --model "/path/to/your/model.gguf" --port 11435 --threads 4

```
Once running, open your browser and navigate to http://localhost:11435 to access the interactive chat dashboard!

### Command-Line Arguments

| Argument | Short | Description | Default |
| --- | --- |--- | --- |
| `--model` | `-m` | Hugging Face Repo ID or absolute path to a local .gguf file | `unsloth/Llama-3.2-1B-Instruct-GGUF` |
| `--port` | `-p` | Port to run the FastAPI server on | `11435` |
| `--host` | | Host IP address (Use 127.0.0.1 to expose to LAN) | `127.0.0.1`
| `--threads` | `-t` | Maximum CPU threads to allocate | `4` |

---

## API Usage/Testing

Pocket-O-Llama acts as a drop-in replacement for OpenAI/Gemini/Anthropic API. You can hit the /v1/chat/completions endpoint exactly as you normally would.

Using Python requests:

```python
import requests

response = requests.post(
    "http://localhost:11435/v1/chat/completions",
    json={
        "messages": [
            {"role": "system", "content": "You are a helpful, brilliant coding assistant."},
            {"role": "user", "content": "Write a Python function to calculate the Fibonacci sequence."}
        ]
    }
)

print(response.json()["choices"][0]["message"]["content"])

```

Using cURL (Streaming):

```bash
curl -X POST http://localhost:11435/api/chat \
-H "Content-Type: application/json" \
-d '{
  "model": "unsloth/Llama-3.2-1B-Instruct-GGUF",
  "messages": [{"role": "user", "content": "Explain quantum computing in one sentence."}],
  "stream": true
}'
```

---

## Roadmap (v0.2.0 & Beyond)
We are actively developing features to push the boundaries of lightweight local AI:

1. The "OS-Sandbox" HITL Agent: Zero-config native tool calling with interactive macOS-style GUI permission prompts (Human-in-the-Loop) before executing code or reading local directories.

2. Dynamic LoRA Swapping: Instant, memory-efficient "Skill Cartridge" swapping for base models.

3. LAN Compute Swarming: Peer discovery to split GGUF inference layers across multiple low-spec devices on the same Wi-Fi network.

---

## License & Author
Created by <a href="https://www.linkedin.com/in/hrutu-surve-767228253/"><span>Hrutu Surve</span></a>

>This project is open-source and fiercely protected against unauthorized closed-source corporate usage under the GNU AGPLv3 License. See the LICENSE file for full details.