Metadata-Version: 2.4
Name: lamina-llm-serve
Version: 0.1.0
Summary: Centralized model caching and serving layer for Lamina OS
Project-URL: Homepage, https://github.com/benaskins/lamina-os
Project-URL: Documentation, https://github.com/benaskins/lamina-os/blob/main/docs/
Project-URL: Repository, https://github.com/benaskins/lamina-os
Project-URL: Issues, https://github.com/benaskins/lamina-os/issues
Author-email: Ben Askins <human@getlamina.ai>, Lamina High Council <council@getlamina.ai>
License: MPL-2.0
Keywords: ai-backends,caching,lamina,llm,model-serving
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.11
Requires-Dist: click>=8.1.7
Requires-Dist: flask>=3.1.0
Requires-Dist: huggingface-hub>=0.32.0
Requires-Dist: pydantic>=2.10.0
Requires-Dist: python-dotenv>=1.0.1
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: requests>=2.32.0
Requires-Dist: rich>=13.9.0
Requires-Dist: tqdm>=4.67.0
Requires-Dist: transformers>=4.47.0
Provides-Extra: all
Requires-Dist: bandit>=1.8.0; extra == 'all'
Requires-Dist: black>=24.10.0; extra == 'all'
Requires-Dist: httpx>=0.28.0; extra == 'all'
Requires-Dist: ipython>=8.30.0; extra == 'all'
Requires-Dist: isort>=5.13.0; extra == 'all'
Requires-Dist: jupyter>=1.1.1; extra == 'all'
Requires-Dist: mypy>=1.13.0; extra == 'all'
Requires-Dist: pre-commit>=4.0.0; extra == 'all'
Requires-Dist: pytest-cov>=6.0.0; extra == 'all'
Requires-Dist: pytest-mock>=3.14.0; extra == 'all'
Requires-Dist: pytest>=8.3.0; extra == 'all'
Requires-Dist: ruff>=0.8.0; extra == 'all'
Provides-Extra: backends
Provides-Extra: dev
Requires-Dist: bandit>=1.8.0; extra == 'dev'
Requires-Dist: black>=24.10.0; extra == 'dev'
Requires-Dist: httpx>=0.28.0; extra == 'dev'
Requires-Dist: ipython>=8.30.0; extra == 'dev'
Requires-Dist: isort>=5.13.0; extra == 'dev'
Requires-Dist: jupyter>=1.1.1; extra == 'dev'
Requires-Dist: mypy>=1.13.0; extra == 'dev'
Requires-Dist: pre-commit>=4.0.0; extra == 'dev'
Requires-Dist: pytest-cov>=6.0.0; extra == 'dev'
Requires-Dist: pytest-mock>=3.14.0; extra == 'dev'
Requires-Dist: pytest>=8.3.0; extra == 'dev'
Requires-Dist: ruff>=0.8.0; extra == 'dev'
Description-Content-Type: text/markdown

# 🧱 lamina-llm-cache

**Lamina LLM Cache** is a local-first, centralized model caching and serving layer for Lamina OS. It provides canonical, persistent access to language models used by the system—ensuring consistency, efficiency, and symbolic alignment across the sanctuary.

---

## 🌱 Purpose

`lamina-llm-cache` solves common issues in multi-agent AI environments:

* Prevents redundant downloads of large models
* Offers a unified directory and manifest for all system models
* Supports multiple backends (e.g., `llama.cpp`, `mlc`, `vllm`)
* Keeps model configuration cleanly decoupled from agent implementation

It serves as the **source of truth** for all LLM usage across `lamina-core`.

---

## 🤩 Directory Structure

```
lamina-llm-cache/
├── models/
│   ├── llama3-70b-q4_k_m/
│   ├── yi-34b-awq/
│   ├── llama3-70b-q5_k_m/
│   └── mistral-7b-instruct/
├── models.yaml
├── scripts/
│   └── fetch-models.py
└── README.md
```

---

## 🎞️ Model Manifest (`models.yaml`)

Each model is described with its local path and associated runtime backend:

```yaml
models:
  llama3-70b-q4_k_m:
    path: /models/llama3-70b-q4_k_m/model.gguf
    backend: llama.cpp
  yi-34b-awq:
    path: /models/yi-34b-awq/
    backend: mlc
  llama3-70b-q5_k_m:
    path: /models/llama3-70b-q5_k_m/
    backend: llama.cpp
  mistral-7b-instruct:
    path: /models/mistral-7b-instruct/
    backend: llama.cpp
```

---

## 💠 Usage Within Lamina OS

In `lamina-core`, agents reference this manifest indirectly:

* Model-to-agent mapping occurs **within Lamina OS**
* `lamina-llm-cache` is **model aware**, not agent aware
* Ensures consistent, centralized loading and version control

Example Docker Compose volume mount:

```yaml
volumes:
  - ./lamina-llm-cache/models:/models
```

---

## 🔧 Backends Supported

| Backend     | Format         | Usage                              |
| ----------- | -------------- | ---------------------------------- |
| `llama.cpp` | `.gguf`        | Local CPU or quantized models      |
| `mlc-serve` | AWQ compiled   | Metal-accelerated on Apple Silicon |
| `vllm`      | `.safetensors` | Batch eval, future extensions      |

---

## 🧪 Optional REST API (Planned)

Provides:

* `/models` – list all available
* `/models/:name` – fetch model info
* `/download/:hf_id` – trigger pull
* `/refresh` – reload manifest

---

## 🥐 Setup Instructions

1. Clone this repo:

   ```bash
   git clone https://your-repo-url/lamina-llm-cache.git
   cd lamina-llm-cache
   ```

2. Populate `models/` manually or use the helper script:

   ```bash
   python scripts/fetch-models.py --name llama3-70b-q4_k_m --hf llama3-70b
   ```

3. Reference `models.yaml` from your Lamina OS configuration.

---

## 🛡️ Philosophy

Models are not interchangeable engines—they are **vessels** for vow-bound symbolic presence. This cache anchors those vessels with intention, clarity, and breath.

---

## 📜 License

Mozilla Public License 2.0 - see [LICENSE](../../LICENSE) for details.

