Metadata-Version: 2.4
Name: ollambda
Version: 0.1.0
Summary: Add your description here
Requires-Python: >=3.13
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: flask>=3.1.3
Requires-Dist: gunicorn>=26.0.0
Requires-Dist: pre-commit>=4.6.0
Requires-Dist: pytest>=9.1.1
Requires-Dist: pytest-cov>=7.1.0
Requires-Dist: pyyaml>=6.0.3
Dynamic: license-file

# ollambda

Simple Flask based API server to proxy ollama calls to a more performant backend (such as llama.cpp)

## How to Use

### Installation

To install ollambda, clone this repository and use `pip install .`

#### venv

Venv usage is encouraged, the simplest way to manage this is to use [uv](https://docs.astral.sh/uv/):

```
cd ollambda
uv venv .venv
source .venv/bin/activate
uv pip install .
```

### Configuration

Currently ollambda proxies requests to an upstream llama.cpp server. The address is configured via environment variables or a YAML config file at `~/.ollambda/config.yaml`:

| Setting    | Env var               | Config key            | Default     |
| ---------- | --------------------- | --------------------- | ----------- |
| Proxy IP   | `OLLAMBDA_PROXY_IP`   | `ollambda_proxy_ip`   | `127.0.0.1` |
| Proxy port | `OLLAMBDA_PROXY_PORT` | `ollambda_proxy_port` | `8080`      |

### Running the server

Start with Gunicorn (included as a dependency):

```bash
gunicorn -c gunicorn.conf.py ollambda:app
```

This spawns one worker per CPU core and binds to `0.0.0.0:5000` by default. Override settings on the command line:

```bash
gunicorn ollambda:app -w 4 -b 0.0.0.0:5000
```

## ollama API coverage

| Method | Endpoint             | Description                                                 | Implemented? |
| ------ | -------------------- | ----------------------------------------------------------- | ------------ |
| POST   | `/api/generate`      | Generate a completion (streaming or non-streaming)          | N            |
| POST   | `/api/chat`          | Generate a chat completion (with tool calling support)      | Y            |
| POST   | `/api/create`        | Create a model from another model, GGUF, or safetensors     | N (no plans) |
| GET    | `/api/tags`          | List locally available models                               | Y            |
| POST   | `/api/show`          | Show model information (details, modelfile, template, etc.) | Y            |
| POST   | `/api/copy`          | Copy a model to a new name                                  | N            |
| DELETE | `/api/delete`        | Delete a model                                              | Y            |
| POST   | `/api/pull`          | Download a model from the Ollama library                    | N            |
| POST   | `/api/push`          | Upload a model to a model library                           | N (no plans) |
| POST   | `/api/embed`         | Generate embeddings from a model                            | Y            |
| GET    | `/api/ps`            | List models currently loaded in memory                      | Y            |
| GET    | `/api/version`       | Retrieve the Ollama server version                          | Y            |
| HEAD   | `/api/blobs/:digest` | Check if a file blob exists on the server                   | N (no plans) |
| POST   | `/api/blobs/:digest` | Push a file blob to the server                              | N (no plans) |

> [!IMPORTANT]
> Current "implementation" may not be fully complete and API aligned.

> [!NOTE]
> Source: https://github.com/ollama/ollama/blob/main/docs/api.md
