Metadata-Version: 2.4
Name: lora-ollama-bridge
Version: 0.1.3
Summary: LoRA adapter bridge between Unsloth and Ollama for dynamic adapter loading.
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mlx
Requires-Dist: torch
Requires-Dist: safetensors
Requires-Dist: requests
Requires-Dist: jinja2
Dynamic: license-file

# Bridge between Unsloth and Ollama

A utility for registering LoRA adapters into Ollama without weight fusion or GGUF quantization. It's tested on MLX models so far, but it should support other variants as well.

Standard conversion paths from Unsloth tuned model (`mlx_vlm.fuse` $\rightarrow$ `convert_hf_to_gguf`) are CPU-bound and result in multi-gigabyte files for a 150MB adapter change. This tool bridge MLX/Unsloth adapters to the HF PEFT schema, allowing for `ADAPTER` registration in Ollama. It removes the need for de-quantization and re-quantization cycles which prevents additional quantization degradation. 

---

## Installation

```bash
pip install lora-ollama-bridge
```

Or install from source:

```bash
git clone https://github.com/filtercodes/LoRA-Ollama-Bridge.git
cd LoRA-Ollama-Bridge
pip install -e .
```

---

NOTE - bridge support is currently pending upstream review in [Ollama PR #17377](https://github.com/ollama/ollama/pull/17377).
To use this tool right now, run the patched Ollama built from source:

```bash
git clone https://github.com/filtercodes/ollama.git
cd ollama
go build .
./ollama serve
```

---

## Quickstart

### 1. Register Adapter in Ollama

Point the tool to the adapter directory (containing `adapters.safetensors` and `adapter_config.json`). It converts the tensors to HuggingFace PEFT schema and registers the model in Ollama:

```bash
lora-ollama-bridge -i ./mlx_adapters --name new-fine-tuned-model
```

### 2. Convert Adapter Only (Skip Ollama)

To convert the adapter format without calling the Ollama API:

```bash
lora-ollama-bridge -i ./mlx_adapters -o ./converted_adapter --skip-ollama
```

### 3. Custom Checkpoint & System Prompt

Specify a checkpoint file, system prompt, or context length:

```bash
lora-ollama-bridge \
  -i ./mlx_adapters \
  -w 0000400_adapters.safetensors \
  --name gemma4-fine-tune \
  -s "You are a helpful assistant." \
  --num-ctx 65536
```

---

## CLI Reference

| Flag | Short | Default | Description |
| :--- | :--- | :--- | :--- |
| `--input-dir` | `-i` | `./` | Path to directory containing MLX or HF PEFT adapter files. |
| `--output-dir` | `-o` | `./converted_adapter` | Output directory for converted PEFT safetensors. |
| `--ollama-base-model` | `-m` | `gemma4:12b-mlx` | Base model registered in Ollama. |
| `--ollama-target-name`, `--name` | `-n` | `gemma4-adapted` | Name for the new model in Ollama. |
| `--adapter-weights` | `-w` | `None` | Specific adapter weights file (e.g. `0000400_adapters.safetensors`). |
| `--system-prompt`, `--system` | `-s` | `None` | System prompt to include in the Modelfile. |
| `--ollama-url` | `-u` | `http://localhost:11434` | Ollama server URL. |
| `--num-ctx` | | `65536` | Context window size (`num_ctx`). |
| `--skip-ollama` | | `False` | Convert adapter format without calling Ollama API. |
| `--force` | | `False` | Bypass safety checks. |

---

## Requirements

- Python 3.10+
- Dependencies: `mlx`, `torch`, `safetensors`, `requests`, `jinja2`

---

## Testing

```bash
python3 -m unittest discover -s tests
```
