Metadata-Version: 2.5
Name: baton-cluster
Version: 0.1.0
Summary: LAN-distributed LLM inference. One model, many laptops, one relay.
Project-URL: Homepage, https://github.com/hemalbadola/baton
Project-URL: Source, https://github.com/hemalbadola/baton
Requires-Python: >=3.11
Requires-Dist: fastapi
Requires-Dist: httpx
Requires-Dist: jinja2
Requires-Dist: numpy
Requires-Dist: psutil
Requires-Dist: safetensors
Requires-Dist: torch
Requires-Dist: transformers
Requires-Dist: typer
Requires-Dist: uvicorn
Requires-Dist: websockets
Requires-Dist: zeroconf
Provides-Extra: dev
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Description-Content-Type: text/markdown

# Baton

Run one large language model across several laptops on the same Wi-Fi. Each laptop holds
a few layers of the model and passes activations to the next laptop, like a relay baton.

## Install

On every laptop, one line. A page opens in the browser.

```
# macOS or Linux
curl -LsSf https://raw.githubusercontent.com/hemalbadola/baton/main/install.sh | sh -s -- app

# Windows (PowerShell)
& ([scriptblock]::Create((irm https://raw.githubusercontent.com/hemalbadola/baton/main/install.ps1))) app

# any system with Python 3.11 or newer
pip install baton-cluster
baton app

# macOS or Linux with Homebrew
brew install hemalbadola/baton/baton
baton app
```

## Use

1. Run `baton app` on every laptop. Each laptop shows the others that are nearby.
2. On one laptop, pick a model and click **Host this model**. Invite the nearby laptops.
3. On each other laptop, click **Accept**.
4. On the host, click **Start**. Chat when the cluster is ready.

The API is OpenAI compatible: `http://<host>:7700/v1`. The first start downloads the model.
Only bf16 weights load today (`--quant none`).

## Commands

```
baton app                                 # the page above
baton serve --model Qwen/Qwen2.5-0.5B-Instruct --quant none --min-workers 2
baton worker                              # joins a head found over mDNS
```

## Develop

```
pip install -e ".[dev]"
pytest
```

Design: see `TICKETS.md`. Tests need the sandbox off for sockets and mDNS.
