Metadata-Version: 2.4
Name: lllm2
Version: 0.5.0
Summary: Local NVIDIA LLM experimentation workbench
Author-email: Giles Knap <giles.knap@diamond.ac.uk>
License-Expression: Apache-2.0
Project-URL: GitHub, https://github.com/gilesknap/lllm2
Project-URL: Documentation, https://gilesknap.github.io/lllm2/
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: typer<1,>=0.16
Dynamic: license-file

[![CI](https://github.com/gilesknap/lllm2/actions/workflows/ci.yml/badge.svg)](https://github.com/gilesknap/lllm2/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/lllm2.svg)](https://pypi.org/project/lllm2/)

# lllm2

A local NVIDIA LLM workbench: download GGUF models, run llama.cpp, compare
settings and save measured results from a browser panel.

## Install and run

On Linux with Python 3.11+ and a working NVIDIA driver,
[install uv](https://docs.astral.sh/uv/getting-started/installation/), then:

```bash
uv tool install --upgrade lllm2
lllm2 engines install cuda
lllm2
```

The engine is downloaded from the matching lllm2 release, with CUDA runtime
libraries included. No host compiler or CUDA toolkit is needed. Existing builds
can also be discovered. Open
<http://127.0.0.1:8082>, download a model and click **Start**. The model API
listens at `http://127.0.0.1:1920/v1`.

<!-- README only content. Anything below this line won't be included in index.md -->

[Documentation](https://gilesknap.github.io/lllm2/) covers
[getting started](docs/tutorials/installation.md), [common tasks](docs/how-to.md),
[reference](docs/reference.md), [measurements](docs/explanations/defaults-and-measurements.md)
and [development and releases](docs/how-to/development.md).

Project structure and tooling follow the
[DLS Python Copier template](https://github.com/DiamondLightSource/python-copier-template).
