Metadata-Version: 2.4
Name: llm-speed
Version: 0.0.7
Summary: Benchmark any LLM on any hardware. CLI for the llm-speed.com flywheel.
Author-email: meadow-kun <2424351+meadow-kun@users.noreply.github.com>
License: Apache-2.0
Project-URL: Homepage, https://llm-speed.com
Project-URL: Source, https://github.com/meadow-kun/llm-speed
Project-URL: Documentation, https://llm-speed.com/methodology
Project-URL: Issues, https://github.com/meadow-kun/llm-speed/issues
Keywords: llm,benchmark,inference,tokens-per-second,ollama,llama.cpp,vllm,mlx
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Environment :: GPU
Classifier: Environment :: GPU :: NVIDIA CUDA
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Benchmark
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx<0.30,>=0.27
Requires-Dist: psutil>=5.9
Requires-Dist: rich>=13.7
Requires-Dist: cryptography>=44.0.1
Requires-Dist: joserfc>=1.0
Provides-Extra: seed
Requires-Dist: praw<8,>=7.7; extra == "seed"
Provides-Extra: mlx
Requires-Dist: mlx-lm>=0.18; extra == "mlx"
Provides-Extra: vllm
Requires-Dist: vllm>=0.6; extra == "vllm"
Provides-Extra: exllamav2
Requires-Dist: exllamav2>=0.2; extra == "exllamav2"
Provides-Extra: mcp
Requires-Dist: mcp>=1.10; extra == "mcp"
Requires-Dist: cachetools>=5; extra == "mcp"
Provides-Extra: test
Requires-Dist: pytest>=8; extra == "test"
Requires-Dist: pytest-cov>=5; extra == "test"
Requires-Dist: jsonschema==4.26.0; extra == "test"
Requires-Dist: rfc3339-validator==0.1.4; extra == "test"
Dynamic: license-file

# llm-speed

Benchmark local LLM inference and inspect the settings behind your results.
[llm-speed.com](https://llm-speed.com/) publishes submitted measurements across hardware and backends. This repository contains the benchmark CLI.

## Install

With Python 3.10 or newer:

```sh
pipx install llm-speed
```

Or use `uv tool install llm-speed`. On a machine without Python, the [installer](https://llm-speed.com/install.sh) can provision it with your consent:

```sh
curl -fsSL https://llm-speed.com/install.sh | sh
```

Check dependencies and backend setup with `llm-speed doctor`. On an interactive terminal it offers setup steps; otherwise it prints guidance.

## Run

```sh
llm-speed --version
llm-speed bench --quick --no-upload
```

The second command saves results locally without uploading. Choose a backend and model with `--backend` and `--model` when needed. Run `llm-speed bench --help` for options. Uploading is optional; review the [privacy documentation](docs/PRIVACY.md) before sharing results.

Supported backend integrations include Ollama, llama.cpp, MLX, vLLM, and ExLlamaV2. Availability depends on your hardware and installed backend. Measurements with different models, quantization, and workloads are not interchangeable rankings. See the [methodology](docs/METHODOLOGY.md).

## Links and license

- [Results and datasets](https://llm-speed.com/)
- [Report an issue](https://github.com/meadow-kun/llm-speed/issues)
- [Security policy](SECURITY.md)

CLI code is licensed under [Apache-2.0](LICENSE). Shared benchmark data is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
