Metadata-Version: 2.4
Name: amd-pace
Version: 1.3.0
Summary: PACE (Platform Aware Compute Engine): high-performance LLM inference on AMD CPUs.
Author: AMD
Maintainer: AMD
License-Expression: MIT
Project-URL: Homepage, https://github.com/amd/amd-pace
Project-URL: Documentation, https://github.com/amd/amd-pace#readme
Project-URL: Repository, https://github.com/amd/amd-pace
Project-URL: Issues, https://github.com/amd/amd-pace/issues
Keywords: amd,pace,llm,inference,pytorch,cpu
Requires-Python: <3.15,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: numpy>=2.2.6; python_version < "3.12"
Requires-Dist: numpy>=2.5.0; python_version >= "3.12"
Requires-Dist: huggingface-hub[hf_xet]==1.21.0
Requires-Dist: transformers==5.12.1
Requires-Dist: jinja2>=3.1.6
Requires-Dist: safetensors>=0.8.0
Requires-Dist: filelock>=3.32.2
Requires-Dist: tqdm>=4.68.3
Requires-Dist: protobuf>=6.31.1
Requires-Dist: psutil>=7.2.2
Requires-Dist: xxhash>=3.4.0
Requires-Dist: fastapi==0.138.2
Requires-Dist: uvicorn==0.49.0
Requires-Dist: uvloop==0.22.1
Requires-Dist: pydantic>=2.13.4
Requires-Dist: aiohttp>=3.14.1
Requires-Dist: requests>=2.34.2
Requires-Dist: prometheus_client==0.25.0
Requires-Dist: backports.strenum<2.0,>=1.3.1; python_version < "3.11"
Dynamic: license-file

# AMD PACE

High-performance LLM inference and serving on AMD EPYC CPUs. PACE combines a PyTorch C++ extension, CPU-native KV-cache backends, fused operators, and a production-ready serving stack. Check out the [GitHub repository](https://github.com/amd/amd-pace) for more information.


## Highlights

- **Inference server** - `pace-server` provides a router/engine serving stack with a `continuous_batch` scheduler that combines chunked prefills and decodes, multi-instance NUMA-aware execution, and built-in Prometheus and scheduler metrics.
- **FusedMoE** - BF16 routed and shared Mixture-of-Experts execution with an optimized TPP expert backend.
- **Automatic prefix caching** - `pace-server` reuses complete prompt-prefix blocks across requests with `SLAB_POOL` and `PAGED`.
- **Broad model support** - Llama 3 and text-only Llama 4, Qwen2/2.5/3 (including Qwen3 MoE), Phi3/4, Gemma 3 and text-only Gemma 4, GPT-J, OPT, and GPT-OSS under one operator and backend framework.
- **Speculative decoding (PARD)** - built-in parallel-draft speculation for greedy decoding.
- **PACE Agent** - the repository also includes a LangGraph-based runtime for profiling, deterministic replay, and optimization of non-model agent work. See the [PACE Agent README](https://github.com/amd/amd-pace/tree/main/packages/pace_agent).

## Requirements

- Linux x86_64 with AVX512F + AVX512_BF16 (AMD Zen4 or newer)
- Python 3.10 – 3.14

## Install

```bash
# 1. CPU PyTorch (the +cpu build is not on PyPI; needs PyTorch's index).
pip install --extra-index-url https://download.pytorch.org/whl/cpu torch==2.13.0+cpu

# 2. amd-pace
pip install amd-pace
```

## Quick example

Inference server (router + engine, OpenAI-compatible endpoint):

```bash
pace-server --server_model meta-llama/Llama-3.1-8B --kv_cache_type SLAB_POOL --serve_type continuous_batch
```

For offline programmatic generation, `pace.llm.LLMModel` resolves the tokenizer
and applies default operator backends unless they are explicitly overridden.
See the runnable scripts at
[`examples/`](https://github.com/amd/amd-pace/tree/main/examples) --
[`pace_llm_basic.py`](https://github.com/amd/amd-pace/blob/main/examples/pace_llm_basic.py)
is the smallest starting point.

## Support

We welcome feedback, suggestions, and bug reports. Should you have any of these, please kindly file an issue on the PACE GitHub page [here](https://github.com/amd/amd-pace/issues).

## License

AMD PACE is licensed under the MIT License. See the [LICENSE](https://github.com/amd/amd-pace/blob/main/LICENSE) file for details. Third-party notices are in [NOTICE](https://github.com/amd/amd-pace/blob/main/NOTICE).
