Metadata-Version: 2.5
Name: onnxruntime-ggml
Version: 0.2.0
Summary: An onnxruntime execution provider that runs ONNX models on ggml (Metal, Vulkan, CPU)
Project-URL: Homepage, https://github.com/thewh1teagle/onnxruntime-ggml
Author: thewh1teagle
License-Expression: MIT
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: onnxruntime<1.31,>=1.29
Description-Content-Type: text/markdown

# onnxruntime-ggml

An onnxruntime execution provider that runs ONNX models on ggml's Metal, Vulkan and CPU kernels.

```console
uv pip install onnxruntime-ggml
```

Requires Python 3.11 or newer.

The wheel includes the native provider and installs a compatible onnxruntime.
Supports macOS 14+ (Apple silicon), Linux x86_64 (glibc 2.35+) and aarch64
(glibc 2.39+), and Windows x64. No compiler or separate ggml installation is needed.

```python
import onnxruntime_ggml as ggml

session = ggml.InferenceSession("model.onnx")            # ggml first, CPU provider as fallback
session = ggml.InferenceSession("model.onnx", {"device": "cpu", "threads": 4})
```

Or keep your own session options:

```python
import onnxruntime as ort, onnxruntime_ggml as ggml

so = ggml.session_options({"device": "gpu"})
session = ort.InferenceSession("model.onnx", so)
```

Set `ORT_GGML_LOG=debug` to see what the provider does with a graph. Full docs: https://github.com/thewh1teagle/onnxruntime-ggml
