Metadata-Version: 2.4
Name: nxrt-ep-cuda
Version: 0.1.0.dev5
Summary: ONNX Runtime plugin execution provider (CUDA 13, EXPERIMENTAL) — bundled cdylib for register_execution_provider_library
Author: Justin Chu
License: MIT
Project-URL: repository, https://github.com/justinchuby/onnx-genai
Keywords: onnx,onnxruntime,execution-provider,cuda,plugin
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Rust
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Microsoft :: Windows
Classifier: Environment :: GPU :: NVIDIA CUDA :: 13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: nvidia-cuda-runtime<14,>=13
Requires-Dist: nvidia-cublas<14,>=13
Requires-Dist: nvidia-cuda-nvrtc<14,>=13
Requires-Dist: nvidia-cuda-cupti<14,>=13
Provides-Extra: onnxruntime
Requires-Dist: onnxruntime>=1.22; extra == "onnxruntime"

# nxrt-ep-cuda (EXPERIMENTAL / PRE-RELEASE)

> ⚠️ **Experimental.** The CUDA execution provider bundled here has **not** been
> validated on physical CUDA hardware yet (tracked by issue #768). Do not rely
> on it for production. APIs and packaging may change without notice.

A pip-installable **ONNX Runtime plugin execution provider (CUDA 13)**. The
wheel bundles the compiled `onnx-runtime-ep-cuda-plugin` shared library
(`libonnx_runtime_ep_cuda_plugin.{so,dll}`) built with the `cuda` cargo feature
and exposes the absolute path to it so ONNX Runtime can load it via
`RegisterExecutionProviderLibrary(registration_name, library_path)`.

The bundled library exports the ORT plugin-EP C ABI
(`CreateEpFactories` / `ReleaseEpFactory`). It is **not** a Python extension.

## Requirements

- **CUDA 13** runtime. The wheel declares the NVIDIA runtime libraries as
  dependencies (`nvidia-cuda-runtime>=13`, `nvidia-cublas>=13`,
  `nvidia-cuda-nvrtc>=13`, `nvidia-cuda-cupti>=13`), so they are installed
  automatically. The NVIDIA **driver** (`libcuda.so.1`) remains a host
  prerequisite.
- Linux (x86_64) and Windows (AMD64) only. There is no macOS build.

## Install

```bash
pip install nxrt-ep-cuda   # pre-release; may require --pre
```

## Usage

```python
import nxrt_ep_cuda

path = nxrt_ep_cuda.get_library_path()   # absolute path to the bundled cdylib

import onnxruntime as ort
so = ort.SessionOptions()
nxrt_ep_cuda.register(so)                # thin wrapper over register_execution_provider_library
```

`register()` prefers `SessionOptions.register_execution_provider_library` when a
`SessionOptions` is passed, and falls back to the module-level
`onnxruntime.register_execution_provider_library`. If `onnxruntime` is not
installed it raises a clear `ImportError`.

## License

MIT. See the [onnx-genai](https://github.com/justinchuby/onnx-genai) repository.
