Metadata-Version: 2.4
Name: orbitkv
Version: 0.1.0b1
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Requires-Dist: sglang==0.5.23 ; python_full_version >= '3.11' and python_full_version < '3.14' and sys_platform == 'linux' and extra == 'sglang'
Requires-Dist: vllm==0.33.0 ; python_full_version >= '3.11' and python_full_version < '3.15' and sys_platform == 'linux' and extra == 'vllm'
Provides-Extra: sglang
Provides-Extra: vllm
License-File: LICENSE
License-File: NOTICE
Summary: Framework adapters and Python bindings for the OrbitKV state cache
Keywords: storage,kv-cache,rust,vllm,sglang,inference
Author: OrbitKV Contributors
License-Expression: Apache-2.0
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Documentation, https://orbitinfra.github.io/OrbitKV/docs/
Project-URL: Homepage, https://orbitinfra.github.io/OrbitKV/
Project-URL: Issues, https://github.com/OrbitInfra/OrbitKV/issues
Project-URL: Repository, https://github.com/OrbitInfra/OrbitKV

# OrbitKV Python package

OrbitKV connects official vLLM and SGLang to an independent Rust Cache Manager.
Reuse prefixes from DRAM and optional SSD after GPU eviction or engine restart.
Engines own GPU memory and scheduling; Rust owns cache state, batches, budgets
and transfer completion.

[Quickstart](../docs/single-node.md) · [Compatibility](../docs/compatibility.md) ·
[Adapter contracts](../docs/adapters.md) · [Test gates](tests/README.md)

## Installation

| CUDA runtime | Distribution | Import |
| --- | --- | --- |
| CUDA 13 | `orbitkv` | `orbitkv` |

CUDA 13 is the only supported runtime, with NVIDIA R580 or newer and a driver
compatible with the selected toolkit's runtime-compiled kernels. Version 0.1.0b1
is being prepared for release; build a complete wheel from the repository root:

```bash
./scripts/build-wheel.sh --release
python -m pip install /absolute/path/to/the-built-wheel.whl
```

The wheel includes the
extension and Manager with OrbiTE linked in; the host supplies compatible Python,
CUDA and RDMA libraries. The Manager also requires compatible PyTorch, which the
base package does not install. See [release requirements](../docs/releases.md).

## Engine integration

Use separate official environments for **vLLM 0.33.0** and **SGLang 0.5.23**,
with the same OrbitKV build in the Manager environment. Ordinary vLLM deployments keep
`VLLM_USE_V2_MODEL_RUNNER=0`; single-group attention and
[Full + SWA](../docs/hybrid-recovery.md#vllm-hybrid-handoff) are admitted. The
[V2 gate](../docs/fault-qualification.md#v2-opt-in-gates) is explicitly opt-in,
uniform Full attention only, and unqualified. SGLang uses the
UnifiedRadixCache external linker and its public load-completion callback.
Engine extras target Linux: vLLM on Python 3.11–3.14 and SGLang on 3.11–3.13. Earlier 0.5.22 Qwen3-8B normal-path
acceptance and 0.5.21 measurements are historical; neither qualifies these
pins or the OrbiTE data path. See [compatibility](../docs/compatibility.md). Pre-capture and pending-admission hooks remain until released
replacements are available.
Follow the [engine quickstart](../docs/single-node.md) for complete commands and
an external-hit check. Native P/D owns its transport; OrbitKV owns independent
cache adapters and Manager peer-cache traffic through OrbiTE.

## Native registration and Restore contract

`register_context_batch(..., tensors=[...])` returns an opaque `GpuRegistration`
that retains actual tensor/exporter objects in registration order.
Explicit registration exports storage IPC metadata in the native
client; Python supplies retained tensors and layouts, without serialized pickle
wrappers. Matching lifecycle-13 packages are required. Storage and view offsets,
GPU UUID, dtype/layout, counter and readiness fields are validated before import.
Earlier installed GPU/adapter and export-error ownership acceptances do not
qualify the current OrbiTE data path; see M1 in the [completion plan](../docs/completion-plan.md). Physical
fault termination and allocation/page-generation qualification remain open.

Use `save(registration, saves)` for Publish. Queued engine callbacks capture that
registration when their producer hands off work; an inactive registration or a
different native client is rejected before Publish. Keeping the object retains
its exporters after unregister, but does not authorize old work or prove device
completion. `start_restore(..., ready_stream=...)` supplies
readiness for previous destination users. Both bundled adapters supply these
arguments. Raw DRAM Restore executes in the engine through imported payload
arenas; Publish and SSD/codec work execute in Manager workers.

Dropping a Python handle or timing out a wait does not release accepted native
operations. Keep logical destination pages held until terminal completion.
Unregister and close drain accepted work. GPU identity uses full UUIDs, so
client and Manager ordinals may differ. Native clients and Manager must be
upgraded together; old wire formats are rejected.

See [ownership and layer/graph readiness](../docs/engine-local-restore.md) and
[the binding stubs](orbitkv/orbitkv.pyi) before implementing another adapter.

