Metadata-Version: 2.5
Name: autoinference-utils
Version: 0.2.7
Summary: Shared endpoint abstractions for autoinference deployments
Requires-Python: >=3.10
Description-Content-Type: text/markdown

# autoinference-utils

Shared endpoint abstractions (`SGLangEndpoint`, `VLLMEndpoint`) for autoinference deployments.

For SGLang models whose custom Transformers code cannot load from symlinked
Hugging Face snapshots, set `AUTOINFERENCE_MATERIALIZE_MODEL_PATH=1`. The endpoint
copies local base/draft model code and metadata into unique temporary directories
and symlinks top-level safetensors weights. Repository IDs pass through unchanged.
Leave the setting unset to preserve the original paths.

## Publishing

Bump `version` in `pyproject.toml` and merge to `main` — CI publishes automatically via trusted publishing.

To publish manually: `cd autoinference_utils && uv build && uv publish`
