Metadata-Version: 2.4
Name: glyd-gpu
Version: 0.27.0rc2
Summary: Glyd on the GPU: a bf16 model's weights held compressed in GPU memory, bit for bit, and multiplied from there. The GPU half of glyd (pip install "glyd[gpu]"), compiled.
License-Expression: BUSL-1.1
Project-URL: Homepage, https://getglyd.com
Project-URL: Documentation, https://getglyd.com/docs/
Project-URL: Changelog, https://getglyd.com/changelog/
Keywords: compression,lossless,llm,gpu,cuda,bfloat16,transformers,pytorch,vllm
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Environment :: GPU :: NVIDIA CUDA :: 12
Classifier: Environment :: GPU :: NVIDIA CUDA :: 13
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: gpu
Requires-Dist: torch>=2.5; extra == "gpu"
Requires-Dist: transformers<6,>=5.17; extra == "gpu"
Requires-Dist: accelerate>=1.1; extra == "gpu"
Requires-Dist: safetensors>=0.8; extra == "gpu"
Requires-Dist: huggingface_hub<2,>=1.5; extra == "gpu"
Provides-Extra: vllm
Requires-Dist: vllm<0.31,>=0.30; extra == "vllm"
Requires-Dist: safetensors>=0.8; extra == "vllm"
Requires-Dist: huggingface_hub<2,>=1.5; extra == "vllm"
Dynamic: license-file

# glyd-gpu

The GPU half of [Glyd](https://getglyd.com): a bf16 model's weights held compressed in GPU memory, bit for bit, and multiplied
from there (about a third less GPU memory), for PyTorch and transformers, and as a vLLM quantization method.

    pip install "glyd[gpu]"      # glyd.from_pretrained, glyd.fit
    pip install "glyd[vllm]"     # vllm serve MODEL --quantization glyd; glyd run MODEL

This package is what those two extras install: `glyd[gpu]` is `glyd-gpu[gpu]` (PyTorch, transformers, accelerate) and `glyd[vllm]` is
`glyd-gpu[vllm]` (vLLM 0.30); installed on its own it needs none of them. It ships as compiled wheels for Linux with an NVIDIA GPU (Ampere or later; CUDA 12
or 13 builds of PyTorch). Import name: `glyd_gpu`; `python -m glyd_gpu` has `fit`, `pack` and `verify`.

## C, C++, Rust

The wheel carries the GPU libraries (`libglyd_gpu_cuda12.so` and `libglyd_gpu_cuda13.so`: they need only the NVIDIA driver, and are
built with the CUDA 12 and 13 runtimes linked in) and their C header, `glyd_gpu.h`, in the package's directory:

    python -c "import glyd_gpu, os; print(os.path.dirname(glyd_gpu.__file__))"

For a program that multiplies by the matrices of a model saved by `glyd pack` (or `glyd.save_pretrained`), or decodes them back to bf16
bit for bit, without Python: the functions the header declares take the arrays of a pack as the save names them, passed as they are,
and choose how to run each product themselves. Linking: `gcc prog.c -I DIR -I CUDA/include -L DIR -lglyd_gpu_cuda13 -L CUDA/lib64 -lcudart -Wl,-rpath,DIR`.

License: Business Source License 1.1 (LICENSE). Its additional use grant covers personal, educational, research and other non-commercial
production use; commercial production use needs a separate license from the licensor.
