Metadata-Version: 2.4
Name: glyd-gpu
Version: 0.29.0
Summary: Glyd on the GPU: a bf16 model's weights held compressed in GPU memory, bit for bit, and multiplied from there. The GPU half of glyd (pip install "glyd[gpu]"), compiled.
License-Expression: BUSL-1.1
Project-URL: Homepage, https://getglyd.com
Project-URL: Documentation, https://getglyd.com/docs/
Project-URL: Changelog, https://getglyd.com/changelog/
Keywords: compression,lossless,llm,gpu,cuda,bfloat16,transformers,pytorch,vllm
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Environment :: GPU :: NVIDIA CUDA :: 12
Classifier: Environment :: GPU :: NVIDIA CUDA :: 13
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: gpu
Requires-Dist: torch>=2.5; extra == "gpu"
Requires-Dist: transformers<6,>=5.17; extra == "gpu"
Requires-Dist: accelerate>=1.1; extra == "gpu"
Requires-Dist: safetensors>=0.8; extra == "gpu"
Requires-Dist: huggingface_hub<2,>=1.5; extra == "gpu"
Provides-Extra: vllm
Requires-Dist: vllm<0.31,>=0.30; extra == "vllm"
Requires-Dist: safetensors>=0.8; extra == "vllm"
Requires-Dist: huggingface_hub<2,>=1.5; extra == "vllm"
Dynamic: license-file

# glyd-gpu

The GPU half of [Glyd](https://getglyd.com): a bf16 model's weights held compressed in GPU memory, bit for bit, and multiplied
from there (about a third less GPU memory), for PyTorch and transformers, and as a vLLM quantization method.

    pip install "glyd[gpu]"      # glyd.from_pretrained, glyd.fit
    pip install "glyd[vllm]"     # vllm serve MODEL --quantization glyd; glyd run MODEL

This package is what those two extras install: `glyd[gpu]` is `glyd-gpu[gpu]` (PyTorch, transformers, accelerate) and `glyd[vllm]` is
`glyd-gpu[vllm]` (vLLM 0.30); installed on its own it needs none of them. It ships as compiled wheels for Linux with an NVIDIA GPU (Ampere or later; CUDA 12
or 13 builds of PyTorch). Import name: `glyd_gpu`; `python -m glyd_gpu` has `fit`, `pack` and `verify`.

## The engine

`glyd run` starts Glyd's own engine, `glyd-engine` (a program in this package, beside the libraries), for a Qwen3 model on a GPU where it is measured at least as fast as vLLM and with less memory, and where it will
certainly run the model (checked before anything is downloaded), and vLLM everywhere else, as before: it is measured on no GPU yet, so until it is every model runs on vLLM. `glyd serve` runs vLLM, which serves more
users at once than the engine's 8. Where the engine does not get up (it refuses, crashes, is killed or goes silent) the command goes on with vLLM, with one line saying why. `GLYD_ENGINE=0` keeps vLLM, and
`GLYD_ENGINE=1` asks for the engine for `glyd serve`. It needs only the NVIDIA driver and this package's library.

## C, C++, Rust

The wheel carries the GPU libraries (`libglyd_gpu_cuda12.so` and `libglyd_gpu_cuda13.so`: they need only the NVIDIA driver, and are
built with the CUDA 12 and 13 runtimes linked in) and their C header, `glyd_gpu.h`, in the package's directory:

    python -c "import glyd_gpu, os; print(os.path.dirname(glyd_gpu.__file__))"

For a program that multiplies by the matrices of a model saved by `glyd pack` (or `glyd.save_pretrained`), or decodes them back to bf16
bit for bit, without Python: the functions the header declares take the arrays of a pack as the save names them, passed as they are,
and choose how to run each product themselves. Linking: `gcc prog.c -I DIR -I CUDA/include -L DIR -lglyd_gpu_cuda13 -L CUDA/lib64 -lcudart -Wl,-rpath,DIR`.

License: Business Source License 1.1 (LICENSE), as all of Glyd from v0.29.0. Free forever for personal and non-commercial use (including education, research and nonprofits) on computers you own or rent for yourself, and for anyone to try, test and develop with. Commercial use needs a license: suryakoritala@getglyd.com. Each version becomes Apache-2.0 four years after its release.
