Metadata-Version: 2.1
Name: gguf-cpp
Version: 0.0.4
Summary: Unified GUI + engine for GGUF models: local LLM server, image generation and GGUF editing/quantization, all on one bundled gguf.cpp engine
Author-Email: gguf <info@gguf.us>
License: MIT
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Project-URL: Repository, https://github.com/gguf-org/gguf.cpp
Requires-Python: >=3.8
Description-Content-Type: text/markdown

# gguf-cpp

One package for working with GGUF models locally: an OpenAI-compatible LLM
server, a diffusion image generator and a GGUF metadata/tensor editor with a
built-in quantizer — three panels on one GUI, powered by one unified
**gguf.cpp** engine compiled in a single build with a shared set of ggml
kernels.

## Install

```bash
pip install gguf-cpp
```

The build compiles the bundled engine (CPU by default, Metal on macOS).
GPU backends are opt-in at install time:

```bash
GGUF_CPP_CUDA=1 pip install gguf-cpp     # NVIDIA
GGUF_CPP_HIP=1 pip install gguf-cpp      # AMD ROCm
GGUF_CPP_VULKAN=1 pip install gguf-cpp   # Vulkan
```

## Run

```bash
gguf-cpp                 # unified GUI — Server / Diffuser / Editor panels
python -m gguf_cpp       # same thing
```

Each panel also runs on its own, exactly like the standalone
gguf-server / gguf-diffusion / gguf-editor packages did:

```bash
gguf-cpp server          # LLM server GUI
gguf-cpp diffuser        # image generation GUI
gguf-cpp editor          # GGUF editor GUI
```

And the engines are directly scriptable from the CLI:

```bash
gguf-cpp server engine -- --model model.gguf --port 8888
gguf-cpp diffuser engine -- -m sd.gguf -p "a lighthouse at dusk" -o out.png
gguf-cpp editor quantize -m in.gguf -o out-q4_k.gguf --type q4_k
gguf-cpp editor devices
```

## Layout

```
vendor/gguf.cpp/         the unified gguf.cpp engine (one CMake build)
  kernels/               shared ggml kernels (CPU + optional GPU backends)
  llm/                   GGUF LLM runtime + the gguf-server HTTP server
  diffusion/             diffusion runtime + CLI
  quantizer/             quantizer shared library
engine/                  the hand written CMake files, overlaid onto the above
scripts/vendor_engine.py assembles vendor/gguf.cpp from the three checkouts
src/gguf_cpp/            the Python package
  server/ diffuser/ editor/   the three panels (backend + web frontend each)
  gui.py static/         the unified 3-panel GUI shell
```

Each front end under `vendor/gguf.cpp/` is one self-contained directory, so an
engine update touches only the directory it belongs to. The tree is generated,
not edited: everything hand written lives in `engine/` and is laid on top by the
vendoring script.

## Updating the engine

`vendor/gguf.cpp/` is assembled from three sibling checkouts — `../server`,
`../diffusion` and `../quantizer`. Pull those, then:

```bash
python scripts/vendor_engine.py
```

The script copies each checkout into its own subdirectory, drops the two
redundant ggml copies in favour of the shared `kernels/`, applies the handful of
mechanical edits the merge needs (each one reported as it runs), and lays
`engine/` over the result. If an edit no longer matches, it fails rather than
emit a tree that would not build. See `engine/README.md` for what changes and
why.
