Metadata-Version: 2.1
Name: gk-server
Version: 0.0.2
Summary: Local OpenAI-compatible LLM server GUI for GGUF models, powered by the gk engine
Author-Email: gguf <info@gguf.us>
License: MIT
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Project-URL: Repository, https://github.com/gguf-io/gk
Requires-Python: >=3.8
Description-Content-Type: text/markdown

# gk-server

Local OpenAI-compatible LLM server GUI for GGUF models, packaged for Python —
the server half of the unified `ggk` package. The GUI runs in your browser
against a local backend; inference is served by the `gguf-server` C/C++
engine, compiled during `pip install` and bundled with the package as a single
binary. The engine evaluates its graphs on **gk**, an independent tensor
library — there is no llama.cpp checkout and no ggml anywhere in the tree.
Model and template files are referenced by filesystem path through a built-in
file browser — nothing is uploaded or copied to temp storage.

## Install

```bash
pip install gk-server
```

Building the bundled engine requires a C/C++ toolchain and CMake ≥ 3.15 (on
Windows: MSVC Build Tools). The engine source is resolved from the vendored
`vendor/server` copy (see `scripts/vendor_engine.py`) or `GK_SERVER_ENGINE_DIR`.
That tree is self-contained — it carries the gk kernels, the GGUF runtime, the
common layer and the HTTP server — so the build compiles the server binary and
nothing else.

The `vendor/server/gk` kernels are shared verbatim with the `gk-diffuser`
package; both are vendored from the same unified ggk engine tree, so the two
packages always compute with the same gk.

### GPU and accelerator backends

The default build is CPU-only (**on macOS, Metal is on by default** — no flag
needed). Backends are opt-in and can be requested with an environment variable
or a CMake define — the env var is usually easier to pass through `pip`:

```bash
GK_SERVER_CUDA=1   pip install gk-server    # NVIDIA (needs the CUDA toolkit)
GK_SERVER_HIP=1    pip install gk-server    # AMD (needs ROCm/HIP)
GK_SERVER_VULKAN=1 pip install gk-server    # cross-vendor (needs the Vulkan SDK)

CMAKE_ARGS="-DGK_SERVER_CUDA=ON" pip install gk-server   # equivalent
```

Available: `CUDA`, `HIP`, `VULKAN`, `METAL`. Each option maps to the gk
backend of the same name, and the finer-grained `GK_*` knobs (`GK_NATIVE`,
`GK_CUDA_ARCHITECTURES`, …) can still be passed straight through as
`-DGK_<NAME>=…`.

A CUDA build works its own architecture list out from nvcc and the installed
GPUs, and embeds PTX for the newest, so an unlisted card JITs rather than
failing. A wheel built on one machine for another should still say what it
targets, e.g. `CMAKE_ARGS="-DGK_SERVER_CUDA=ON -DGK_CUDA_ARCHITECTURES=89"`.

### HTTPS / OpenSSL

Off by default. OpenSSL is only needed to *download* models over HTTPS
(`-hf` / URL arguments); this package always hands the engine local file
paths. Turn it on with `GK_SERVER_OPENSSL=1` if you script the engine directly
and want HTTPS downloads.

## Run

```bash
gk-server              # GUI on http://127.0.0.1:8642, opens the browser
python -m gk_server    # same thing
gk-server --port 0     # pick a free port; --no-browser to stay headless
```

The LLM server the GUI manages defaults to port 8888 and exposes the usual
OpenAI-compatible endpoints (`/v1/chat/completions`, `/v1/completions`,
`/v1/embeddings`, `/v1/rerank`, `/v1/messages`, `/props`, `/health`, …).

The engine is directly scriptable from the CLI too:

```bash
gk-server engine -- --model model.gguf --port 8888
gk-server engine -- --help
```
