Metadata-Version: 2.4
Name: gguf-inspect
Version: 0.1.0
Summary: A from-scratch GGUF parser and CLI inspector - standard library only.
Project-URL: Homepage, https://github.com/shalinis97/gguf-cli-inspector
Project-URL: Issues, https://github.com/shalinis97/gguf-cli-inspector/issues
Author: Shalini S
License-Expression: MIT
License-File: LICENSE
Keywords: ggml,gguf,llama.cpp,llm,quantization
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Filesystems
Requires-Python: >=3.10
Provides-Extra: test
Requires-Dist: gguf>=0.10; extra == 'test'
Requires-Dist: pytest>=7; extra == 'test'
Description-Content-Type: text/markdown

# gguf-inspect

A command-line inspector for GGUF files — the binary container llama.cpp uses
for quantized LLMs. The parser is written from the spec using **only the Python
standard library**; the point is to understand the byte layout, so the code is
commented as a guided tour of the format.

```bash
pip install gguf-inspect
gguf-inspect model.gguf
```

Or straight from a clone, with no install at all:

```bash
python3 -m gguf_inspect model.gguf
pip install .        # from the repo root, if you want the command on your PATH
pip install -e .     # editable, if you plan to change the code
```

**Zero dependencies.** The tensor *data* is never read — only the descriptors
that point at it — so a 2.2 GB model is inspected in about a quarter of a
second, and a 40 GB one costs no more memory than a 4 KB one.

## As a library

```python
from gguf_inspect import parse_gguf, compute_sizes, group_by_layer

f = parse_gguf("phi3-mini-4k-q4.gguf", array_limit=6)

f.get("general.architecture")               # 'phi3'
f.total_elements                            # 3_821_079_552
f.metadata["tokenizer.ggml.tokens"].count   # 32064, without loading 32k strings

t = max(f.tensors, key=lambda t: t.nbytes or 0)
t.name, t.type_name, t.shape, t.nbytes      # 'output.weight', 'Q6_K', (32064, 3072), 80801280

sizes = compute_sizes(f)
sizes.checks                                # [] when every size reconciles with the file
group_by_layer(f, sizes)                    # bytes rolled up per transformer block
```

`dims` is the raw GGUF order (fastest-varying axis first); `shape` is the same
numbers in NumPy row-major order. See the gotchas below.

## Usage

```
python3 -m gguf_inspect PATH [options]

  --json              dump the parsed structure as JSON instead of the report
  --indent N          JSON indentation; 0 for one line (default: 2)
  --max-array N       keep at most N elements of each metadata array
                      (default: 6 for the report, unlimited for --json)
  --key KEY           print one metadata value, unquoted, and exit
  --layers            add a per-layer size rollup (blk.N.* collapsed per row)
  --sort {offset,name,size,elements}    tensor table ordering
  --limit N           max rows in the tensor/layer tables (default: 10)
  -a, --all           print every row (same as --limit 0)
  --no-metadata / --no-tensors / --no-summary
  --width COLS        wrap tables to COLS instead of the terminal width
  --no-color
```

Exit status is 0 on success, 1 on any parse or I/O failure.

The tensor and layer tables print 10 rows by default — a 7B model has ~290
tensors and would otherwise bury the summary. **Row limits only affect what is
printed; every figure in the summary is computed over the whole file**, and a
trimmed table always states the true total. `--sort size --limit 10` reads as
"the ten biggest tensors". `--json` is never trimmed.

```console
$ python3 -m gguf_inspect model.gguf --key general.architecture
llama
$ python3 -m gguf_inspect model.gguf --key tokenizer.ggml.tokens | wc -l
32000
$ python3 -m gguf_inspect model.gguf --json | jq -r '.tensors[] | "\(.name)\t\(.size_bytes)"'
```

## The file format

```
+--------------------------------------------------+  offset 0
| magic              "GGUF"          4 bytes       |
| version            uint32          (2 or 3)      |   header
| tensor_count       uint64                        |
| metadata_kv_count  uint64                        |
+--------------------------------------------------+
| metadata_kv_count x {                            |
|     key         gguf_string (uint64 len + utf8)  |   metadata
|     value_type  uint32                           |
|     value       depends on value_type            |
| }                                                |
+--------------------------------------------------+
| tensor_count x {                                 |
|     name          gguf_string                    |
|     n_dimensions  uint32                         |   tensor
|     dimensions    uint64 * n_dimensions          |   descriptors
|     ggml_type     uint32                         |
|     offset        uint64 (relative to tensor_data)|
| }                                                |
+--------------------------------------------------+
| padding to general.alignment (default 32)        |
+--------------------------------------------------+  <- tensor_data_offset
| raw tensor data                                  |
+--------------------------------------------------+
```

Everything is little-endian. Nothing is seekable: each section's length depends
on the contents of the one before it, so the parse is a single forward pass.

### Three things that are easy to get wrong

**Dimensions are stored in reverse of NumPy order.** `dims[0]` is the
fastest-varying (contiguous) axis, matching ggml's `ne[]`. A layer that PyTorch
calls `(4096, 11008)` is written as `(11008, 4096)`. `TensorInfo` keeps both:
`.dims` as stored, `.shape` reversed into row-major order.

**Tensor offsets are relative to the tensor data section**, not to the start of
the file. Add `tensor_data_offset` for an absolute position.

**Element count tells you nothing about size** until you know the quantization
block layout. `n_bytes = n_elements / block_elems * block_bytes`, and the table
in [`constants.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/gguf_inspect/constants.py) spells out the struct arithmetic
for every type. Q4_K packs 256 weights into 144 bytes — 4.5 bits per weight, not
4, the extra half-bit being the per-sub-block scales.

There is a free consistency check for that last one: tensors sit back-to-back in
offset order, so the gap between consecutive offsets must equal the computed
size plus alignment padding. If the block table is wrong, the file says so.

## Layout

| Path | |
|---|---|
| [`gguf_inspect/constants.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/gguf_inspect/constants.py) | The two enums and the block-size table |
| [`gguf_inspect/reader.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/gguf_inspect/reader.py) | `ByteReader`: a forward-only cursor over an mmap, one method per wire primitive |
| [`gguf_inspect/parser.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/gguf_inspect/parser.py) | Header → metadata → tensor descriptors |
| [`gguf_inspect/sizes.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/gguf_inspect/sizes.py) | Byte sizes, the offset-gap check, layer grouping |
| [`gguf_inspect/report.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/gguf_inspect/report.py) | The terminal report |
| [`gguf_inspect/serialize.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/gguf_inspect/serialize.py) | The `--json` structure |
| [`gguf_inspect/format.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/gguf_inspect/format.py) | Column widths, tables, humanized numbers |
| [`gguf_inspect/cli.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/gguf_inspect/cli.py) | argparse and exit codes |

## Tests

```
./run_tests.sh
```

Creates `./.venv`, installs the reference `gguf` package and pytest *there*,
generates the fixtures, and runs 95 tests. The venv exists only for the tests —
the inspector itself always runs on bare `python3`, and a test walks the
package's ASTs to prove it imports nothing outside the standard library.

The suite has two halves:

- [`tests/test_parser.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/tests/test_parser.py) — 74 tests needing **no
  dependencies at all**, run against byte strings assembled by
  [`tests/handmade.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/tests/handmade.py). That file packs GGUF by hand with
  `struct`, which is both how the malformed cases are produced (bad magic,
  truncated string, overlapping tensors, nested arrays the reference writer has
  no API for) and a decent way to check you have understood the layout — every
  helper is the mirror image of a `ByteReader` method.
- [`tests/test_against_gguf.py`](https://github.com/shalinis97/gguf-cli-inspector/blob/main/tests/test_against_gguf.py) — the cross-check.
  Takes a fixture written by the canonical `gguf` writer, reads it with both
  parsers, and demands they agree on the header, every metadata value, every
  tensor descriptor, and every byte size. It also asserts our hand-transcribed
  block table equals `gguf.GGML_QUANT_SIZES` entry for entry. Skips itself if
  `gguf` is absent.

If you point the tool at a real downloaded model, the offset-gap check in the
summary is the fastest confirmation the block-size table is right.
