gguf-inspect 0.1.0

A from-scratch GGUF parser and CLI inspector - standard library only.

gguf-inspect

A command-line inspector for GGUF files — the binary container llama.cpp uses for quantized LLMs. The parser is written from the spec using only the Python standard library; the point is to understand the byte layout, so the code is commented as a guided tour of the format.

pip install gguf-inspect
gguf-inspect model.gguf

Or straight from a clone, with no install at all:

python3 -m gguf_inspect model.gguf
pip install .        # from the repo root, if you want the command on your PATH
pip install -e .     # editable, if you plan to change the code

Zero dependencies. The tensor data is never read — only the descriptors that point at it — so a 2.2 GB model is inspected in about a quarter of a second, and a 40 GB one costs no more memory than a 4 KB one.

As a library

from gguf_inspect import parse_gguf, compute_sizes, group_by_layer

f = parse_gguf("phi3-mini-4k-q4.gguf", array_limit=6)

f.get("general.architecture")               # 'phi3'
f.total_elements                            # 3_821_079_552
f.metadata["tokenizer.ggml.tokens"].count   # 32064, without loading 32k strings

t = max(f.tensors, key=lambda t: t.nbytes or 0)
t.name, t.type_name, t.shape, t.nbytes      # 'output.weight', 'Q6_K', (32064, 3072), 80801280

sizes = compute_sizes(f)
sizes.checks                                # [] when every size reconciles with the file
group_by_layer(f, sizes)                    # bytes rolled up per transformer block

dims is the raw GGUF order (fastest-varying axis first); shape is the same numbers in NumPy row-major order. See the gotchas below.

Usage

python3 -m gguf_inspect PATH [options]

  --json              dump the parsed structure as JSON instead of the report
  --indent N          JSON indentation; 0 for one line (default: 2)
  --max-array N       keep at most N elements of each metadata array
                      (default: 6 for the report, unlimited for --json)
  --key KEY           print one metadata value, unquoted, and exit
  --layers            add a per-layer size rollup (blk.N.* collapsed per row)
  --sort {offset,name,size,elements}    tensor table ordering
  --limit N           max rows in the tensor/layer tables (default: 10)
  -a, --all           print every row (same as --limit 0)
  --no-metadata / --no-tensors / --no-summary
  --width COLS        wrap tables to COLS instead of the terminal width
  --no-color

Exit status is 0 on success, 1 on any parse or I/O failure.

The tensor and layer tables print 10 rows by default — a 7B model has ~290 tensors and would otherwise bury the summary. Row limits only affect what is printed; every figure in the summary is computed over the whole file, and a trimmed table always states the true total. --sort size --limit 10 reads as "the ten biggest tensors". --json is never trimmed.

$ python3 -m gguf_inspect model.gguf --key general.architecture
llama
$ python3 -m gguf_inspect model.gguf --key tokenizer.ggml.tokens | wc -l
32000
$ python3 -m gguf_inspect model.gguf --json | jq -r '.tensors[] | "\(.name)\t\(.size_bytes)"'

The file format

+--------------------------------------------------+  offset 0
| magic              "GGUF"          4 bytes       |
| version            uint32          (2 or 3)      |   header
| tensor_count       uint64                        |
| metadata_kv_count  uint64                        |
+--------------------------------------------------+
| metadata_kv_count x {                            |
|     key         gguf_string (uint64 len + utf8)  |   metadata
|     value_type  uint32                           |
|     value       depends on value_type            |
| }                                                |
+--------------------------------------------------+
| tensor_count x {                                 |
|     name          gguf_string                    |
|     n_dimensions  uint32                         |   tensor
|     dimensions    uint64 * n_dimensions          |   descriptors
|     ggml_type     uint32                         |
|     offset        uint64 (relative to tensor_data)|
| }                                                |
+--------------------------------------------------+
| padding to general.alignment (default 32)        |
+--------------------------------------------------+  <- tensor_data_offset
| raw tensor data                                  |
+--------------------------------------------------+

Everything is little-endian. Nothing is seekable: each section's length depends on the contents of the one before it, so the parse is a single forward pass.

Three things that are easy to get wrong

Dimensions are stored in reverse of NumPy order. dims[0] is the fastest-varying (contiguous) axis, matching ggml's ne[]. A layer that PyTorch calls (4096, 11008) is written as (11008, 4096). TensorInfo keeps both: .dims as stored, .shape reversed into row-major order.

Tensor offsets are relative to the tensor data section, not to the start of the file. Add tensor_data_offset for an absolute position.

Element count tells you nothing about size until you know the quantization block layout. n_bytes = n_elements / block_elems * block_bytes, and the table in constants.py spells out the struct arithmetic for every type. Q4_K packs 256 weights into 144 bytes — 4.5 bits per weight, not 4, the extra half-bit being the per-sub-block scales.

There is a free consistency check for that last one: tensors sit back-to-back in offset order, so the gap between consecutive offsets must equal the computed size plus alignment padding. If the block table is wrong, the file says so.

Layout

Path
gguf_inspect/constants.py	The two enums and the block-size table
gguf_inspect/reader.py	ByteReader: a forward-only cursor over an mmap, one method per wire primitive
gguf_inspect/parser.py	Header → metadata → tensor descriptors
gguf_inspect/sizes.py	Byte sizes, the offset-gap check, layer grouping
gguf_inspect/report.py	The terminal report
gguf_inspect/serialize.py	The --json structure
gguf_inspect/format.py	Column widths, tables, humanized numbers
gguf_inspect/cli.py	argparse and exit codes

Tests

./run_tests.sh

Creates ./.venv, installs the reference gguf package and pytest there, generates the fixtures, and runs 95 tests. The venv exists only for the tests — the inspector itself always runs on bare python3, and a test walks the package's ASTs to prove it imports nothing outside the standard library.

The suite has two halves:

• tests/test_parser.py — 74 tests needing no dependencies at all, run against byte strings assembled by tests/handmade.py. That file packs GGUF by hand with struct, which is both how the malformed cases are produced (bad magic, truncated string, overlapping tensors, nested arrays the reference writer has no API for) and a decent way to check you have understood the layout — every helper is the mirror image of a ByteReader method.
• tests/test_against_gguf.py — the cross-check. Takes a fixture written by the canonical gguf writer, reads it with both parsers, and demands they agree on the header, every metadata value, every tensor descriptor, and every byte size. It also asserts our hand-transcribed block table equals gguf.GGML_QUANT_SIZES entry for entry. Skips itself if gguf is absent.

If you point the tool at a real downloaded model, the offset-gap check in the summary is the fastest confirmation the block-size table is right.