tyle
TYLE (Tiled Inference Layout Encoding): a CPU-native, single-file model format for local LLM inference. Spec: https://csp.gitbook.io/genai/foss/tile
Status
Basic container implemented per spec v0.2 draft:
-
64-byte header with flags, magic
TYLE, little-endian -
GGUF-style KV metadata block (all value types, arrays of scalars/strings)
-
Tensor directory with encoding, quant bits, tile layout, exec order, GPU offload score
-
Optional schedule hints block and NUMA shard map
-
Page-aligned tensor data, ordered by
exec_order_indexwhenFLAG_EXEC_ORDERis set -
Reader returns zero-copy
numpyviews over anmmap -
LUT packing (spec §4.3):
tyle.pack_lut_weightsis a port of T-MACpreprocess_weights.tyle.make_lut_tensorsbuilds the<name>(LUT indices) and<name>.scalestensors with the tuning parameters stored aslut.*.tyle.lut_viewreshapes a loaded tensor to the kernel layout and rejects a tuning mismatch.
Not implemented yet: kernels, AMX packing (payloads are stored as given), sparsity block.
Per-tensor parameters (e.g. lut.bits, amx.source_quant) go in Tensor.extras and are stored in the
metadata block as tensor.<name>.<key>.
Usage
import numpy as np
import tyle
from tyle import constants as C
tf = tyle.TyleFile(
metadata={
"tyle.arch": "llama",
"tyle.name": "demo",
"tyle.context_length": tyle.Typed(C.VT_U32, 2048),
"tyle.embedding_length": tyle.Typed(C.VT_U32, 64),
"tyle.block_count": tyle.Typed(C.VT_U32, 2),
"tyle.head_count": tyle.Typed(C.VT_U32, 4),
"tyle.head_count_kv": tyle.Typed(C.VT_U32, 4),
"tyle.target_isa": ["avx512_vnni"],
"tyle.tokenizer.model": "bpe",
"tyle.tokenizer.tokens": ["a", "b"],
},
tensors=[tyle.Tensor("w", (4, 4), np.zeros((4, 4), np.float32))],
)
tyle.write(tf, "demo.tyle")
print(tyle.load("demo.tyle").tensor("w").data.shape)
LUT tensors
# w: uint8 (M, K), values already biased by 2**(bits-1); scales/zeros: (M, K // group_size)
params = tyle.LutParams(bits=2) # g=4, bm=512, kfactor=16, simd_n_in=16, simd_n_out=8
tensors = tyle.make_lut_tensors("blk.0.attn_q", w, scales, zeros, params)
# ... tyle.write(tyle.TyleFile(metadata, tensors), "model.tyle")
packed = tyle.lut_view(tyle.load("model.tyle").tensor("blk.0.attn_q"), expected=params)
Shapes must satisfy: K % g == 0, (K//g) % kfactor == 0, (M*bits) % bm == 0.
The packed layout is only valid for the tuning it was built with.
Install
/opt/miniconda3/envs/test12/bin/pip install -r requirements.txt
Metadata
Release files for tyle 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| tyle-0.1.0.tar.gz | 11.1 kB | Without provenance | Details for tyle-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| tyle-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for tyle-0.1.0-py3-none-any.whl |
Total release size: 23.2 kB
Release files / tyle-0.1.0.tar.gz
| Download URL | tyle-0.1.0.tar.gz |
|---|---|
| Size | 11.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
aee27b5d4df5f2707103c31d807815e7079391e3e48d664e470eeae03e242591
|
|
BLAKE2b-256 checksum How to use checksums |
50b7794bff1401fa41faeb64efe87a04a148d21c6f2f9fbbcd47b802dd394219
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.12
|
Release files / tyle-0.1.0-py3-none-any.whl
| Download URL | tyle-0.1.0-py3-none-any.whl |
|---|---|
| Size | 12.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
cc63df9883ad2b0128fda55977defab60d155064014bf3bb68486cc3f3458365
|
|
BLAKE2b-256 checksum How to use checksums |
32487084952fed8a0dbeadbf210abb8e10da6d7f4efd79b5f4c05ee113dcccc1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.12
|
Security for tyle 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.