ternary_QAT

Lightweight ternary QAT for Ternary-Bonsai unpacked text models.

Qwen 3.5 is currently being tested.

This is designed from the start to be Unsloth compatible.

Weights are ternarized to {-1, 0, 1} per group of (128/64/user-defined) consecutive weights along the last dim, matching the Bonsai on-disk format (verified bit-exact). Embeddings + all nn.Linear modules (attn, MLP, lm_head) are ternarized; norms stay FP.

Based on Prism-ML's whitepaper: https://github.com/PrismML-Eng/Bonsai-demo/blob/main/ternary-bonsai-8b-whitepaper.pdf

Install

Have your preferred torch version installed first so that this doesn't install the CPU version (which you probably don't want)

pip install ternary_QAT                 # core (torch only)
pip install "ternary_QAT[peft,transformers]"  # + LoRA / model loading

if you use uv:
uv pip install ternary_QAT --torch-backend=auto

Use

Full-finetune

from transformers import AutoModelForCausalLM
from ternary import swap_linear, TernaryConfig

model = AutoModelForCausalLM.from_pretrained("prism-ml/Ternary-Bonsai-1.7B-unpacked")
swap_linear(model, TernaryConfig(group_size=128))
# ... train normally; ternarize fires in every Linear.forward

LoRA (ternary frozen base + FP adapters)

from peft import LoraConfig, get_peft_model
from ternary import swap_linear, TernaryConfig, ternarize_lora_params, reternarize_merged_linears

model = ...  # load model
swap_linear(model, TernaryConfig(group_size=128))
model = get_peft_model(model, LoraConfig(r=128, lora_alpha=128, ...))

# ... train ...

# at save: ternarize adapter, merge, re-ternarize merged linears
ternarize_lora_params(model)
model = model.merge_and_unload()
reternarize_merged_linears(model)
model.save_pretrained("./out")

See examples/ for LoRA, FFT, and Unsloth examples.

Learning rate

Ternary QAT needs 10-50x higher LR than standard fine tuning.

The lowest usable LR I've found so far is around 7e-4, so experiment in that range up to the e-3s, depending on rank and dataset size.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ternary_qat-0.1.5.tar.gz (6.3 kB view details)

Uploaded Aug 3, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

ternary_qat-0.1.5-py3-none-any.whl (7.4 kB view details)

Uploaded Aug 3, 2026 Python 3

File details

Details for the file ternary_qat-0.1.5.tar.gz.

File metadata

Download URL: ternary_qat-0.1.5.tar.gz
Upload date: Aug 3, 2026
Size: 6.3 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for ternary_qat-0.1.5.tar.gz
Algorithm	Hash digest
SHA256	`24ea7a2a5c4c4d84c006fcb7eb9c9fb74ba42d19642d6f89e83c7b5593832ae6`
MD5	`051d674971e014b6a6cc33f2fa3b8f46`
BLAKE2b-256	`9aef3d978e83f412fb884d3970a2de20eb4454523f85abbe7c03c6277c06fc2e`

See more details on using hashes here.

Provenance

The following attestation bundles were made for ternary_qat-0.1.5.tar.gz:

Publisher: publish.yml on electroglyph/ternary_QAT

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: ternary_qat-0.1.5.tar.gz
- Subject digest: 24ea7a2a5c4c4d84c006fcb7eb9c9fb74ba42d19642d6f89e83c7b5593832ae6
- Sigstore transparency entry: 2334492804
- Sigstore integration time: Aug 3, 2026
Source repository:
- Permalink: electroglyph/ternary_QAT@43081d1f21ac04bb0ba3e44e362214835231891d
- Branch / Tag: refs/tags/v0.1.5
- Owner: https://github.com/electroglyph
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@43081d1f21ac04bb0ba3e44e362214835231891d
- Trigger Event: push