g2n — run models that don't fit, on the GPU you already have
A PyTorch compiler (custom FX fusion passes, a Triton LayerNorm kernel, a
persistent compile cache) plus a license-gated runtime that serves models and
runs local LLMs larger than your VRAM. Also ships g2n.quantum, a
statevector quantum circuit simulator (classical simulation — not quantum
hardware).
Measured on a single RTX 4050 Laptop (6 GB) with
benchmarks/bench.py
in this repo — run it on your own card and post what you get:
- EuroLLM-9B at int4 weights, generating at 2.89 tok/s on a 6 GB card. 33 of its 42 layers stay resident; the rest stream from pinned host RAM with the transfers overlapped against compute. The planner sizes that split automatically and lands within 0.3% of the best split found by sweeping every option by hand.
- 3.5× faster cold compile than stock
torch.compileon a post-reboot run.
pip install g2n torch
import torch, g2n
compiled = g2n.compile(model) # == torch.compile(model, backend="g2n")
y = compiled(x)
Free (Community) gives you the g2n fusion passes on stock Inductor and quantum circuits up to 24 qubits. A license key unlocks more — activation is one command and then fully offline:
export G2N_LICENSE_KEY=G2N-XXXX-XXXX-XXXX
g2n activate && g2n status
| Unlock | Tier |
|---|---|
| Persistent compile cache (warmup once per machine, not per run) | Pro |
| Enhanced planner: epilogue fusion + custom Triton kernels | Pro |
Serving platform (pip install g2n-enterprise): registry, HTTP node, quantization incl. weight-only int8, CUDA graphs |
Pro |
| Quantum: unlimited qubits + circuit fusion | Pro |
| max-autotune, dynamic batching, batched quantum sweeps, model zoo | Enterprise |
Run a .g2n packaged model — free, no license
A .g2n pack is a model that is already quantized on disk, so opening it is an
mmap instead of a re-quantization. Reading one is free and unlicensed and
lives right here in the Apache-2.0 package:
from g2n.pack import load_packed, inspect
print(inspect("qwen3-8b-int4.g2n")["precision"]) # works with no torch installed
model, manifest = load_packed("qwen3-8b-int4.g2n")
The loader builds the module tree on device="meta" (allocating nothing), then
binds the packed tensors straight from the memory-mapped file — no fp16
intermediate and no quantization work at load. Creating a pack (g2n pack) is a
paid feature; running one never is.
Quantum in 20 seconds
import g2n.quantum as qf
c = qf.Circuit(2).h(0).cnot(0, 1) # Bell state
c.measure_all(shots=1000) # {'00': ~500, '11': ~500}
c.expectation("ZZ") # tensor(1.)
Guarantees
- Never worse than eager: any compile failure returns your unmodified model with a one-line warning.
- Offline after activation: license tokens are Ed25519-verified locally; no phone-home during runs.
- Honest numbers: no benchmark claim without named hardware and a script to reproduce it — including on your own machine.
Docs, pricing, benchmarks: g2n.dev · full manual at g2n.dev/docs · questions: support@g2n.dev.
What's open and what isn't
This repository is the free core: the compiler, the fusion passes and the
quantum simulator, Apache-2.0, no obfuscation. The LLM offload and serving
numbers quoted above come from g2n-enterprise, which is proprietary and paid.
Nothing in either package phones home during a run.
1 maintainer
Metadata
1 maintainer
Release files for g2n 1.16.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| g2n-1.16.0.tar.gz | 73.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| g2n-1.16.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 140.5 kB
Release files / g2n-1.16.0.tar.gz
| Download URL | g2n-1.16.0.tar.gz |
|---|---|
| Size | 73.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
74becdb7bb453a7d9979ac11da1b01695b1e4b7f3e8f96d8e4936d82c251e057
|
|
BLAKE2b-256 checksum How to use checksums |
d88bbf5e17f119df9785459f8baca8174c429f2cc38563cf52f2181edf19cd86
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.3
|
Release files / g2n-1.16.0-py3-none-any.whl
| Download URL | g2n-1.16.0-py3-none-any.whl |
|---|---|
| Size | 67.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
26b06173d0974cdecd04781482cbec86a76ccb498c9eee1625cdea9bfac04f21
|
|
BLAKE2b-256 checksum How to use checksums |
a57c73fc7737b13d80489e8e397768ac2c842d56950513df543f624c892c99db
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.3
|
