g2n-enterprise 3.19.4

A platform to optimize AND run PyTorch models: license-gated compiler (enhanced planner, persistent cache, multi-accelerator routing), a quantum circuit simulator with batched parameter sweeps, plus a model registry, inference server, and local LLM runtime (int4 quantization, partial offload, hf:/gguf: sources, streaming generation), on top of open-core g2n.

g2n — optimize and run PyTorch models

g2n is a PyTorch platform with two halves:

• Optimize — a torch.compile backend: custom fusion passes, Triton kernels, a persistent compile cache, and (on Enterprise) full autotuning. Honest framing: steady-state latency is ~torch.compile parity on most blocks — the wins are cold-start time (cached kernels load in <1 ms per restart), VRAM, and the serving layer.
• Run — a serving layer that turns those models into a production inference node: model registry, HTTP server, dynamic batching, quantization (int8w/int4w), CUDA-graph replay, and VRAM management for small GPUs. This includes local LLMs: hf: and gguf: models with packed int4 quantization (fused int4 GEMM on CUDA — faster, lower-VRAM decode), partial CPU offload for models bigger than VRAM, and streaming generation over HTTP.

It also ships a quantum circuit simulator (g2n.quantum) — a classical statevector simulator built on torch, for developing and testing quantum algorithms. It is a simulator, not quantum hardware, and never claims otherwise.

import torch, g2n_enterprise as g2n

g2n.activate("G2N-XXXX-XXXX-XXXX")        # once per machine; cached offline

# Optimize: drop-in
model = g2n.compile(my_model)              # or torch.compile(m, backend="g2n")

# Run: registry -> HTTP inference node on :8900
g2n.register_model("clf", "torchscript:/models/clf.pt",
                   precision="int8w",      # weight-only int8: ~4x less weight memory
                   cuda_graph=True, max_batch=16)
g2n.serve()                                # GET/POST http://host:8900/v1/...

# Run a local LLM (Pro+): int4w fits an 8B model on a 6 GB card
g2n.register_model("qwen", "hf:Qwen/Qwen2.5-7B-Instruct",
                   framework="llm", precision="int4w")
g2n.generate("qwen", "Hello!", on_token=print)

Install

pip install g2n-enterprise   # full stack: torch, triton, open-core g2n,
                             # transformers (hf: LLMs), llama.cpp (gguf: LLMs)
pip install g2n              # open-core only (free, Community tier)

The full runtime ships in the base install — no extras to remember. (One caveat: llama-cpp-python compiles from source, so the install needs a C/C++ toolchain, or a prebuilt wheel from the llama-cpp-python docs.)

Python ≥ 3.10. Everything degrades gracefully: no GPU → CPU paths; no license → Community tier; a compile failure → your unmodified model. Your code never breaks because of g2n.

Tiers

	Community (free)	Pro ($49/mo)	Enterprise ($499/mo)
Hybrid fusion + JIT codegen	✓	✓	✓
Enhanced buffer planner (memory fusion)		✓	✓
Persistent compile cache		✓	✓
Model registry + inference server		✓	✓
Quantum simulator: unlimited qubits + fusion	≤24 qubits	✓	✓
Dynamic request batching			✓
Batched quantum parameter sweeps			✓
Multi-accelerator routing + max-autotune			✓
Validated model-zoo configs, priority support			✓

Buy at g2n.dev · seats: Pro 5, Enterprise 25 · 14-day Pro trial on request ([e-mail]).

Documentation

Full documentation lives at https://g2n.dev/docs.

Doc	What it answers
Overview	What g2n is, how the pieces fit, what it is NOT
Getting started	Install → activate → first compile → first serve
Optimize	The compiler: what each tier unlocks, cache, autotune
Serving	The inference node: precision/quantization, batching, HTTP API
LLM	Local LLMs: hf:/gguf: engines, int4w/fp8w, SDPA, offload, streaming /generate
Quantum	The circuit simulator
Licensing	Keys, activation, seats, offline use, renewal

Honest numbers, always

g2n never ships fabricated benchmarks. Every published number is measured by a script in benchmarks/ on named hardware, and the tooling to measure on your hardware is built in:

import g2n_enterprise as g2n
from g2n_enterprise.serve.reference import example_inputs
g2n.benchmark("mlp", example_inputs(batch=32), rounds=200)
# -> eager vs optimized median latency + peak VRAM, on THIS box

Speedups depend on your model and GPU. Measure before you trust — that includes our numbers.