ai-gen1 0.1.1

Generate only 1 thing: the outcome.
High-throughput, reflexive semantic branching for agents and LLM applications without unnecessary autoregressive generation.

No waitlist. Open-source & local-first. Run open weights locally on consumer GPUs, vLLM, llama.cpp, or Apple Silicon today.

---

⚡ The Problem: The Generative Tax on Agent Decisions

Most agent and application decisions are small: route this prompt, verify citations, check guardrails, or score customer intent. Standard generative LLMs answer these by spending 1,500ms to 3,000ms decoding verbose JSON strings that your application immediately parses back into an if statement:

Generative LLM (Autoregressive Decode):
User Input ──► [Prefill Pass] ──► Decode Token 1 ──► Decode Token 2 ──► ... ──► Decode Token 80 ──► JSON Parse ──► if (route == "billing")
Latency: 1,840 ms | Tokens Decoded: 77 | Failure Mode: Syntax errors, truncated JSON, hallucinated keys

The gen1 Solution: Terminal Logit Projection

Named after Daniel Kahneman's System 1 (fast, reflexive, instinctive thinking), gen1 eliminates the decode loop entirely. It reads the model's terminal logit distributions (logits[:, -1, :]) during the initial prefill forward pass and directly projects them into normalized candidate choice probabilities:

gen1 (Outcome Only):
User Input ──► [Single Prefill Forward Pass] ──► Terminal Logit Projection ──► Outcome: "billing" (p=0.985)
Latency: 188 ms (9.8x faster) | Tokens Decoded: 0 | Failure Mode: Mathematically 0% schema errors

• 0 Tokens Decoded: No conversational filler, no token streaming, and no autoregressive decoding overhead.
• 10x – 25x Speedup: Sub-200ms decision latency on ordinary consumer desktop GPUs and laptops.
• 100% Schema Guarantee: Invalid JSON, syntax errors, and schema violations are mathematically impossible.
• Native Typed Interface: First-class support for Python Enum, Pydantic models, and Shannon entropy uncertainty scores.

---

📊 Empirical Benchmarks (Consumer Hardware)

All benchmarks below are measured on real consumer hardware with open weights (Gemma 4-E4B 4-bit quantized) under empirical execution — zero cloud mocks.

1. Latency & Token Generation Comparison

2. Decision Throughput & Accuracy Agreement

Empirical Results Table

NVIDIA GeForce GTX 1660 Super (6 GB GDDR6 · CUDA)

Decision Primitive / Task	Normal LLM (Chat + JSON)	gen1 (Outcome Only)	Tokens Decoded	Speedup
Binary Guardrail (Noul)	1,780.2 ms	185.4 ms	68 → 0 tokens	9.6x faster
Multi-Class Routing (Choice)	1,840.5 ms	188.1 ms	77 → 0 tokens	9.8x faster
High-Context 8k Routing (Choice)	3,120.0 ms (14.2s cold)	342.0 ms (560ms cold)	102 → 0 tokens	9.1x – 25.3x faster
Rubric Scoring (Score)	2,140.0 ms	181.6 ms	91 → 0 tokens	11.8x faster
In-Process Direct Logits	~450.0 ms	2.1 ms	50 → 0 tokens	214x faster

Apple MacBook Pro (M4 Pro · 24 GB Unified Memory)

Decision Primitive / Task	Normal LLM (Chat + JSON)	gen1 (Outcome Only)	Tokens Decoded	Speedup
Binary Guardrail (Noul)	1,463.9 ms	168.1 ms	68 → 0 tokens	8.7x faster
Multi-Class Routing (Choice)	1,663.0 ms	170.7 ms	77 → 0 tokens	9.7x faster
High-Context 8k Routing (Choice)	2,419.6 ms (10.8s cold)	271.8 ms (423ms cold)	102 → 0 tokens	8.9x – 25.6x faster
Rubric Scoring (Score)	1,959.5 ms	163.7 ms	91 → 0 tokens	12.0x faster

Accuracy Agreement: gen1 achieves 99.2% empirical decision agreement with 100-token autoregressive JSON outputs while eliminating 100% of schema failures.

---

🚀 Quickstart

Installation

# Python (PyPI)
pip install ai-gen1

# JavaScript / TypeScript (Node.js)
npm install ai-gen1

---

💡 Code Examples

1. Native Typed Decision Engine (Python Enum + Entropy)

Compile-time enum safety and Shannon entropy uncertainty scores:

from enum import Enum
import gen1

class Route(str, Enum):
    BILLING  = "Invoices, payment failures, refunds, and subscriptions"
    SECURITY = "Compromised credentials, vulnerabilities, or auth errors"
    SUPPORT  = "General product troubleshooting and how-to questions"

engine = gen1.load("Qwen/Qwen2.5-7B-Instruct", backend="hf")

decision = engine.decide(
    context="Customer received unauthorized 2FA SMS prompt.",
    options=Route,
)

print(decision.winner)            # Route.SECURITY
print(decision.confidence)        # 0.9912 (calibrated logit probability)
print(decision.entropy)           # 0.08 bits (low entropy = high certainty)
print(decision.tokens_generated)  # 0

2. Multi-Primitive Semantic Decision Client (Python)

Evaluate multiple routing questions, guardrails, and rubrics in a single direct pass:

from gen1 import Client, Choice, Noul, Score

with Client() as client:
    response = client.decide(
        state={"ticket": "Charged $49 twice for invoice #1042. Please refund ASAP!"},
        questions={
            "is_billing": Noul(instructions="Is this ticket a billing dispute?"),
            "priority":   Choice(instructions="Priority tier", criteria={"HIGH": None, "LOW": None}),
            "sentiment":  Score(instructions="Customer sentiment", criteria=["angry", "neutral", "happy"]),
        }
    )

print(response.nouls["is_billing"].passed)      # True
print(response.choices["priority"].choice)      # "HIGH" (confidence = 0.985)
print(response.scores["sentiment"].score)       # 1.2
print(response.usage.output_tokens)             # 0 (Outcome only, 185ms)

3. TypeScript / Node.js Client

import { Gen1Client } from "ai-gen1";

const client = new Gen1Client({
  endpoint: "http://127.0.0.1:8080",
});

const res = await client.choice({
  state: "Kubernetes worker node crashed with OOMKilled 137.",
  question: {
    prompt: "Route alert to responding engineering team",
    options: ["DevOps", "Billing", "Security", "Sales"],
  },
});

console.log(res.choice);          // "DevOps"
console.log(res.confidence);      // 0.984
console.log(res.tokensGenerated); // 0 (168ms)

---

🧩 Core Decision Primitives

Primitive	Purpose	Use Cases	Return Type
Noul	Binary True/False assertion	Guardrails, safety gates, escalation triggers	bool + calibrated probability
Choice	Multi-class semantic routing	Intent routing, triage, tool dispatch	str / Enum + confidence distribution
Score	Normalized rubric rating	Sentiment scoring, RAG relevance, citation check	float on calibrated continuous scale
Advanced	Multi-question atomic evaluation	Run all questions in a single forward pass	Combined typed response dictionary

---

📁 Repository Structure

gen1/
├── sdks/                      # Multi-language client SDKs
│   ├── python/               # Python SDK package (ai-gen1 on PyPI)
│   │   ├── gen1/             # Engine, DecisionResult, Client, Choice, Noul, Score
│   │   ├── tests/            # Python unit & integration test suite
│   │   └── pyproject.toml
│   └── typescript/           # TypeScript / JavaScript client package (ai-gen1 on npm)
│       ├── dist/             # Compiled CJS, ESM, and types
│       ├── test/             # Local client tests
│       └── package.json
│
├── resources/                 # Production resources & evaluations
│   ├── assets/               # Logos & benchmark vector graphics (SVG/PNG)
│   ├── benchmarks/           # Hardware benchmark scripts & datasets
│   │   ├── run_benchmark_1660_super.py
│   │   ├── compare_llm_speed.py
│   │   └── generate_benchmark_charts.py
│   ├── cookbooks/            # 11 runnable production recipes
│   │   ├── 01_classification_confidence.py
│   │   ├── 03_llm_guardrails.py
│   │   ├── 04_rag_passage_classification.py
│   │   ├── 05_semantic_rerank.py
│   │   └── 06_function_calling_router.py
│   └── examples/             # Production integration patterns
│       ├── 01_fastapi_router.py
│       ├── 02_guardrail_gate.py
│       └── 03_pydantic_branch.py
│
└── website/                  # Production documentation & benchmark site
    ├── src/                  # Astro + Starlight components & interactive demos
    └── tests/                # Documentation route verification tests

---

📚 Production Cookbooks

Explore 11 end-to-end runnable recipes located in resources/cookbooks/:

1. Hierarchical Classification — Two-tier topic classification with early exit.
2. LLM Guardrails & Safety Gating — Zero-token input moderation before expensive LLM calls.
3. RAG Passage Filtering — Filter retrieved chunks with sub-millisecond precision.
4. Semantic Reranking — Calibrated rubric scoring across candidate documents.
5. Function Calling Router — Route user queries directly to internal tools.
6. Speculative Cascade — Fast small model routing with fallback to large LLM on low confidence.
7. Citation Fact-Checking — Verify LLM claim consistency against source documents.

---

📄 License

Apache-2.0 License. Free for commercial and research use.