nullspan

Single-pass System One structured extraction, multi-numeric contract disambiguation, calibrated p_null abstention, character-offset [start, end] spans, calibrated noul/choice/score /v1/systemone decisions, cross-field algebraic invariant verification, and calibrated quantile estimation (p10, median, p90) in JavaScript, Python, and Browser WebAssembly ONNX (512-token context + long-document windowing).

• Interactive Playground (Browser WASM ONNX + Zero-Dep Engine): https://hemanth.github.io/nullspan/
• NullspanBench Leaderboard (JevBench-Style 19-System Evaluation, N=500): https://hemanth.github.io/nullspan/bench.html
• Hugging Face Space: https://huggingface.co/spaces/gnumanth/nullspan
• Fine-Tuned Hugging Face Model (345.3M Extraction + 331.1 MB Int8 ONNX + 435M System One Cross-Encoder): https://huggingface.co/gnumanth/nullspan
• Hugging Face Dataset: https://huggingface.co/datasets/gnumanth/nullspan-dataset
• Empirical Science Report: bench/SCIENCE_REPORT.md

In Plain English

Standard LLMs extract data and make typed workflow judgments by generating JSON token-by-token, which is slow, bills per output token, can get hijacked by "ignore previous instructions" prompt injections, and often hallucinates values when a field is missing. nullspan acts like a grounded highlighter + calibrated System One judge instead of a chatbot:

1. Points to Exact Text ([start, end] Span): Highlights exact character positions in your input text (0 generated output tokens), making fabricated strings and prompt-injection payloads impossible.
2. Knows When to Say "Not There" (p_null): Computes a calibrated probability that a field is missing (p_null) and returns null with abstention_reason when evidence is absent (87.00% EM / 89.36% F1 on Stanford SQuAD v2.0).
3. Disambiguates Multi-Numeric Contracts & Checks Math (constraint & verified): Separates competing dollar caps, percentages, day windows, and numeric tranches across multi-section agreements (96/96 on unseen contracts) and verifies equations like subtotal == nodes * hourly_rate.
4. Calibrated /v1/systemone Typed Decisions (decide()): Evaluates noul (true/false), choice (options), and ordinal score (min..max) rubrics with calibrated probability distributions—ranking #2 Overall / #1 Non-LLM Encoder (75.40%) on LocalLLaMA/typed-decisions and hitting 100.0% (48/48) on JevBench Easy (100.0% strict schema validity).

Install

npm install nullspan
pip install nullspan

Usage (JavaScript / ESM)

import { extract } from 'nullspan';

const incident =
  "Incident INC-40928 opened on Apr 12, 2026 with a 502 Bad Gateway errors and a TLS handshake timeouts. " +
  "Issued $1240.50 plus $310.25 SLA credit. Severity is sev-1 and customer data loss was false. " +
  "Root cause CVE not yet assigned. Cluster stabilized on Apr 19, 2026. Reported by Mira";

const result = extract(incident, {
  incident_id: { type: 'text', instructions: 'Incident ticket identifier' },
  opened_on: { type: 'date', instructions: 'Date the incident opened' },
  total_credit: { type: 'number', decimals: 2, unit: 'USD', instructions: 'Total credit including SLA penalty' },
  outage_days: { type: 'duration', unit: 'days', integer: true, instructions: 'Days from incident open to cluster stabilization' },
  failure_modes: { type: 'list', items: 'text', instructions: 'Each failure mode triggered' },
  severity: { type: 'enum', options: ['sev-1', 'sev-2', 'sev-3'], instructions: 'Incident severity tier' },
  data_loss: { type: 'boolean', instructions: 'Whether customer data loss occurred' },
  cve_id: { type: 'text', instructions: 'Assigned CVE identifier if present' },
  remediation_cost: { type: 'estimate', unit: 'USD', min: 100, max: 25000, scale: 'log', step: 10, instructions: 'Estimated engineering remediation cost' },
});

Usage (Python Extraction + /v1/systemone Typed Decisions)

from nullspan import extract, decide

invoice = (
    "Invoice from TensorCore Cloud\n"
    "Invoice #88310 | Issued: 2026-08-14 18:20\n"
    "Nodes: 4 @ $3.50 each\n"
    "Subtotal: $14.00\n"
    "Egress Fee: $0.00\n"
    "Priority Surcharge: $1.75"
)

# 1. Grounded extraction + cross-field algebraic invariants (backend="fast" | "hf" | "onnx")
out = extract(
    state=invoice,
    fields={
        "nodes": {"type": "number", "integer": True, "instructions": "Provisioned GPU node count"},
        "hourly_rate": {"type": "number", "decimals": 2, "unit": "USD", "instructions": "Hourly rate per node"},
        "subtotal": {
            "type": "number",
            "decimals": 2,
            "unit": "USD",
            "constraint": "subtotal == nodes * hourly_rate",
            "instructions": "Compute subtotal before fees and surcharge",
        },
        "promo_rebate": {"type": "number", "decimals": 2, "unit": "USD", "instructions": "Promotional rebate amount if present"},
    },
    backend="hf",
    model_id="gnumanth/nullspan",
)

# 2. TypeSafe-compatible System One typed decisions (noul / choice / score)
judgments = decide(
    state={"invoice_id": "INV-88310", "subtotal": 14.0, "nodes": 4, "unit_price": 3.5, "approved_vendor": True},
    questions={
        "math_valid": {"type": "noul", "instructions": "Does subtotal equal nodes * unit_price?"},
        "routing": {"type": "choice", "options": ["auto_pay", "manual_review", "reject"], "instructions": "Select workflow action."},
        "risk_tier": {"type": "score", "min": 1, "max": 5, "instructions": "Rate audit risk from 1 (low) to 5 (critical)."},
    },
    model_id="gnumanth/nullspan",
)

To run the live TypeSafe /v1/systemone HTTP server for jevbench or LocalLLaMA/typed-decisions:

python training/serve_systemone.py --model-dir gnumanth/nullspan --port 8010

Official External Public Benchmarks (NVIDIA L4 GPU · Zero Test Contamination)

Reproducible via bench/eval_public_benchmarks.py with results stored in bench/public_benchmarks_results.json:

Architecture (NullspanForExtraction 345.3M + NullspanSystemOneEngine 435M)

1. null_head + qa_null_head (Span-Attended Calibrated Abstention): Bilinear span-summary + [CLS] margin residual with held-out temperature scaling (p_null AUROC 0.9933, ECE 0.0142; 0.9951 / 0.0127 in 331.1 MB Int8 ONNX) and abstention_reason (unassigned_placeholder vs missing_field).
2. span_head (Grounded Pointer + Shannon Entropy): Projects 512-token encoder states (plus query-conditioned sliding windows on ultra-long contracts) to [start, end] character spans with calibrated confidence and Shannon span_entropy_bits.
3. bio_head (Multi-Span List Tagging): Extracts ordered text and numeric tranche arrays (type: "list") in a single pass.
4. op_head + quantile_head + Invariant Verifier: Routes 10 System One extraction/math operations, predicts monotone (p10, median, p90) quantiles, and verifies cross-field algebraic constraints (verified: true/false).
5. NullspanSystemOneEngine (/v1/systemone Cross-Option & Cross-Question Judge): DeBERTa-v3-Large (r=32 LoRA) with top-placed programmatic numeric/temporal checks, Head + IDF-Clause + Tail windowing, 2-layer CrossOptionDecisionHead self-attention across candidate options, Cross-Question Workflow Propagation across multi-question batches, and Cumulative Ranked Probability Score (RPS) ordinal calibration.

NullspanBench Leaderboard (bench/run_bench.py · 19 Systems · $N=500$ Zero-Leakage Decisions · NVIDIA L4 GPU)

Modeled after BenchmarkHeaven's JevBench, NullspanBench v3.2 evaluates 19 systems (13 local open-weights/open-code models running live forward passes on Google Cloud NVIDIA L4 GPU, live Cloudflare Workers AI @cf/cloudflare/clairic + @cf/cloudflare/clef, 2 reference JSON-mode LLM decoders, and 3 published JevBench v1.7 reference anchors) across 500 zero-leakage observations (336 Open structured items + 164 Sealed items: 60 Stanford SQuAD v2.0, 96 Multi-Numeric Legal/Commercial Contract fields across 12 unseen agreements, and 8 Adversarial prompt-injection traps) on four axes: Intelligence (I), Calibration (C), Speed (S), and Cost (K).

Explore interactive weight sliders, Pareto frontier plots, and 2-system radar comparisons at https://hemanth.github.io/nullspan/bench.html:

See full methodology, proper scoring definitions, request-type breakdowns (100.0% on Multi-Numeric Contracts n=96, 100.0% Choice n=41, 100.0% Math & Invariants n=51), and head-to-head analysis in bench/SCIENCE_REPORT.md.

License

MIT © Hemanth.HM