CPU-LLM: Evidence-Based AI Engine

Deterministic Document Checks, Financial Covenant Verification & Evidence-Preserving Retrieval on Commodity CPUs.

cpu-llm / cpullm is an embeddable Python engine for source-backed document questions and explicit rule checks. It is intended for architects evaluating controlled evidence workflows in financial compliance, private credit and regulated enterprises, where unsupported answers, arithmetic errors and data egress carry material risk.

Unlike unrestricted generative answering, its strict evidence path selects quotations, executes recognized calculations or returns an explicit nonanswer. This is a bounded evidence contract: it does not establish that a source is true, that every question is understood, or that a deployment is compliant.

This distribution: 1.11.2. Use it instead of 1.11.0 for HTTP; it repairs a cached-appliance SQLite thread-ownership defect. Author: Amin Parva, Independent Engineer.

Licenses: code Apache-2.0; bundled WikiText-2 data has separate CC BY-SA 3.0/GFDL terms and attribution. See third-party notices.

Verification contract compared with generative RAG

This compares execution contracts, not measured product accuracy or cost. Generative RAG systems can also add citations, calculators and deployment controls.

“Zero hallucination” is not a universal accuracy guarantee. Strict selection can preserve an incorrect source, and an unsupported request can be refused. Ordinary mode has different verification limits, including optional learned producers.

Core capabilities

Strict evidence: quote or abstain

After setup, ingest atomic statements and ask an explicit attribute question. Save this as a script; use the main guard on Windows:

from cpu_llm.pipeline import Appliance
from cpu_llm.documents import SourceDocument

def main():
    with Appliance(namespace="label_example", prompt_backend="rules") as app:
        app.ingest(SourceDocument("fictional-label", (
            "The strength of Fictional Luma is 0.75 mg.",
        )))
        for question in (
            "What is the strength of Fictional Luma?",
            "Which antibiotic should Case Birch receive?",
        ):
            result = app.ask(question, strict_evidence=True, small_talk=False)
            print(result["answer_status"])
            print(result["reply"]["text"])
            print(result.get("answer_evidence", []))

if __name__ == "__main__":
    main()

The recorded healthcare demo shows:

What is the strength of Fictional Luma?
status: evidence_quote
The strength of Fictional Luma is 0.75 mg.

Which antibiotic should Case Birch receive?
status: unsupported
Unsupported strict request; use explicit named attribute questions.

Evidence includes document IDs, quotation offsets and chunk hashes. These are fictional label-handling fixtures, not a diagnostic or prescribing system. Use one factual statement per record for the recognized strict grammar; multisentence records can be rejected rather than losing a qualifier. Strict grammar and limits.

Deterministic finance: explicit fields and precision

LedgerCovenant evaluates supplied covenant fields. RefundProof reviews supplied sales-tax findings; it is a separate check, not a covenant synonym. The direct validator needs no database or encoder:

from cpu_llm.finance import validate_message

result = validate_message("""Decide this LedgerCovenant test.
comparator: lte
threshold: 4.50
numerator: compliance_certificate 9000001
denominator: compliance_certificate 2000000
ratio places: 6
""")
print(result["text"])
print(result["checks"])

Raw ratio:               9000001 / 2000000 = 4.5000005
Six-place half-up value: 4.500001x
Allowed threshold:      lte 4.500000x
Decision:               BREACH

The block summarizes the inspectable calculation; the command prints the full result and precedence trace. 4.500001 is the policy-rounded value, not the raw quotient. Precision is explicit, not a universal six-decimal rule or binary-float approximation. Supplied fields are not authenticated financial records, and a breach result is not a legal declaration of default.

Verified document generation: bounded inverse grammar

Using an appliance configured as above:

app.ingest(SourceDocument("helios-manual", (
    "Helios is a backup service.",
    "Helios stores encrypted backups.",
    "Helios uses PostgreSQL for metadata.",
    "Helios does not support automatic regional failover.",
)))
result = app.ask("Write a detailed article about Helios in a friendly tone.")
print(result["reply"]["text"])

Supported predicate joins can produce:

Helios stores encrypted backups and uses PostgreSQL for metadata.

The tone request is removed from the retrieval topic. Bounded inverse grammar checks supported constructions against source predicates while retaining proofs, numbers, negation and qualifiers. Unsupported constructions remain quotations; missing sections remain explicit. Strict documents bypass tone rewriting. This produces bounded new wording, not arbitrary creative prose or new facts. Article/report receipts · Document and tone manual.

Six-signal retrieval and explicit routing

The ordinary pipeline registers 27 graph nodes, including dense, relational, disentangled, hyperbolic, identity and causal retrieval signals backed by PostgreSQL/pgvector. Only dense and identity projection heads are trained and bundled; the other channels use their implemented signals or symbolic paths. A similarity score or connection is not factual proof. Strict requests and document workflows can bypass the ordinary graph; not every request runs every node. See the architecture and prompt-to-node mapping.

Quickstart and reproduction

Python 3.11+ is required. Release checks used Windows/Python 3.12.10.

python -m pip install --upgrade "cpu-llm==1.11.2"
python -m pip check

Document queries require PostgreSQL/pgvector and the pretrained sentence-transformers/all-mpnet-base-v2 encoder assets. The trained projection is bundled; no retraining is required for these demos. Follow the database setup and encoder preparation. Recorded installed-package checks used native PostgreSQL 18/pgvector.

Docker alternative — configuration documented; container startup remains unverified on the maintainer's current host. For a disposable development instance, this command binds only to loopback:

docker run -d --name pgvector-test -p 127.0.0.1:5434:5432 -e POSTGRES_PASSWORD=postgres pgvector/pgvector:pg16
docker exec pgvector-test pg_isready -U postgres -d postgres

After readiness succeeds, enable the extension:

docker exec pgvector-test psql -U postgres -d postgres -c "CREATE EXTENSION IF NOT EXISTS vector WITH SCHEMA public;"

Set the matching connection string before running Python. POSIX shell:

export VECTORPRISM_PG_DSN='postgresql://postgres:postgres@127.0.0.1:5434/postgres'

PowerShell:

$env:VECTORPRISM_PG_DSN = 'postgresql://postgres:postgres@127.0.0.1:5434/postgres'

After database and encoder preparation:

cpu-llm-demo healthcare --out healthcare-demo-output
cpu-llm-demo conversation --out conversation-demo-output

Use fresh output directories. The commands ingest demo records and save responses, evidence and outcome checks. Initial encoder download time depends on connectivity and caching; it is preparation, not training. For disconnected deployments, prepare the Python wheelhouse, database installation/image and encoder cache before transfer. Offline flags are not a substitute for network isolation.

Enterprise and commercial evaluation

For pilot discussions with Insight IT Solutions, visit CPU-LLM research and commercial enquiries. Potential scopes include:

• Private-credit covenant monitoring: map explicit source fields and policies into LedgerCovenant checks, with reviewable arithmetic and evidence.
• On-premises and sovereign deployments: prepare CPU runtime assets and evaluate isolation, identity and gateway controls in the target environment.
• Regulatory workflow adapters: integrate existing tenant/source permissions with enterprise RBAC, retention and audit systems. SOX/HIPAA requirements need a deployment-specific assessment; no certification is claimed.

These are pilot integration opportunities, not claims of deployed customer results, automatic legal compliance or a turnkey zero-trust enclave.

Verification and deployment limits

The recorded 1.11.1 source run reports 905 passed, 56 skipped, 3 expected failures: 54 skips were disabled opt-in integration cases; two lacked optional extractor checkpoints. Separate installed-wheel checks covered strict/ordinary HTTP and the healthcare demo's 12/12 authored outcomes. Commands, scope and receipts. No independent clinical, customer-corpus or hosted-model comparison establishes superiority.

The authenticated HTTP service exposes POST /v1/ask, /v1/ingest, /v1/correct and /v1/delete. It serves requests serially; lazy factories must construct appliances on the serving thread. There is no built-in TLS or production gateway, and long work blocks other requests. See OpenAPI and the consumer guide.

Included in 1.11.2: learned writer fallback requires CPU_LLM_ALLOW_LEARNED_WRITER=1; HTTP bodies are capped at 1 MiB with a 10-second socket inactivity timeout. Explicit child-safety controls warn that no pipeline gate is applied; the standalone gate remains unvalidated. These changes extend published 1.11.1. See the hardening specification.

Known limits include bounded English grammars, lexical topic matching, incomplete aliases/joins and conflict detection. Hashes prove consistency with stored text, not source authenticity or truth. Inspect response status, evidence and coverage for the intended application.

Documentation

• 1.11.1 release notes and build validation.
• Whitepaper and 1.11.1 PDF.
• Strict evidence, tone specification, flag mapping.
• Extended implementation reference and release provenance.

For citations: cpu-llm: Evidence-Based AI Engine, Amin Parva, repository URL, and the exact version/commit evaluated. The Apache code label does not replace bundled-data notices.