cpu-llm 1.10.0

A local, CPU-first Python library for questions about your documents and explicit rules. It combines trained retrieval with bounded sentence construction, structured finance checks, source evidence and explicit nonanswers. It is not an unrestricted conversational or clinical decision model.

Included in this release

• Strict evidence policy v3: source quotations, recognized conflict checks, bounded batches/joins, exact arithmetic and explicit unsupported outcomes.
• A reviewed default conversation pack: eight basic intents, 51 phrases, original application identity/help, controlled variants and disable controls.
• A verified stylistic companion node for five factual relations. It retains accepted claims/proofs; unsupported expressive tones can use neutral fallback.
• Prompt interpretation, request separation, clarification, response budgets, source permissions, numeric precision fixes and Markdown/HTML output layouts.
• RefundProof, LedgerCovenant and Ledgerlock supplied-field checks.
• The trained VectorPrism projection, sample corpora, conversation pack and fictional healthcare/conversation demo datasets.
• Installed demo and conversation-candidate commands; no repository clone or Git executable is needed for those commands.

Install and prepare

After this version has been published:

python -m pip install "cpu-llm==1.10.0"

Before asking document questions, configure PostgreSQL with pgvector and set VECTORPRISM_PG_DSN. The package includes the trained projection; the pretrained sentence-transformers/all-mpnet-base-v2 encoder assets must also be downloaded on first use or cached for offline use. Installation does not start PostgreSQL or ingest your documents. No retraining is needed for the bundled demos.

For optional spaCy prompt parsing, install cpu-llm[prompt_nlp]==1.10.0 and provision en_core_web_sm. The examples below use the built-in rules parser.

from cpu_llm.documents import SourceDocument
from cpu_llm.pipeline import Appliance

def main():
    with Appliance(namespace="my_demo", strict_evidence=True,
                   prompt_backend="rules") as app:
        app.ingest(SourceDocument("manual", (
            "The material of Example Device is steel.",
        )))
        result = app.ask("What is the material of Example Device?")
        print(result["answer_status"])
        print(result["reply"]["text"])
        print(result["answer_evidence"])

if __name__ == "__main__":
    main()

Keep the guarded main() on Windows because the engine starts worker processes. Reuse an appliance and close it after use. Production applications should use authenticated caller contexts or the HTTP entry service for tenant/source scope.

Run the packaged demos

With a demo database and cached encoder assets:

cpu-llm-demo healthcare --out healthcare-demo-new-run
cpu-llm-demo conversation --out conversation-demo-new-run

Each output directory must be new. The commands create isolated namespaces, ingest packaged fictional records and save actual responses and expected-result checks. Healthcare examples demonstrate label/record lookups, equivalent units, conflicts, missing information, treatment refusal and arithmetic. Correct nonanswers count as expected outcomes, not correct medical answers.

The candidate review tool is also installed:

cpu-llm-conversation-candidates --help
cpu-llm-serve --help
cpu-llm-ingest --help

Response and safety boundaries

Use answer_status, evidence, per-task outcomes and structured calculation fields together. A nonempty reply.text alone is not proof of correctness. Default social replies have response_kind="conversation", empty evidence and no clinical validation. Disable them with small_talk=False or service --no-small-talk. Pure conversational matching never replaces a substantive healthcare/finance question with a greeting response.

Strict mode bypasses learned answer generators, including the automatic fact-bridge FLAN-T5 fallback. It does not validate source truth, infer arbitrary prose contradictions or guarantee 100% accuracy/clinical safety. Outside strict mode, optional learned fallback paths can require additional model assets that are not bundled. Coreferee, FastCoref and spaCy model assets are also not bundled.

The tone companion is opt-in through Python style=...; it does not replace the canonical reply or give unverified facts an audit. Strict mode bypasses it. Financial checks operate on supplied fields/rules, not independently authenticated documents or professional tax/legal advice. Fictional record lookup does not establish diagnostic or prescribing capability.

Documentation and real examples

• Full guide and setup
• Strict evidence contract
• Healthcare sample chat
• Tone-node chat examples
• Default conversation examples
• Local medical acceptance comparison

The project is Apache-2.0 licensed. The shipped default pack is original; external dialogue corpora are not bundled. Dataset review tooling does not import external assistant answers into the runtime or medical knowledge base.