System 1: Local Decision Runtime for AI Agents

Teach a repeatable decision skill from examples or by observing a teacher such as Jev. Validate it, take over locally, and save the skill for reuse.

Version 1.0.1: improved teaching examples, routing calibration, and game policies, plus a public banking-support dataset demo. The core remains teaching bounded decisions, validated local takeover, portable skills, and explicit tool policies. See the results and remaining limits and 1.0 migration guide.

Quickstart · Policy guard · Integrations · Benchmarks · Limits and deployment · Contributing

What it does

• Observe → teach → validate → run locally: keep the teacher answering until the observed skill passes held-out checks, then use the same call site locally. Save and reload the validated skill with its uncertainty behavior intact.
• Structured classification: choice, boolean, multi-choice, and score fields using local NumPy projections, with optional MLX acceleration.
• Uncertainty handling: calibration and conformal prediction sets for routing uncertain decisions to application-defined review or fallback paths.
• Deterministic permissions: PolicyEngine evaluates explicit rules; SystemOneGuard(enforcement_profile=True) requires a matching permission grant, signing key, and durable ledger before returning ALLOW.
• Audit evidence: Ed25519 software signatures (RFC 8032) on decision receipts and a SHA-256 hash-chained SQLite ledger. Applications remain responsible for authenticating callers and enforcing decisions at the tool boundary.
• Local execution: the core decision path requires no network or cloud API. Optional fallback clients, telemetry exporters, and application tools can use the network.

Quickstart

The project is packaged on PyPI as system1. Use Python 3.11 or newer:

python -m pip install system1

To try the code in this repository:

git clone https://github.com/steph4n-gh/system1.git
cd system1
python -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[dev]'

Start with a complete taught skill:

python examples/support_triage.py
system1 decide "Please correct the invoice address" --model .system1/examples/support_triage/skill.s1m --json

The example teaches support routing, calibrates uncertainty, saves the skill, reloads it, and checks separate evaluation cases. It runs entirely locally. For the smallest self-contained Python API example, see teach one skill. A predicted value is not permission to execute a tool.

Observe a teacher, then take over

python examples/observe_routing.py

This example observes answers, teaches one routing skill, passes the normal promotion gates, disconnects the teacher, evaluates fresh requests locally, then saves and reopens the skill. Both the offline run and the live Jev run promoted after 357 observations, answered 40/40 fresh synthetic tickets correctly, accepted all 40, and saved a 5 KB skill. Reloading preserved answers, probabilities, and review decisions. Live observation took about two minutes; local median decisions took about 0.4 ms on the review machine.

The default teacher is a rule over a small structured-ticket vocabulary. --teacher jev uses actual Jev responses with your API key; another API can use the existing teacher callback. These tickets demonstrate the lifecycle, not broad language understanding. See the adapter contract, live result, and three-skill Jev comparison.

Promotion depends on evidence, not a fixed turn count. Insufficient evidence keeps the teacher active, and uncertain local responses still request review.

Teach a skill

Give System 1 labeled examples of one task, save the resulting .s1m skill, and reuse it locally. A person or System 2 can supply the examples; teaching does not require an LLM. The existing compiler fits a small decision head using NumPy.

python examples/support_triage.py
python examples/model_routing.py
python examples/agent_guard.py
system1 decide "Please correct the invoice address" --model .system1/examples/support_triage/skill.s1m --json

Each command teaches from explicit examples, uses separate calibration cases, then saves, reloads, and evaluates the skill. On the review machine, teaching plus calibration took about 0.2 seconds, saved skills were 25–32 KiB, and uncached decisions took about 0.5 ms. Preparing and reviewing labeled examples takes additional work.

These are current results on the primary authored demonstration cohorts; the support result contains one accepted error. A harder fresh routing cohort reaches only 41/44 correct among accepted answers (93.2%). The recorded actual Jev comparison reproduces the original 1.0 lessons separately. See data and confidence intervals and 1.0.1 results and remaining errors. These results do not establish general Jev parity or security detection capability.

For an externally labeled example, teach three banking-support intents from the bundled BANKING77 data:

python examples/banking_support.py
system1 decide "My card was stolen" --model .system1/examples/banking_support/skill.s1m --json

On the complete official test slice for those three intents, the saved skill accepts 118/120 queries, with 115/118 correct (97.5%). Teaching and calibration take about 0.2 seconds locally. This measures three intents, not the full 77-intent benchmark. See source, license, and split details.

For the smallest API example, see teach one skill. The teaching guide covers your own data and evaluation. In Python, use compile(examples, augment=False) and System1Engine(..., strict_mode=True) for this workflow.

Policy guard

This executable example permits one configuration lookup for one application-supplied principal, records the actual lookup result, and verifies the ledger. Other keys or tools have no grant. It persists a local demonstration key across runs.

from pathlib import Path
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from system1 import (
    ActionLedger, ActionProposal, DecisionOutcome, PolicyEngine, PolicyRule,
    RiskLevel, SystemOneGuard, load_private_key, save_keypair,
)

key_path = Path(".system1/demo-identity/identity.key")
if not key_path.exists():
    save_keypair(Ed25519PrivateKey.generate(), key_path.parent)
signing_key = load_private_key(key_path)

policy = PolicyEngine(rules=[PolicyRule(
    rule_id="read_service_name",
    tools=["read_config"],
    allowed_principals=["agent-worker"],
    allowed_tenants=["demo"],
    allowed_scopes=["config:read"],
    argument_limits={"key": ["service_name"]},
    outcome=DecisionOutcome.ALLOW,
    risk=RiskLevel.READ_ONLY,
)])

with ActionLedger(".system1/demo-audit.sqlite", require_durable=True) as ledger:
    guard = SystemOneGuard(
        policy=policy, ledger=ledger, signing_key=signing_key,
        enforcement_profile=True,
    )
    proposal = ActionProposal.create(
        tenant_id="demo", principal_id="agent-worker", scope="config:read",
        tool="read_config", arguments={"key": "service_name"},
        canonical_target="config:service_name", purpose="Inspect service name",
    )
    auth = guard.evaluate_proposal(proposal)
    if auth.outcome != DecisionOutcome.ALLOW:
        raise PermissionError(auth.reason)

    # Execute exactly the authorized operation and arguments.
    result = {"service_name": "system1-demo"}[proposal.arguments["key"]]
    ledger.record_execution_outcome(
        action_id=proposal.action_id,
        receipt_digest=auth.receipt.compute_digest(),
        status="SUCCEEDED", result_payload={"value": result},
        tenant_id=proposal.tenant_id, principal_id=proposal.principal_id,
        scope=proposal.scope, trusted_public_key=signing_key.public_key(),
    )
    assert ledger.verify_integrity(trusted_public_key=signing_key.public_key())
    print(result)

The application must derive identity from its authenticated session, constrain tool arguments and targets, and keep agent code from bypassing the guard. Policy correctness is the operator's responsibility. The default guard without enforcement_profile=True can use statistical classification to allow actions. See deployment boundaries before granting consequential permissions.

Integrations

LangChain

Install python -m pip install 'system1[langchain]'. Attach a configured guard to actual tool invocations:

from system1.integrations import SystemOneGuardCallbackHandler

# guard is your configured SystemOneGuard, with a live ledger and signing key.
handler = SystemOneGuardCallbackHandler(
    guard=guard, tenant_id="demo", principal_id="agent-worker",
)
result = your_tool.invoke(tool_arguments, config={"callbacks": [handler]})

The callback uses scope langchain:tools:exec and the serialized tool input in its proposal. Match policy rules to that contract. SystemOneGuardBlockedException propagates to the caller on a deny or approval requirement. wrap_langchain_tool also supports guarding Python callables directly.

MCP

System1MCPProxy intercepts JSON-RPC tool calls. Supply your configured guard and authenticated identity, then use handle_call(request, executor) or async_handle_call(request, executor) to connect it to your application's executor. See integration tests for the dispatch contract. Diagnostic mode does not produce trusted signed enforcement evidence by default.

FastAPI / ASGI

add_system1_gateway(app, schema=YourSchema) can respond to confident classification requests locally and pass uncertain requests to the downstream application. Install your ASGI framework separately. Configure authentication and request-size limits outside this middleware: local responses bypass downstream endpoint dependencies. This is a classification gateway, not a tool authorization boundary. See deployment guidance.

gRPC

python -m pip install 'system1[grpc]'
system1 serve --grpc --host 127.0.0.1 --port 50051

The bundled protobuf contract supports clients generated for other languages. The CLI server is a local diagnostic service with insecure gRPC transport. Configure signing, ledger, and policy through the Python serve(...) API for a custom deployment. The repository does not supply a published Docker image or Kubernetes manifests.

Observability

Install system1[observability] for Prometheus or system1[otel] for OpenTelemetry. See the observability guide and Grafana dashboard. Starting a metrics server or configuring an exporter changes the application's network behavior.

Benchmarks

Check current manual teaching and the recorded 1.0 Jev baseline from the repository root:

python benchmarks/quality/evaluate_release.py

This command uses saved real responses and blocks network access during local evaluation. It exits unsuccessfully if any primary skill misses 95% accepted accuracy or 80% acceptance. These are observed demonstration targets, not population guarantees.

The following historical 0.2.2 seed-model measurements remain available for comparison:

python benchmarks/quality/run_quality_benchmarks.py
system1 bench --schema triage --iterations 200 --json

The 100-example datasets are small, repository-authored evaluations, not independent security certifications. The launch-review run measured:

See the recorded results and environment and methodology. These are raw classification results, not the accuracy of authorized tool actions. Latency is workload- and hardware-dependent; these measurements do not establish durable end-to-end authorization latency or a service-level guarantee. No cloud providers were measured in this run.

Teaching from the existing examples improves raw accuracy in a separate five-fold development check: intent routing reaches 68% and security triage 71% with 2048 features. In that recorded pre-1.0 run, both required review on every case under strict uncertainty gating; their calibration sets are too small. This is evidence that examples help, not release acceptance evidence. See the teaching comparison for default-dimension results, protocol, and limitations.

Conformal coverage applies to prediction sets under exchangeability and appropriate held-out calibration. It does not guarantee that a singleton prediction is safe, control the error rate conditional on local acceptance, or imply a particular local-retention percentage. Distribution shift and model updates require reevaluation. See limits and the conformal prediction introduction.

Examples and research

• Support triage, model routing, and agent guard teach and check one skill each. Their datasets and measured results are included, along with a comparison of teaching close distinctions.
• Teach one skill, advanced expert examples, and observed teaching and takeover. Cloud modes require explicit configuration.
• Gaming examples explore simulated environments and optional local emulation; they are not independently verified world records. ROMs are not included. The first-use evaluation measures game rules and legal actions separately from raw classifier quality.
• Research manuscripts describe the design and earlier experiments. Their historical timing and quality claims are not release acceptance criteria; use the reproducible measurements above.

Both import system1 and the legacy import reflex expose the same API. The reflex namespace can conflict with the separate Reflex web-framework package, so use separate environments when needed. The TypeSafe adapter supports the basic sync/async decision API, structured state, typed response accessors, and ordinal score distributions. Its documented contract does not include the entire SDK transport/Pydantic surface.

CLI

system1 decide "How do I reset my password?" --schema triage --json
system1 decide "Read documentation" --schema guard --sign --ledger audit.sqlite --json
system1 verify-receipt receipt.json --public-key identity.pub
system1 calibrate --dataset data.json --schema triage --bins 10
system1 compile --schema triage --output triage.s1m --json

Use system1 --help or system1 <command> --help for options. Receipt verification requires a trusted public key supplied independently of the receipt.

Development

python -m pip install -e '.[dev,langchain]'
python -m pytest tests/ -q

Or use the committed lockfile with uv sync --locked --extra dev --extra langchain and uv run --no-sync pytest -q. CI tests Linux and macOS, checks built distributions, and exercises real LangChain dispatch. Optional MLX and emulator tests require their extras and suitable hardware/assets. See contributing and security reporting.

License

Apache License 2.0.