🦔 SmartCheck

Check before you sign off. Catch the AI when it's confidently wrong.

IAIso §2 · Verification · SmartTasks.cloud · the Smart* family

Your job is now catching the AI when it's confidently wrong.

As AI reshapes how we work, a new gap opens: you review ai output you didn't produce; confident-wrong slips through. SmartCheck closes it — check at the exact moment the gap bites, and it works the second you clone it (a synthetic demo ships in demo/).

Install

SmartCheck is not published on PyPI or any other package registry yet. Until this section says otherwise, a package called sf-smartcheck on any registry is not ours, and neither is smartcheck.

Install from a clone (Python 3.10 or later):

git clone https://github.com/SmartTasksOrg/sf-smartcheck
cd sf-smartcheck
python -m venv .venv
. .venv/bin/activate          # Windows PowerShell: .\.venv\Scripts\Activate.ps1
python -m pip install .
sf-smartcheck --demo

Status

• Version 3.0.0, experimental. A small deterministic command-line tool with a bundled synthetic demo and two smoke tests.
• Published: nowhere yet; install from a clone (above).
• Tested: the 2 smoke tests in tests/ on Python 3.12, Linux, on every push to master and every pull request (.github/workflows/ci.yml).
• Not tested: Windows and macOS; Python versions other than 3.12.
• Ports: Go, Java, Node and PHP ports in ports/ are checked against the Python reference by ports/conformance/run.sh (run by hand, not in CI); they are not published on any registry.
• Security review: none independent. Report vulnerabilities as described in SECURITY.md.

Run it in your stack

What's in this repo

• Core engine — src/sf_smartcheck/: check() -> Verdict. Deterministic, dependency-free.
• CLI — sf-smartcheck --demo (and --version): a deterministic demo of the core.
• Language ports — ports/: native Go, Java, Node, PHP implementations that reproduce the Python reference, with a shared conformance harness.
• Framework integrations — integrations/: Flowise, OpenAI/Anthropic function-calling, GitHub Action, LangChain, LlamaIndex, MCP server, pre-commit, VS Code extension — each a thin wrapper over one adapter.py bound to the core.
• Also included — a runnable demo/, examples/, the IAIso mapping spec/iaiso-map.json, a browser site/playground.html, plus public smoke tests in tests/.

How it works

Rule IDs are namespaced CHECK-* so output looks kin to the rest of the family (SmartPangolin's SEC-*, etc.). Deterministic, dependency-free, fail-loud.

The data objects (UML)

These are real dataclasses in src/sf_smartcheck/models.py — the diagram and the code are the same thing:

classDiagram
    class Issue {
      +type: str
      +confidence: float
      +span: str
    }
    class Verdict {
      +passed: bool
      +issues: list[Issue]
      +citation_coverage: float
    }
    class IAIsoControl {
      +section: str
      +name: str
    }
    Verdict ..> IAIsoControl : conforms to

Where it sits in the architecture

SmartCheck doesn't stand alone — it stacks with the family, and everything conforms to the IAIso standard — the same standard that governs SmartTasks' own apps, while each tool here stays standalone and drops into your architecture:

graph LR
    IAIso([IAIso standard]):::std
    Cloud([SmartTasks.cloud]):::cloud
    SmartPangolin[SmartPangolin]:::tool
    SmartPrompt[SmartPrompt]:::tool
    SmartCheck[SmartCheck]:::tool
    SmartSeal[SmartSeal]:::tool
    SmartStandard[SmartStandard]:::tool
    SmartSim[SmartSim]:::tool
    SmartMoat[SmartMoat]:::tool
    SmartRoute[SmartRoute]:::tool
    SmartFeed[SmartFeed]:::tool
    SmartPangolin -->|emits clean artifacts to| SmartSeal
    SmartPrompt -->|hands secret/PII flags to| SmartPangolin
    SmartPrompt -->|enforces prompt rules from| SmartStandard
    SmartCheck -->|stamps verified output with| SmartSeal
    SmartCheck -->|checks against rules from| SmartStandard
    SmartSeal -->|issues receipts consumed by| SmartCheck
    SmartSeal -->|issues receipts consumed by| SmartRoute
    SmartStandard -->|supplies rule sets to| SmartPrompt
    SmartStandard -->|supplies rule sets to| SmartCheck
    SmartSim -->|feeds role forecasts to| SmartMoat
    SmartSim -->|draws signals from| SmartFeed
    SmartMoat -->|consumes forecasts from| SmartSim
    SmartRoute -->|verifies receipts from| SmartSeal
    SmartRoute -->|enforces the standard from| SmartStandard
    SmartFeed -->|feeds signals to| SmartSim
    SmartFeed -->|feeds signals to| SmartMoat
    SmartPangolin -.conforms.-> IAIso
    SmartPangolin -.shares IAIso with.-> Cloud
    SmartPrompt -.conforms.-> IAIso
    SmartPrompt -.shares IAIso with.-> Cloud
    SmartCheck -.conforms.-> IAIso
    SmartCheck -.shares IAIso with.-> Cloud
    SmartSeal -.conforms.-> IAIso
    SmartSeal -.shares IAIso with.-> Cloud
    SmartStandard -.conforms.-> IAIso
    SmartStandard -.shares IAIso with.-> Cloud
    SmartSim -.conforms.-> IAIso
    SmartSim -.shares IAIso with.-> Cloud
    SmartMoat -.conforms.-> IAIso
    SmartMoat -.shares IAIso with.-> Cloud
    SmartRoute -.conforms.-> IAIso
    SmartRoute -.shares IAIso with.-> Cloud
    SmartFeed -.conforms.-> IAIso
    SmartFeed -.shares IAIso with.-> Cloud
    IAIso -.governs.-> Cloud
    classDef tool fill:#1c232d,stroke:#f5b83d,color:#efe9f5;
    classDef std fill:#04121f,stroke:#46d6c8,color:#46d6c8;
    classDef cloud fill:#1a1327,stroke:#a78bfa,color:#a78bfa;
    style SmartCheck stroke-width:3px,stroke:#ff6b6b;

• SmartCheck stamps verified output with SmartSeal →
• SmartCheck checks against rules from SmartStandard →

Open site/playground.html for the interactive version.

Part of the Smart* family

One system, not nine projects — same mascot, same manifesto voice, same rule-ID style, all aligned to the IAIso standard. Each is an independent, open-source, single-purpose tool you can integrate into your own architecture:

Backed by the standard: SmartCheck implements IAIso §2 · Verification. Open-source edition: this repo is the simplified, single-purpose version, built for any org to integrate into its own architecture. SmartTasks' desktop app and SmartTasks.cloud run a more advanced, deeply-integrated implementation of the same IAIso governance — a separate product, not this code bundled.

Who's behind this

• Roen Branham — CEO & AI Strategy Architect · CISSP-certified AI, security & governance architect; author of IAIso and sole inventor of the Z4 Semantic Fabric patent application. LinkedIn
• Le Vu Tanh — CTO & Core Engineering Lead · Chief architect of the Cortex engine; large-scale system reliability and low-latency infrastructure — the engineer who ships what gets architected. LinkedIn

The team behind IAIso & SmartTasks: a CISSP-certified security & governance architect and a large-scale systems engineer — 20+ years shipping secure, AI-driven platforms for regulated, blue-chip environments (Allianz, BMW, Rolls-Royce, Heidenhain).

Runs on governed local models

Every card publishes known-answer accuracy and a transparency probe (viewpoint-alignment / over-refusal) — measurable claims you can verify, the way SmartCheck verifies answers.

This tool is local-first, so pair it with models you can actually vet. SmartTasks publishes 21+ governance-validated GGUF builds on Hugging Face — each with a machine-readable scorecard (capability tiers L1 Layman → L5 Agentic, IAIso conformance invariants (pass/warn/fail), OWASP-mapped garak red-team, transparency probes (viewpoint-alignment / over-refusal), and per-file SHA-256). Gate model selection on evidence, not vibes — and every finding, including warnings, is published in full.

→ SmartTasks on Hugging Face · Qwen3.6-27B (L5 agentic) · react-agent-coder-llama-3.1-8b (agentic coder) · gpt-oss-20b (open reasoning)

Get in touch

• Companies & enterprises: enterprise@smarttasks.cloud — we help teams integrate SmartCheck + IAIso into their architecture so governance and audit-readiness become a byproduct of how they already work.
• The standard: IAIso · iaiso.org
• The product: SmartTasks.cloud

Built by SmartTasks Lab. Apache-2.0. Contributions welcome.

Measured effectiveness (benchmarked)

SmartTasks tests this tool against live local models, not just unit fixtures. Headline recall across difficulty levels: 75%.

How to read this. These come from the Smart* effectiveness benchmark: a local model is driven to produce content of increasing difficulty; the tool (flags confident-but-unsourced numbers (incl. percentages & magnitudes) and self-contradictions) is then run and its verdict scored against an independent oracle (broader than the tool's own rules, so a miss is a real gap).

• Recall — of cases that genuinely contained the target, the share the tool caught. Low recall = coverage gap.
• Precision — of what the tool flagged, the share that were real problems. Below 100% = false positives.
• Levels — 0 canary · 1 basic · 2 realistic · 3 obfuscated · 4 adversarial (hardest).
• Invalid — the model failed to produce the scenario (e.g. emitted a placeholder, not a real secret); not scored, so the tool is neither credited nor penalized.
• Sample size — model output varies run-to-run; small n is noisy. Pooled numbers combine recent runs.

What this run shows:

• Instrument check (canary) passes — the fixed sanity cases are all correct, so the higher-level numbers are trustworthy.
• Strong at: basic, realistic, adversarial — near-complete recall.
• Gap at level 3 (obfuscated, hidden via encoding or rephrasing): recall 0% — it misses 100% of confirmed cases here. This is a known coverage limit, tracked for a future improvement.
• No false positives observed (precision 100%) — the tool does not flag clean input.

Source: run 20260804T134233-fad704 · 2026-08-04T13:45:23 · model(s): llama-3.1-8b-lexi-uncensored-v2 · repeats 8. Numbers reflect these model(s); output varies run-to-run, so re-run and regenerate to refresh.