🔍 trustandverify

Agentic knowledge verification using Subjective Logic confidence algebra.

trustandverify decomposes research questions into verifiable claims, gathers evidence from multiple search backends, scores confidence using formal Subjective Logic mathematics (Jøsang 2016), and produces provenance-rich reports with per-claim uncertainty quantification.

The Problem

Every major AI system treats confidence as a single scalar — or ignores it entirely. A confidence = 0.5 is meaningless: it could mean "strong evidence that the probability is 50%", or "we have literally no evidence and are guessing." These require fundamentally different downstream decisions.

trustandverify uses Subjective Logic opinion tuples (belief, disbelief, uncertainty, base_rate) so you always know not just what the AI thinks, but how much evidence it has.

Quick Start

pip install trustandverify[tavily,gemini]

import asyncio
from trustandverify import verify

report = asyncio.run(verify("Is remote work more productive than office work?"))
print(report.summary)
for claim in report.claims:
    op = claim.opinion
    print(f"  [{claim.verdict}] {claim.text}")
    print(f"    P={op.projected_probability():.3f}  b={op.belief:.3f} d={op.disbelief:.3f} u={op.uncertainty:.3f}")

Configured Usage

from trustandverify import TrustAgent, TrustConfig
from trustandverify.search import TavilySearch
from trustandverify.llm import GeminiBackend
from trustandverify.storage import SQLiteStorage
from trustandverify.export import JsonLdExporter, MarkdownExporter

config = TrustConfig(num_claims=5, max_sources_per_claim=3)

agent = TrustAgent(
    config=config,
    search=TavilySearch(),
    llm=GeminiBackend(),
    storage=SQLiteStorage("reports.db"),
)

report = await agent.verify("Is nuclear energy safer than solar?")

JsonLdExporter().render_to_file(report, "report.jsonld")
MarkdownExporter().render_to_file(report, "report.md")

Byzantine-Resistant Fusion

Byzantine fusion is off by default — standard cumulative fusion is used. When enabled, it removes discordant, low-trust sources before fusion:

config = TrustConfig(
    enable_byzantine=True,         # activate Byzantine filtering
    byzantine_threshold=0.15,      # discord score cutoff
    byzantine_min_agents=2,        # never reduce below this many
)

When Byzantine is off, score_claim() still runs a lightweight diagnostic and returns meta["byzantine_recommended"] — a flag indicating whether filtering would help, along with cohesion and discord details. This lets you surface a recommendation in your UI without changing the scoring behavior.

CLI

trustandverify "Is coffee healthy?"
trustandverify "Is coffee healthy?" --claims 5 --llm gemini --format markdown
trustandverify ui   # Launch Streamlit dashboard

Why Subjective Logic?

Confidence algebra (from jsonld-ex 0.7.0):

• Cumulative fusion — more independent agreeing sources → lower uncertainty
• Byzantine-resistant fusion — optionally filter discordant, low-trust sources before fusion
• Trust discount — .gov/.edu sources weighted higher than Reddit
• Cohesion scoring — measures overall source agreement (0–1)
• Pairwise conflict & opinion distance — surfaces where sources disagree, quantified with both evidential tension and metric distance

Install Options

pip install trustandverify                              # core only
pip install trustandverify[tavily,gemini]               # minimal working setup
pip install trustandverify[tavily,brave,openai,sqlite]  # typical setup
pip install trustandverify[all]                         # everything

Architecture

trustandverify/
├── core/        — TrustAgent, pipeline, models, config
├── scoring/     — Subjective Logic algebra (wraps jsonld-ex 0.7.0: fusion, Byzantine, cohesion, distance)
├── search/      — SearchBackend protocol + Tavily, Brave, Bing, SerpAPI
├── llm/         — LLMBackend protocol + Gemini, OpenAI, Anthropic, Ollama
├── storage/     — StorageBackend protocol + memory, SQLite, Postgres, Neo4j
├── cache/       — CacheBackend protocol + file cache, Redis
├── export/      — ExportBackend protocol + JSON-LD, Markdown, HTML, PDF
├── cli/         — Typer CLI
└── ui/          — Streamlit dashboard

Documentation

• Getting Started — installation, quick start, CLI, Jac integration
• Backends Guide — configure search, LLM, storage, cache, and export backends
• Confidence Algebra — the Subjective Logic mathematics
• API Reference — full module and function documentation

References

• Jøsang, A. (2016). Subjective Logic: A Formalism for Reasoning Under Uncertainty. Springer.
• jsonld-ex — JSON-LD 1.2 extensions with Subjective Logic confidence algebra
• W3C PROV-O — Provenance Ontology

License

MIT — see LICENSE.