Skip to main content

Wauldo fact-checking as a NeMo Guardrails output rail

Project description

wauldo-nemo

Wauldo fact-checking as a NeMo Guardrails output rail. Every bot message is verified against its retrieved context before it reaches the user; unsupported answers are refused, weakly-supported ones are annotated.

This is a thin adapter — all verdict logic lives in the wauldo SDK. This repo only maps a /v1/fact-check result to a rail decision and never re-implements parsing.

Install

pip install wauldo-nemo[nemo]

Requires wauldo>=0.19 (the version that exposes HttpClient.fact_check). While 0.19 is unreleased, install the SDK from source: pip install -e ../wauldo-monorepo/sdk-python.

How it works

bot message ──▶ wauldo_fact_check action ──▶ HttpClient.fact_check(text, context)
                                              │
                                              ▼
                                     policy.decide(result, thresholds)
                                              │
                          allow ──▶ pass   review ──▶ annotate   block ──▶ refuse

The policy never downgrades the server's verdict — thresholds can only make the rail stricter (e.g. strict=True turns review into a refusal, max_hallucination_rate forces a refusal above a ceiling).

Usage

from nemoguardrails import LLMRails, RailsConfig
from wauldo_nemo import register, PolicyThresholds

rails = LLMRails(RailsConfig.from_path("examples/config"))
register(rails, thresholds=PolicyThresholds(strict=True, max_hallucination_rate=0.4))

See examples/ for a runnable config (config.yml + rails.co).

Configuration

Threshold Default Effect
min_confidence 0.0 Below it, an allow is downgraded to annotate.
max_hallucination_rate 1.0 Above it, the response is refused.
strict False When set, a server review becomes a refusal.

Environment: WAULDO_API_KEY (required), WAULDO_BASE_URL (defaults to https://api.wauldo.com).

Develop

pip install -e '.[dev]'
pytest        # policy tests, no guardrails runtime needed

License

MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

wauldo_nemo-0.1.0.tar.gz (6.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

wauldo_nemo-0.1.0-py3-none-any.whl (6.5 kB view details)

Uploaded Python 3

File details

Details for the file wauldo_nemo-0.1.0.tar.gz.

File metadata

  • Download URL: wauldo_nemo-0.1.0.tar.gz
  • Upload date:
  • Size: 6.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for wauldo_nemo-0.1.0.tar.gz
Algorithm Hash digest
SHA256 9a41e47423a98161e818746c5a02679c21b02f80b0ec4980d98233f0cecfa774
MD5 ea258a9b12d2d963e172e49636482641
BLAKE2b-256 160682e4d8bc7cec26c1f84876fd309ba64779c4385f3f2d472596aa0e280cd5

See more details on using hashes here.

File details

Details for the file wauldo_nemo-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: wauldo_nemo-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 6.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for wauldo_nemo-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 1ed5ebb5faefe27eec70102e3067168f50c71dd6e5df7b07a88de7f7734072f7
MD5 99fa42fa9f0525174894952c86e28ae5
BLAKE2b-256 f9230d40951effb8f72d1aee49bf34f1341ecbb7535f7f4aaf5de15410cd699b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page