SFS SDK — Spatial Faithfulness Score

A lightweight Python SDK for measuring numerical faithfulness in Retrieval-Augmented Generation (RAG) systems. SFS extracts numerical claims from LLM-generated text and verifies them against ground-truth evidence.

SFS = verified_claims / total_claims

Installation

pip install sfs-sdk

Quick Start

from sfs_sdk import SFSClient

client = SFSClient(preset="property_valuation")

report = client.compute(
    text="The median price is $850,000 located 12.5 km from CBD.",
    evidence=[
        {"key": "median_price", "value": 842000, "unit": "AUD"},
        {"key": "cbd_distance", "value": 12.3, "unit": "km"},
    ],
)

print(f"SFS Score: {report.sfs_score}")        # 1.0
print(f"Verified: {report.verified_claims}/{report.total_claims}")

Features

• Zero LLM dependency — deterministic regex-based claim extraction
• Domain presets — property valuation, medical, legal, financial
• Custom domains — define your own extraction patterns and tolerances
• Pydantic models — fully typed, serializable data structures
• Lightweight — only depends on pydantic

Usage

Extract Claims

from sfs_sdk import SFSClient

client = SFSClient(preset="medical")
claims = client.extract("Patient received 500 mg of amoxicillin for 7 days.")

for claim in claims:
    print(f"  {claim.claim_type}: {claim.extracted_value} {claim.unit}")

Verify Claims

report = client.verify(
    claims=claims,
    evidence=[
        {"key": "dosage", "value": 500, "unit": "dosage"},
        {"key": "duration", "value": 7, "unit": "duration"},
    ],
)
print(f"SFS: {report.sfs_score}")

Verify a Single Claim

result = client.verify_single(
    claim_text="The property is 12.5 km from CBD.",
    claim_type="distance",
    extracted_value=12.5,
    unit="km",
    evidence=[{"key": "cbd_distance", "value": 12.3, "unit": "km"}],
)
print(f"Verified: {result.verified}, Error: {result.error:.2%}")

Custom Domain Configuration

from sfs_sdk import SFSClient, DomainConfig, ExtractionPattern

config = DomainConfig(
    name="My Domain",
    tolerance=0.10,
    extraction_patterns=[
        ExtractionPattern(
            claim_type="temperature",
            pattern=r'([\d.]+)\s*°[CF]',
            unit="degrees",
            min_value=-50,
            max_value=200,
        ),
    ],
)

client = SFSClient(config=config)

Low-Level API

For direct access without the client wrapper:

from sfs_sdk import extract_claims, compute_sfs, EvidencePool, EvidenceItem

claims = extract_claims("The price is $500,000.")
evidence = EvidencePool(items=[
    EvidenceItem(key="price", value=495000, unit="USD"),
])
report = compute_sfs(claims, evidence)

Domain Presets

from sfs_sdk import SFSClient
presets = SFSClient.available_presets()

How It Works

1. Extract — Regex patterns scan the LLM output for numerical claims (prices, distances, percentages, etc.)
2. Match — Each claim is matched to the closest evidence item by unit
3. Verify — Relative error is computed: |claimed - actual| / actual
4. Score — A claim passes if its error is within the domain tolerance

Companion: SFS MCP Toolkit

For LLM-native integration via the Model Context Protocol, see sfs-mcp-toolkit — same engine, exposed as MCP tools for Claude Desktop, Claude Code, and other MCP clients.

Citation

If you use SFS in your research, please cite:

@software{rathnasinghe2025sfs,
  author  = {Rathnasinghe, Ganusha},
  title   = {SFS SDK: Spatial Faithfulness Score for RAG Verification},
  year    = {2025},
  url     = {https://github.com/rganushachadika/sfs-sdk},
}

License

MIT License. See LICENSE for details.