zero-harm-ai-detectors 0.2.8

Privacy & content safety detection with heuristic and AI modes

Zero Harm AI Detectors

Privacy & content safety detection with heuristic default and optional AI config.

One API, Two Execution Paths

Feature	detect(text)	detect(text, ai_config=...)
Speed	⚡ 1–5ms	🐢 50–200ms
Email, Phone, SSN	✅ 95–99%	✅ 95–99%
Credit Card (Luhn)	✅ 99%	✅ 99%
Secrets / API Keys	✅ 95%	✅ 95%
Person Names	⚠️ 20–60% (tiered)	✅ 85–95%
Locations	❌	✅ 80–90%
Organisations	❌	✅ 75–85%
Harmful Content	✅ Pattern-based	✅ Contextual AI
Extra dependencies	None	transformers/torch and/or spacy

Installation

# Heuristic mode only (fast, no ML dependencies)
pip install zero_harm_ai_detectors

# With AI mode (~2 GB model download on first use)
pip install 'zero_harm_ai_detectors[ai]'

On Python 3.9, the ai extra installs the spaCy 3.7 line for compatibility.

Quick Start

from zero_harm_ai_detectors import detect, AIConfig

# Heuristic mode (default) — fast, great for structured data
result = detect("Email: [e-mail], SSN: 123-45-6789")
print(result.redacted_text)
# → Email: [REDACTED_EMAIL], SSN: [REDACTED_SSN]

# AI mode — enabled when ai_config is provided
result = detect("Contact John Smith at Microsoft in NYC", ai_config=AIConfig())
print(result.detections)
# → [Detection(PERSON, ...), Detection(ORGANIZATION, ...), Detection(LOCATION, ...)]

Detection Result

Both modes return an identical DetectionResult:

result = detect(text)  # heuristic
result_ai = detect(text, ai_config=AIConfig())  # ai

result.original_text   # str   — original input
result.redacted_text   # str   — sensitive content replaced
result.detections      # list  — List[Detection]
result.mode            # str   — "heuristic" or "ai"
result.harmful         # bool  — harmful content found
result.severity        # str   — "none" | "low" | "medium" | "high"
result.harmful_scores  # dict  — per-category scores

result.to_dict()       # full dict with all fields
result.get_pii()       # List[Detection] — PII only
result.get_secrets()   # List[Detection] — secrets only

Target Selection

from zero_harm_ai_detectors import detect, DetectTarget

# Default = all targets
detect(text)

# PII only
detect(text, targets=DetectTarget.PII)

# Secrets only
detect(text, targets=DetectTarget.SECRET)

# Harmful only
detect(text, targets=DetectTarget.HARMFUL)

# Combine
detect(text, targets=DetectTarget.PII | DetectTarget.SECRET)

PII Detection

Structured data (heuristic, 95–99% accuracy)

from zero_harm_ai_detectors import (
    detect_emails,
    detect_phones,
    detect_ssns,
    detect_credit_cards,
    detect_bank_accounts,
    detect_dob,
    detect_addresses,
)

emails = detect_emails("Contact [e-mail] or [e-mail]")
# → [Detection(EMAIL, '[e-mail]', ...), Detection(EMAIL, '[e-mail]', ...)]

cards = detect_credit_cards("Card: 4532-0151-1283-0366")
# → [Detection(CREDIT_CARD, '4532-0151-1283-0366', confidence=0.99)]  # Luhn validated

Person names — heuristic mode (tiered confidence)

Heuristic-mode person name detection uses three tiers to balance recall against false positives. Every PERSON detection carries metadata["tier"] and a confidence value from the documented set {0.20, 0.45, 0.60}.

Tier	Confidence	Trigger	Precision
1 — titled	0.60	Name preceded by a title: Mr, Mrs, Ms, Miss, Dr, Prof, Rev, Capt, Lt, Sgt, Cpl, Pvt	High
2 — context	0.45	Name follows a context keyword: Name:, Contact:, From:, To:, Cc:, Bcc:, Signed by:, Authored by:	Medium
3 — bare	0.20	Plain "First Last" capitalised pair with no surrounding signal — high false-positive rate	Low

Tiers 1 and 2 are checked first; any span they cover is suppressed from Tier 3, so a titled name never appears as a duplicate bare match.

from zero_harm_ai_detectors import detect_person_names_heuristic

detections = detect_person_names_heuristic(
    "Dr. Alice Brown arrived. Contact: Bob White. Meet James Green tomorrow."
)

for d in detections:
    print(f"{d.text!r:20}  tier={d.metadata['tier']:8}  confidence={d.confidence}")

# 'Dr. Alice Brown'     tier=titled    confidence=0.6
# 'Bob White'           tier=context   confidence=0.45
# 'James Green'         tier=bare      confidence=0.2

Filtering by confidence threshold is the recommended way to tune precision vs. recall for your use case:

# High-precision only: titled and context names
precise = [d for d in detections if d.confidence >= 0.45]

# Maximum recall: include bare matches too
all_names = detections  # confidence >= 0.20

# Inspect tier directly
titled_only = [d for d in detections if d.metadata.get("tier") == "titled"]

Tip: For reliable person-name detection (85–95% accuracy) use detect(text, ai_config=AIConfig()) instead. The heuristic tiers are best suited to structured forms and templated text where titles or context keywords are consistently present.

Names, locations, organisations (AI mode only)

from zero_harm_ai_detectors import AIConfig

result = detect("Dr. Jane Wilson visited Stanford University in Palo Alto", ai_config=AIConfig())

for d in result.detections:
    print(f"{d.type}: {d.text} ({d.confidence:.0%})")
# PERSON: Dr. Jane Wilson (92%)
# ORGANIZATION: Stanford University (88%)
# LOCATION: Palo Alto (85%)

Secrets Detection

Three-tier detection, always uses heuristic matching (95%+ accuracy):

from zero_harm_ai_detectors import detect_secrets_heuristic

secrets = detect_secrets_heuristic("""
    OPENAI_KEY = "sk-1234567890abcdef1234567890abcdef"
    AWS_KEY    = "AKIAIOSFODNN7EXAMPLE"
    GH_TOKEN   = "ghp_ABCDEFGHIJKLMNOPQRSTUVWXYZabcd1234"
""")

Supported patterns: OpenAI · AWS (access + secret) · GitHub · Stripe · Google · Slack · Twilio · SendGrid · npm · PyPI · Anthropic · Generic secrets (context + entropy)

Harmful Content Detection

from zero_harm_ai_detectors import DetectTarget

result = detect(text, targets=DetectTarget.HARMFUL)

print(result.harmful)         # True / False
print(result.severity)        # "none" | "low" | "medium" | "high"
print(result.harmful_scores)  # {"insult": 0.6, "threat_phrases": 0.8, ...}

Severity rules:

Condition	Severity
identity_hate found	high
Explicit threat phrase	high
2+ threat words or 6+ total matches	high
1 threat word or 4+ total matches	medium
2+ obscene terms	medium
Any other match	low

Redaction Strategies

text = "Email: [e-mail]"

detect(text, redaction_strategy="token")     # Email: [REDACTED_EMAIL]
detect(text, redaction_strategy="mask_all")  # Email: ****************
detect(text, redaction_strategy="mask_last4")# Email: ************.com
detect(text, redaction_strategy="hash")      # Email: [HASH:a1b2c3d4e5f6]

Selective Detection

from zero_harm_ai_detectors import DetectTarget

# PII only
result = detect(text, targets=DetectTarget.PII)

# Secrets only
result = detect(text, targets=DetectTarget.SECRET)

# PII + Secrets
result = detect(text, targets=DetectTarget.PII | DetectTarget.SECRET)

AI Configuration

from zero_harm_ai_detectors import detect, AIConfig

config = AIConfig(
    backend="transformers",  # higher NER accuracy
    ner_model="dslim/bert-base-NER",
    ner_threshold=0.8,   # higher = fewer false positives
    harmful_threshold=0.6,
    device="cuda",       # or "cpu"
)

result = detect(text, ai_config=config)

SpaCy NER option:

config = AIConfig(
    backend="spacy",
    spacy_model="en_core_web_sm",
)
result = detect(text, ai_config=config)

Harmful detection behavior in AI path:

• Uses transformer harmful model when available.
• If unavailable/fails, falls back to heuristic harmful detection.
• Non-default harmful_model values emit a warning.

Architecture

┌────────────────────────────────────────────────────────────┐
│      detect(text, targets=..., ai_config=...)              │
├──────────────────────────┬─────────────────────────────────┤
│ default (no ai_config)   │ ai_config provided              │
│ heuristic_detectors.py   │ ai_detectors.py                 │
│                          │ - structured PII via heuristic  │
│ - structured PII         │ - NER via transformers or spacy │
│ - secrets via heuristic  │ - harmful via transformer       │
│ - harmful via heuristic  │   with heuristic fallback       │
└──────────────────────────┴─────────────────────────────────┘
                         ↓
                    DetectionResult

Integration Example — GitHub App PR Scanner

from zero_harm_ai_detectors import detect, AI_AVAILABLE, AIConfig, DetectTarget

def scan_pr_diff(diff: str, is_paid_user: bool) -> dict:
    ai_config = AIConfig() if (is_paid_user and AI_AVAILABLE) else None

    result = detect(
        diff,
        targets=DetectTarget.PII | DetectTarget.SECRET,
        ai_config=ai_config,
    )

    blocking_types = {"API_KEY", "SECRET", "SSN", "CREDIT_CARD"}
    return {
        "has_issues":   len(result.detections) > 0,
        "should_block": any(d.type in blocking_types for d in result.detections),
        "detections":   result.to_dict()["detections"],
    }

Performance

Text length	Default (heuristic)	AI (ai_config)
~50 chars	1–2ms	50–100ms
~500 chars	2–3ms	100–150ms
~5 000 chars	3–5ms	150–200ms
Throughput	~500/sec	~5–10/sec

Testing

# All tests
pytest tests/ -v

# With coverage
pytest tests/ --cov=zero_harm_ai_detectors

# Skip AI tests (if dependencies not installed)
pytest tests/test_core_and_heuristic.py -v

Licensing

The open-source core of this project is licensed under MIT.

• Open-source license: https://scalapps.com/docs/LICENSE
• Commercial licensing terms: https://scalapps.com/docs/LICENSE-COMMERCIAL

Commercial licensing is available for organizations that need contractual terms such as support/SLA, warranty, indemnity, or other enterprise conditions. Contact: [e-mail]

Links

• PyPI: https://pypi.org/project/zero-harm-ai-detectors/
• Issues: https://scalapps.com