mafhoom 0.1.0

Machine-Assisted Formalization of Hardware Objectives, Obligations, and Mandates

MAFHOOM

Machine-Assisted Formalization of Hardware Objectives, Obligations, and Mandates

MAFHOOM is an NLP compiler for hardware specifications. It accepts natural-language hardware requirements and compiles them into a typed, machine-readable Hardware Requirement Intermediate Representation (HRIR), on which deterministic analyses and artifact-generation backends operate.

Quick Start

Installation

pip install mafhoom                # From PyPI
pip install mafhoom[embeddings]    # With embedding-based duplicate detection
pip install -e ".[dev]"            # Development install from source

Offline Demo (no API key required)

mafhoom demo

This runs the full pipeline on a FIFO controller specification, producing:

• Extracted requirements and HRIR objects
• Ambiguity, contradiction, duplicate, and completeness findings
• SystemVerilog Assertion candidates
• Verification test plan entries
• Traceability matrix and coverage report

With Claude API

export ANTHROPIC_API_KEY="your-key-here"
mafhoom models                          # List available models
mafhoom compile examples/fifo_spec.md   # Full pipeline

Hybrid Mode (rules + Claude FP filtering)

mafhoom compile examples/fifo_spec.md --mode hybrid

Uses rule-based extraction and deterministic analysis, then sends adjudicable findings (contradictions, ambiguities, near-duplicates) to Claude for true/false positive classification.

CLI Commands

mafhoom version                         # Show version
mafhoom models                          # List Claude models
mafhoom ingest PATH                     # Ingest a document
mafhoom extract PATH                    # Extract requirements
mafhoom compile PATH [--output DIR]     # Full pipeline
mafhoom compile-project P1 P2 ...       # Multi-doc compilation
mafhoom analyze PATH                    # Run analysis passes
mafhoom export-hrir PATH --fmt csv      # Export HRIR (json/yaml/csv)
mafhoom generate-assertions PATH        # Generate SVA from HRIR
mafhoom generate-tests PATH             # Generate test plans
mafhoom html-report PATH                # Interactive HTML report
mafhoom diff OLD_PATH NEW_PATH          # Change impact analysis
mafhoom demo                            # Offline demonstration
mafhoom report RUN_DIR                  # Report from a run
mafhoom evaluate SPEC [--gold GOLD]     # Evaluate across systems
mafhoom run-experiment SPEC SYSTEM      # Run single system
mafhoom validate-dataset PATH           # Validate dataset

Pipeline Architecture

Document ingestion
        |
Layout-aware text segmentation
        |
Candidate requirement detection (rule-based / Claude)
        |
Atomic requirement decomposition
        |
Semantic parsing -> HRIR construction
        |
Normalization
        |
Deterministic compiler passes (8 passes)
        |
[Hybrid mode] Claude finding filter (FP removal)
        |
Artifact generation backends
        |
Evaluation and reporting

Compiler Passes

1. Schema Validation - Invalid fields, ranges, references
2. Ambiguity Detection - Vague terms, unclear references, with safe pronoun context filtering
3. Completeness - Missing information (timeout values, reset polarity, etc.) with spec-wide context
4. Duplicate Detection - Exact, normalized, and near-duplicate requirements
5. Contradiction Detection - Numeric, timing, modality, and polarity conflicts with entity overlap gating
6. Timing Consistency - Impossible bounds, conflicting deadlines
7. State Consistency - Unreachable states, missing transitions
8. Dependency Analysis - Requirement graph, decomposition siblings, shared-target edges

Supported Formats

• Markdown (.md)
• Plain text (.txt)
• PDF (.pdf)
• DOCX (.docx)
• JSON (.json)
• JSONL (.jsonl)

Running Tests

# Unit tests (no API key needed)
python3 -m pytest tests/unit/ -v

# Integration tests (no API key needed)
python3 -m pytest tests/integration/ -v

# All offline tests (225 tests)
python3 -m pytest tests/ -v -k "not Live"

# API tests (requires ANTHROPIC_API_KEY)
python3 -m pytest tests/api/ -v -m api

Benchmark Results

Pipeline Output (10 clean specs, 549 requirements)

Spec	Reqs	HRIR	Findings
ADC Controller	66	97	16
APB	53	83	12
AXI-Lite	58	96	24
Cache Controller	69	102	15
DMA	62	105	15
FIFO	28	35	15
I2C	55	106	20
Interrupt Controller	56	83	10
SPI	51	91	10
UART	51	81	19
Total	549	879	156

Seeded Defect Detection (43 defects across 8 evaluated specs)

• Recall: 100% (43/43 seeded defects detected)
• Total FPs: 80 across evaluated specs (most are valid findings beyond the seeded defects)
• Clean spec F1: 50.0% - 66.7% | Dirty spec F1: 43.5% - 52.6%

Dirty Variant Results

Spec	Reqs	HRIR	Findings
FIFO (dirty)	49	61	18
UART (dirty)	53	71	17
Cache Controller (dirty)	66	93	14

Configuration

Configuration files are in configs/. Key settings:

api:
  model: ""              # Claude model (or set MAFHOOM_MODEL env var)
  temperature: 0.0       # Low for structured extraction
  cache_read: true       # Use cached API responses
  cache_write: true      # Save API responses to cache

analysis:
  duplicate_threshold: 0.85
  contradiction_candidate_threshold: 0.6

experiment:
  random_seed: 42
  output_dir: "results"

Project Structure

mafhoom/          # Core package
  models/         # HRIR schema, enums, findings
  ingestion/      # Document parsers
  extraction/     # Requirement detection, decomposition, semantic parsing
  normalization/  # Signal, timing, unit normalization
  ontology/       # Hardware ontology (YAML)
  passes/         # Deterministic compiler passes + hybrid filter
  backends/       # SVA, test plan, traceability generation
  api/            # Claude API client with caching
  evaluation/     # Metrics and evaluation
  experiments/    # Experiment runner
  pipeline.py     # Main pipeline orchestration
  cli.py          # CLI interface
  config.py       # Configuration management
prompts/          # Versioned Claude prompts
configs/          # YAML configuration files
data/             # Datasets, annotations, and gold standards
examples/         # Example specifications
scripts/          # Evaluation and benchmark scripts
tests/            # Unit, integration, API tests (225 tests)
docs/             # Documentation

Documentation

• Architecture
• HRIR Specification
• Experiment Design
• API Usage
• Reproducibility
• Annotation Guidelines
• Limitations

License

MIT