SV-Gap

Make the gap between “passes the benchmark” and “reviewable by a chip-design team” explicit.

SV-Gap is an open evaluation layer for AI-generated digital RTL: LLM-written Verilog and SystemVerilog scored today by functional benchmarks. It preserves the functional result, adds declared design intent for clock-domain crossing (CDC), reset-domain crossing (RDC), metastability containment, X behavior, power-on state, temporal/protocol behavior, and reference equivalence, and reports the evidence a hardware verification or signoff team can review: which production questions are answered, failed, or still unknown.

Supply RTL and evaluation evidence. Receive a reviewable account of what that evidence establishes, what it contradicts, and what evidence would resolve the remaining uncertainty.

Choose your first step

Do not send proprietary RTL or confidential constraints through GitHub or email. A public or synthetic artifact is enough for the first experiment.

What the demo proves

candidate  functional  structural  finding
safe       pass        pass        none
unsafe     pass        fail        REF-RDC-001

Both implementations pass the supplied functional test. Declared reset-release intent and configured structural evidence distinguish them. This is an executable existence result: it is not a defect-rate estimate, certification, or silicon signoff.

Supported today

Read the full methodology, limitations, and scope boundary before making claim-bearing use of a profile.

Trust and security boundary

• SV-Gap runs locally and performs no telemetry or artifact uploads. A model generator command supplied by the user may contact its configured provider.
• Generated RTL and functional commands are untrusted input. Do not evaluate them on a workstation containing credentials or sensitive source trees.
• The recommended two-stage workflow generates in the credentialed environment and evaluates saved responses in a network-disabled, read-only container.
• Only open-source runtime tools are assumed by default. Tool versions, provenance, unknowns, and errors remain in the evidence record.
• SV-Gap is evidence infrastructure, not a replacement for organizational review, commercial verification, or signoff.
• GitHub's automatic contributor graph reflects commit authors, including disclosed AI assistance; it is not a roster of verified human researchers. Maintainer accountability and accepted contributions are documented in CONTRIBUTORS.md.

Follow the isolated evaluation recipe for model or contributor outputs you have not reviewed.

Run locally

The browser result above is the fastest path from a new visit to the research idea. The container includes the complete open RTL toolchain, but its first large image pull depends on network speed; the demo itself runs in under two minutes once the image is cached:

docker run --rm ghcr.io/shsridhar-beep/svgap:v0.3.0-alpha.13 demo

docker run --rm ghcr.io/shsridhar-beep/svgap:v0.3.0-alpha.13 demo -- scenario comb-crossing

For a native macOS installation:

brew install yosys icarus-verilog
python3 -m venv .venv
.venv/bin/python -m pip install svgap==0.3.0a13
.venv/bin/svgap doctor
.venv/bin/svgap study quickstart --output my-first-svgap-study

quickstart evaluates a clearly labelled bundled unsafe fixture, writes a portable HTML evidence profile, and prints the exact report to pass to svgap explain. It teaches the workflow; it is not a model result. Run .venv/bin/svgap demo for the paired safe/unsafe executable witness.

Ubuntu, Debian, CI, and troubleshooting instructions are in Linux installation and doctor checks. If doctor finds a missing prerequisite, it prints the installation command or container fallback rather than leaving the user at a missing-tool report.

Evaluate a model or existing RTL

Any model harness can participate: read a prompt from stdin and write the model response to stdout.

svgap study run reset-release-v0.2 \
  --command "python3 my_generate.py" \
  --label my-model-a \
  --smoke \
  --output my-first-svgap-study

The run produces a portable summary, evidence-file list, reports, and static HTML profile. Replace --smoke with --full for the frozen eight-task, three-sample protocol. See Evaluate your model.

For existing RTL, use svgap init, validate, check, and explain; the bring-your-own-RTL tutorial includes an executable manifest and imported-result path. Python integrations can call svgap.evaluate(manifest); see the Python API.

Current evidence

• Twenty-two controlled witness pairs have identical functional outcomes and different configured oracle outcomes. Seventeen exercise structural rules; five expose protocol/temporal and synthesis-equivalence blind spots. Twelve of the structural pairs expand detector coverage; they are newly added calibration fixtures without independent review yet, not new prevalence evidence.
• A frozen 72-call reset-release study contains 57 functional passes; at least 14 contain the declared raw-reset pattern.
• A frozen 48-call follow-up across protocol, temporal, and synthesis-equivalence tasks contains 41 functional passes. Forty have a determinate contributing-oracle result; 11 fail the configured protocol or temporal property. The equivalence stratum is a bounded 0/12 null.
• Default Verilator and Verible lint configurations emit no RDC-specific diagnostic for any of those 14 functionally passing reset-gap candidates; Verilator likewise identifies none of the 11 expanded-study failures. Schema v2 retains lint beside the contributing evidence without conflation.
• A heuristic inventory covers 508 public RTL-generation tasks across VerilogEval, RTLLM, and CVDP.
• A separate audit of those 508 tasks inventories stated power-on intent and recognizable unknown-initial-state scoring.
• A temporal/equivalence audit finds a conservative lower bound of 98 tasks with explicit temporal contracts but no recognizable property/formal score, and 16 tasks requiring equivalence that synthesize without comparing against their supplied original RTL.
• Two reproducible open-weights profiles demonstrate the public submission path; they are maintainer-produced anchors, not independent replications.

Controlled result · Reset result · RTL lint baseline · Benchmark audit · Power-on audit · Temporal/equivalence audit · Expanded contract result · Evidence profiles · Compact research note

These are bounded existence, taskpack-conditional, and heuristic results. They are not a population defect estimate, general model ranking, or signoff claim.

Collaborate

The preferred entry point is one question a functional RTL evaluation leaves unanswered. A 30-minute scoping call should end with a bounded qualification experiment, explicit claim boundary, and go/revise/stop decision.

• Research-call intake
• Report a Harbor run or setup blocker
• Current research collaboration pulse
• Private maintainer email
• Design-partner workflow
• One-page experiment contract
• Replication and co-design discussion

Joining a call is not contributor status. Named credit follows accepted, attributable protocol design, redistributable evidence, task design, analysis, validation, documentation, or code. See Contributors and Contributing.

Extend and integrate

• Submit a result
• Write a checker backend
• Integrate an existing benchmark
• Use the GitHub Action or container
• Good first issues
• Roadmap

Project status and citation

SV-Gap is early research software maintained by Shraddha S, who is accountable for project direction, incorporated changes, research claims, and releases. Material AI development assistance is disclosed in CONTRIBUTORS.md.

Cite the exact release used. The independently fetched and verified alpha.13 archive is doi:10.5281/zenodo.22087102. Its extracted tree matches the tagged release file-for-file and byte-for-byte. The all-versions DOI always resolves to the latest archived release.

Apache-2.0. External tools and imported datasets retain their own licenses.