svgap 0.3.0a2

Production-readiness evaluation for AI-generated RTL

SV-Gap

Make the gap between “passes the benchmark” and “reviewable by a chip-design team” explicit.

SV-Gap is an open evaluation layer for AI-generated digital RTL. It preserves the functional result produced by a research workflow, adds declared clock and reset intent plus structural evidence, and reports which configured production questions are answered, failed, or still unknown.

Supply RTL and evaluation evidence. SV-Gap tells you what that evidence establishes, what it contradicts, what remains unresolved, and what evidence would reduce the uncertainty.

SV-Gap begins with clock-domain crossing (CDC) and reset-domain crossing (RDC). It is early research software, not a replacement for commercial signoff.

Who this is for

You are…	Use SV-Gap to…	Opportunities to build
A frontier-model researcher	Test whether models produce evidence-complete RTL, diagnose missing evidence, and repair structural findings without regression	Model baselines, agent policies, diagnosis methods, repair methods
An RTL benchmark maintainer	Learn whether tasks carry enough clock/reset intent to evaluate production-oriented properties	Benchmark adapters, intent-bearing contracts, result submissions
A chip-design AI builder	Add a structural evidence gate after generation and functional testing	Pipeline integrations, CI adapters, evidence dashboards
An RTL or verification engineer	Make the reason a functionally passing candidate is blocked explicit	Minimal counterexamples, disputed findings, independent evidence
An EDA researcher or tool author	Compare structural backends under a shared evidence contract	Open checker backends, differential studies, digital RTL taskpacks
An evaluation researcher	Study when offline success does not identify deployment validity	Abstention semantics, construct-validity studies, cross-domain methods

See the gap in two minutes

Install Python 3.11+, Yosys, and Icarus Verilog. On macOS:

brew install yosys icarus-verilog
python3 -m venv .venv
.venv/bin/python -m pip install svgap==0.3.0a2
.venv/bin/svgap demo

The demo produces one functional result and two different structural outcomes:

candidate  functional  structural  findings
safe       pass        pass        none
unsafe     pass        fail        REF-RDC-001

Both implementations satisfy the supplied functional test. Declared reset-release intent separates them structurally. Preserve the complete reproducer with svgap demo --output demo-output.

The same workflow is available in the open-tool container:

docker run --rm ghcr.io/shsridhar-beep/svgap:v0.3.0-alpha.2 demo

Use SV-Gap on your own RTL

svgap init path/to/design.sv \
  --top top \
  --candidate-id experiment-001 \
  --output path/to/manifest.toml

svgap validate path/to/manifest.toml
svgap check path/to/manifest.toml
svgap explain path/to/build/report.json

init deliberately does not guess intent. validate exposes missing functional, clock, reset, and relationship evidence before execution. explain translates the resulting report into answered, failed, and unanswered questions.

Follow the complete bring-your-own-RTL tutorial, including an executable manifest and imported-result path.

Research with SV-Gap

Generation

Ask whether a model can produce RTL with passing and determinate functional and structural evidence—not merely code that passes a testbench.

Diagnosis

Ask whether a model distinguishes evidence that establishes a property, evidence that contradicts it, and a question that remains unresolved.

Repair

Ask whether a model removes a structural finding while preserving functional acceptance, checker coverage, candidate identity, and freedom from new rule regressions.

The public challenges/v0.1 contract defines all three tracks. Profiles remain multidimensional rather than hiding failure modes in a single score. See the baseline registry to reproduce or submit a result.

Current evidence

• Four controlled safe/unsafe CDC/RDC witness pairs pass the same functional tests and are separated by declared structural rules.
• A frozen 72-call reset-release study contains 57 functional passes; at least 14 contain the declared raw-reset pattern.
• A heuristic inventory covers 508 public RTL-generation tasks across VerilogEval, RTLLM, and CVDP.
• A small exploratory frontier-model baseline exposes a crossed failure mode: one configuration preserves epistemic uncertainty but misses the repair; another repairs the configured finding but overstates what the diagnosis evidence establishes.

These are an executable existence result, a taskpack-conditional demonstration, and a heuristic inventory. They are not a population defect estimate or a model ranking.

Controlled result · Reset result · Benchmark audit · Compact research note

Build with us

SV-Gap is shared research infrastructure. You do not need to agree with the reference checker to contribute.

Add a taskpack

Create public digital RTL tasks with explicit clock/reset intent, executable functional evidence, and calibrated references.

Add a checker backend

Implement one operation:

check(manifest) -> CheckResult

Backends preserve pass, fail, unknown, and tool_error. Missing intent or unsupported syntax must never become a pass. See the backend SDK.

Add a benchmark adapter

Import an existing benchmark’s functional result while binding it cryptographically to the evaluated RTL.

Contribute model evidence

Submit generation, diagnosis, or repair results using the public registry contract. Negative results, abstentions, and disagreements are welcome.

Challenge the oracle

Contribute a minimal false positive, false negative, competing backend result, or expert adjudication. Disagreement is evidence, not a project failure.

Good first issues · Contribution guide · Discussions · Roadmap

High-value open research problems

1. Which production-oriented properties are absent from current RTL benchmarks?
2. Can models recognize missing intent instead of inventing it?
3. Does structural feedback improve repair without creating functional regressions?
4. How stable are results across independent open checker backends?
5. What evidence package makes generated RTL reviewable downstream?
6. How should multidimensional evidence be compared without hiding unknown states?

Scope and limitations

SV-Gap covers digital RTL and digital verification. See the explicit scope boundary.

The built-in Yosys backend is deliberately narrow and is not signoff-grade. A structural pass means only that the configured backend emitted no failing finding within its declared coverage. It is not proof of silicon safety.

The generic trace-adjudication scaffold uses prerecorded fixtures. The real reset-release perturbation instrumenter remains unimplemented pending patent and employer review.

See methodology, architecture, and limitations before interpreting claim-bearing results.

Install and integrate

From PyPI

python3 -m pip install svgap==0.3.0a2
svgap doctor

From source

git clone https://github.com/shsridhar-beep/svgap
cd svgap
python3 -m venv .venv
.venv/bin/python -m pip install -e .
.venv/bin/svgap doctor

GitHub Actions

Use the reusable action in .github/actions/svgap to attach normalized structural evidence to an existing RTL pipeline.

Requirements

• Python 3.11–3.13
• Yosys and Icarus Verilog for the built-in backend
• Only open-source tools are assumed by default

Community and citation

Ask integration questions and propose research directions in GitHub Discussions. Use the issue templates for reproducible bugs, taskpacks, backends, and false results. See GOVERNANCE.md, SECURITY.md, and SUPPORT.md.

Cite the exact GitHub release used. Archival DOI metadata is being replaced after a provenance-label redaction; a new DOI will be recorded only after the sanitized archive is publicly verified.

Apache-2.0. External tools and imported datasets retain their own licenses.