VerifySignal

AI writes the validation. A deterministic runtime proves it.

AI multiplied your features. Validating each one is still manual. VerifySignal turns a product flow (login, checkout, onboarding) into an automatic, repeatable validation with evidence, starting from your repository or a live product URL when source access is unavailable.

Your coding agent (Claude Code or Codex) authors the validation and grounds it against your live app. The VerifySignal runtime runs it deterministically and leaves a report you can audit. No model decides pass/fail. Same input, same verdict, every time.

This repo is the open half (Apache-2.0): the verifysignal CLI, the .verifysignal/ workspace, and the agent skills. The runtime (VerifySignal Core) is a signed download, unlocked with a free email. No account.

Quickstart

You need Python 3.11+ and Node 24+. Install the CLI:

uv tool install verifysignal-spec        # or: pipx install verifysignal-spec

Set up your project and check readiness:

cd your-project
verifysignal init --here --integration claude   # or: codex
verifysignal check

Then run the whole flow from your agent, in one line:

/verifysignal "Validate that a user can sign in against https://staging.example.com"

The agent drafts the use case from your source or synthesized browser-first product understanding, asks you to confirm the test target, grounds its selectors against that app, validates, runs, and repairs. It stops only for real unknowns, missing credentials, or a write it should not make on its own. Credentials can come from the current environment or from an explicit, owner-only test environment file that VerifySignal prepares and Git-ignores.

No source access?

To begin without source access, initialize an empty local engagement directory and provide the live target conversationally or with --url:

/verifysignal-understand --url https://staging.example.com

The agent opens a visible browser, lets you authenticate directly, maps a bounded read-safe scope, and persists only synthesized product signals and candidate journeys. It does not persist DOM snapshots, screenshots, cookies, storage state, form values, or URL query values. See Browser-first understanding.

uv tool install verifysignal-spec --from git+https://github.com/RigelRise/verifysignal-spec.git

What you get

• AI authors, you approve. The agent drafts and repairs. Nothing runs until it is validated.
• Deterministic execution. A fixed action set, no inference. Green means the same thing every time.
• Evidence, not a checkmark. Every run leaves a report, screenshots, and a network log you can review and share.
• Write safety. Flows that mutate data declare a side-effect policy, are watched at run time, and gate their own reruns.

How it works

  repo / engagement   verifysignal CLI          VerifySignal Core         evidence
  .verifysignal/  ->  open · Apache-2.0      ->  signed runtime       ->  report.md
  use cases           authoring · gates          deterministic run        report.json
  skills · state      workflow · repair          no model at runtime      screenshots
                                                                           network log

The CLI owns authoring, gates, workflow state, and repair. Core owns execution. They meet at one versioned JSON contract, nothing else. Every command speaks --json, and exit codes are stable (0 ok, 2 validation, 3 core, 4 approval, 5 input).

What a run leaves behind

.verifysignal/runs/login/<run-id>/
  report.md            human-readable, gate by gate
  report.json          machine-readable (qa-report/v1)
  browser/screenshots  captured evidence
  browser/network.ndjson

report.md says what passed, what failed and why, and links each claim to its evidence. See examples/ for two full samples, including a write flow.

Open-core

Authoring and safety are open and forkable. Execution stays signed so "green" means the same thing for everyone. See GOVERNANCE.md.

Safety

• Secrets stay out of managed artifacts. Credentials resolve at run time from the current environment or an explicit owner-only, Git-ignored test file. Values are never copied into use cases, reports, or command output.
• Writes are declared and watched. A write flow states what it may touch, is checked at run time, and needs approval to rerun after a commit.
• The runtime wins. If the agent and the run disagree, the run result stands. Agents stop and ask instead of inventing selectors.

What it is not

• Not a replacement for your unit tests or CI. It makes manual product validation repeatable.
• Not an agent that decides pass/fail. Execution is a fixed action set; the agent only authors and repairs.
• Not a service. Everything it manages lives in your repository or local engagement directory under .verifysignal/.

CLI

Docs and community

• Documentation, Installation, Golden Path, and Browser-first understanding
• Contributing, Governance, Roadmap
• Issues, Discussions
• Security: please use private reporting (SECURITY.md)

Development

python -m pip install -e ".[dev]"
python -m pytest

Needs Python 3.11+, and a browser for the cross-repo dogfood. To run against the toolchain CI declares instead of whatever your machine has:

scripts/verify-docker.sh

That mounts the sibling checkouts too, so the cross-repo tests actually run. Without them they skip, which reads like a pass. It takes any pytest arguments: scripts/verify-docker.sh tests/unit -q.

The cross-repo gate (product-truth.yml in Core) additionally pins every repository path. A suite that behaves differently under a pin than under a scan is environment-dependent, so reproduce that configuration before relying on a green run — the paths are container-side, where /w is the mounted parent directory:

VERIFYSIGNAL_CORE_DIR=/w/<core-dirname> \
VERIFYSIGNAL_SPEC_DIR=/w/<spec-dirname> \
VERIFYSIGNAL_BACKEND_DIR=/w/<backend-dirname> scripts/verify-docker.sh

The suite runs against a full fake Core, so you can build and test everything without the runtime.

License

Apache-2.0