Oneport Testgap

Diff-scoped test-gap analysis that doesn't stop at a number.

Codecov and diff-cover tell you "71% patch coverage" and leave. Qodo generates tests and hopes they work. Testgap does the whole job:

1. Finds exactly which changed lines have zero test coverage (deterministic: git diff ∩ pytest --cov, never an LLM guess),
2. Ranks each gap by risk — payment, auth, and data-write logic outrank logging — with a one-sentence explanation of why it's risky,
3. Generates pytest tests for the riskiest gaps and — the part nobody else does — executes every generated test before showing it to you. Tests that fail, or that don't actually cover the gap lines they were written for, are discarded. What lands in tests/generated/ has already passed.

$ oneport-testgap analyze --staged --generate 3

Oneport Testgap — staged changes
1 file(s) analyzed · 6 changed line(s) · 4 with zero coverage

[CRITICAL] payment.py:4-7  (charge)
           Fee arithmetic and the negative-amount guard ship completely untested.

Generated tests (only VERIFIED tests are written to disk):
  VERIFIED  tests/generated/test_gap_charge.py — passed, covers 4/4 gap line(s)
            of payment.py [1 attempt(s)]

RESULT: critical-risk changed lines have zero coverage — exit 1 (CI gate).

Install

pip install oneport-testgap

Requires pytest + pytest-cov in the project you're analyzing (Testgap no-ops gracefully with a clear message if they're missing, or you can hand it an existing report with --coverage-file coverage.xml).

0-BYOK — no API keys, ever. The AI layer (risk ranking + test generation) runs on OnePort's own managed models and is metered against your account's token balance; you never bring or expose a model key. Get a token at oneport.co.in → Console → API Tokens, then log in once:

onp account login <op_live_...>

Usage

oneport-testgap analyze                    # staged changes (default)
oneport-testgap analyze --head             # the last commit
oneport-testgap analyze https://github.com/org/repo/pull/42   # a PR (run on a checkout of its branch)

oneport-testgap analyze --generate 3       # write verified tests for the top 3 gaps
oneport-testgap analyze --min-risk high    # only DISPLAY high/critical gaps
oneport-testgap analyze --fail-on high     # FAIL CI on high+critical gaps (default: critical)
oneport-testgap analyze --format json      # machine-readable, for pipelines
oneport-testgap analyze --format sarif     # GitHub Code Scanning
oneport-testgap analyze --coverage-file coverage.xml   # reuse an existing report
oneport-testgap analyze <PR_URL> --post    # comment the gap report on the PR

CI gate

Exit codes are the contract:

--min-risk only filters the report; the exit code is controlled by --fail-on (default critical — raise to high/medium/low to be stricter). The two are independent, and waived gaps never block. A ready-made GitHub Action lives in examples/workflows/testgap.yml.

Waivers — accept a known gap on the record

Some code is untested on purpose (a generated migration, a __main__ shim). Approve it narrowly instead of turning the gate off, in .oneport/testgap-waivers.yml:

waivers:
  - file: "**/migrations/*.py"     # glob on the gap's file (path or basename)
    reason: "generated migrations are exercised in staging"
    expires: "2026-12-31"
  - file: "app/cli.py"
    function: "main"               # optional — restrict to one function
    risk: high                     # optional — cap the risk this waiver will accept
    reason: "entrypoint smoke-tested by the e2e suite"

A waived gap is still shown (marked [WAIVED]) and still exported to SARIF as a suppressions entry, but it never trips --fail-on. Expired waivers are ignored, so the gap re-blocks — an approval can't outlive its reason.

How it works (and what the LLM is never allowed to do)

git diff ──► changed lines ─┐
                            ├─► gaps = changed ∩ uncovered   ← deterministic, always
pytest --cov ► coverage.xml ┘
                                    │
                                    ▼
                        LLM: rank by risk + explain why      ← judgment only
                                    │  --generate N
                                    ▼
                        LLM writes a pytest file
                                    │
                        temp file → real pytest run → real coverage check
                                    │
                       pass + gap lines covered?  ──no──► discarded (one repair
                                    │yes                   round, then gone)
                                    ▼
                        tests/generated/  (review it, commit it yourself)

The model never computes coverage facts. It sees gaps that are already proven uncovered and only decides how much they matter and how to test them. Its tests earn their place by executing — a generated test you see is a test that already ran green and demonstrably covered the lines it targets.

vs. the alternatives

Configuration

Testgap reads the suite-wide .oneportrc (same file as oneport-review). Top-level keys are shared; a testgap: section overrides them for this tool:

model: gemini-flash-latest
ignore_paths:
  - migrations/**
  - "**/*_pb2.py"

testgap:
  fail_on: critical         # risk level that fails CI (critical | high | medium | low)
  output:
    min_risk: medium        # what to DISPLAY — never affects the exit code
    format: inline          # inline | json | sarif
  generate:
    repair_rounds: 1        # failed test gets its pytest output fed back once
    test_timeout: 120
    output_dir: tests/generated
  coverage:
    pytest_args: ["-m", "not slow"]
    timeout: 600

Team guidelines in .oneport/guidelines.md (the file oneport learn writes) are injected into both the ranking and generation prompts — testing rules like "all payment code needs edge-case tests" shift the ranking and shape the generated tests.

Privacy

Coverage runs and gap detection happen entirely on your machine — your code never leaves it. Only when the AI layer is used are the already-built ranking and generation prompts (the gap snippets, not your whole repo) sent to the OnePort backend, which calls the managed model with its server-side key, meters your tokens, and returns the result. No telemetry, nothing retained. See PRIVACY.md.

Part of the Oneport suite

Testgap is one gate in Oneport, the AI pre-ship check: oneport-review (AI code review) · oneport-testgap (this) · more coming.