quelltest

Your docstrings say what your code should do. Quelltest proves it.

Quelltest reads your Python docstrings, Pydantic models, and PySpark schemas — extracts every testable requirement — generates a verified pytest test for each gap — and writes it to disk only after proving it catches real violations.

v1.0.0 — infrastructure-aware verified testing: QuellGraph SQLite code-intelligence graph, ephemeral Docker containers, and per-test confidence scores.

No LLM key required. No code leaves your machine.

pip install quelltest
quell check src/ --fix --no-llm

# v1.0.0: build code-intelligence graph, run with auto-containers
quell graph build src/
quell check src/ --with-containers --fix --min-confidence 70

What "verified" means

Every generated test goes through two phases before it's written:

1. Phase 1 — pass on correct code. The test runs against your original source. If it fails here, something is wrong with the generated test itself. Rejected.
2. Phase 2 — fail on violated code. Quelltest injects a targeted violation (comments out the raise, weakens the Field bound, flips nullable=False). The test must fail. If it passes, it proved nothing. Rejected.

Only tests that survive both phases are written. This is the moat between Quelltest and every other test generation tool.

Use cases

Docstrings with Raises: / Returns:

def process_payment(amount: float, currency: str) -> PaymentResponse:
    """
    Process a payment.

    Args:
        amount: Payment amount in the specified currency.
        currency: ISO 4217 currency code.

    Returns:
        PaymentResponse with a unique transaction_id.

    Raises:
        ValueError: If amount is <= 0.
        ValueError: If currency is not supported.
    """

quell check src/payments.py --fix --no-llm
# → writes tests/test_payments.py with 2 verified MUST_RAISE tests

Generated test (simplified):

def test_process_payment_raises_valueerror_amount():
    """MUST_RAISE ValueError — auto-generated by Quelltest 0.6.9."""
    with pytest.raises(ValueError):
        process_payment(amount=0, currency="USD")

This test is verified: it passes on the original process_payment and fails when the raise ValueError is commented out.

Pydantic v2 Field constraints

from pydantic import BaseModel, Field
from typing import Literal

class PaymentRequest(BaseModel):
    amount: float = Field(gt=0, le=10_000)
    currency: Literal["USD", "EUR", "GBP", "INR"]
    idempotency_key: str = Field(min_length=8, max_length=64)

quell check src/models.py --fix --no-llm
# → BOUNDARY test for amount (>0, ≤10000)
# → ENUM_VALID test for currency (must be one of 4 values)
# → BOUNDARY test for idempotency_key (length 8–64)

PySpark schemas

from pyspark.sql.types import StructType, StructField, DoubleType, StringType

payment_schema = StructType([
    StructField("amount", DoubleType(), nullable=False),
    StructField("currency", StringType(), nullable=False),
    StructField("transaction_id", StringType(), nullable=True),
])

quell check src/schemas.py --fix --no-llm
# → NOT_NULL test for amount
# → NOT_NULL test for currency
# → TYPE_CHECK test for schema structure

Requirement coverage in CI

# Fail the pipeline if requirement coverage drops below 80%
quell check src/ --ci --threshold 0.80

# Post a gap report as a PR comment
quell pr 42 --comment

GitHub Actions workflow (generated by quell install --pr):

- name: Quelltest requirement coverage
  run: |
    pip install quelltest
    quell check src/ --ci --threshold 0.80

Reproduce a bug from plain English

quell reproduce "payment function accepts zero amount and does nothing"
# → finds the relevant function
# → generates a failing test that proves the bug
# → test is verified against the current (buggy) code

All commands

How it works

Spec readers (docstring, Pydantic, PySpark)
        │
        ▼
  list[Requirement]
        │
        ▼
Coverage checker (AST scan of test files — no test execution)
        │
        ▼
Rule engine (deterministic, no LLM) → covers ~75% of requirements
LLM engine (fallback, only if configured) → covers complex cases
        │
        ▼
Verification engine — THE MOAT
  Phase 1: test passes on original code
  Phase 2: test fails on violated code
  Always restores source in finally block
        │
        ▼
Writer (libcst injection — preserves formatting)
  Backup before write. Validate CST. Restore on failure.

Constraint kinds

Why not just use an LLM?

LLM-generated tests match current behavior, not stated requirements. If your code has a bug (accepts zero amounts), an LLM writing tests against the live code will write a test that passes amount=0 — because that's what the code currently does.

Quelltest reads the requirement (Raises: ValueError: If amount is <= 0) and generates a test that proves the requirement is enforced. If the code is buggy, the test fails. That's the point.

Comparison

Installation

pip install quelltest

Requires Python 3.11+. Works on Linux, macOS, Windows.

Optional extras

pip install quelltest[pyspark]   # PySpark schema scanning
pip install quelltest[mcp]       # MCP server for AI agents

Configuration

quell init   # adds [tool.quell] to pyproject.toml

[tool.quell]
llm_provider = "anthropic"        # "anthropic" | "openai" | "ollama" | "none"
llm_model    = "claude-sonnet-4-5"
enable_docstring = true
enable_types     = true
enable_mutations = false          # mutmut integration (opt-in)
auto_write       = false
score_threshold  = 0.0

LLM API key — only needed for quell reproduce and LLM fallback:

export ANTHROPIC_API_KEY=sk-ant-...
# or
export OPENAI_API_KEY=sk-...
# or login via browser (no key needed)
quell auth login

Python SDK

from quell import Quell

q = Quell()

# Scan for gaps (no LLM)
result = q.check("src/")
print(f"Score: {result.score:.0%} — {len(result.uncovered)} gaps")

# Reproduce a bug (uses LLM)
q.reproduce("payment accepts zero amount")

# Project score with badge
score = q.score()
print(f"Requirement coverage: {score.percentage}%")

Full SDK reference: quell.buildsbyshashank.tech/docs/sdk/overview

Development

git clone https://github.com/shashank7109/quelltest_lib.git
cd quelltest_lib
uv sync --dev

uv run pytest tests/ -v
uv run ruff check . --fix
uv run mypy quell/

# Run quelltest on itself
uv run quell check quell/ --no-llm

Links

• Docs: quell.buildsbyshashank.tech/docs
• Quickstart: quell.buildsbyshashank.tech/docs/quickstart
• Blog: quell.buildsbyshashank.tech/blog
• PyPI: pypi.org/project/quelltest
• Author: shashankbindal.me

License

MIT — see LICENSE