quelltest 0.7.0

Your code says what it should do. Quell proves it.

quelltest

Your docstrings say what your code should do. Quelltest proves it.

Quelltest reads your Python docstrings, Pydantic models, and PySpark schemas — extracts every testable requirement — generates a verified pytest test for each gap — and writes it to disk only after proving it catches real violations.

No LLM key required. No code leaves your machine.

pip install quelltest
quell check src/ --fix --no-llm

---

What "verified" means

Every generated test goes through two phases before it's written:

1. Phase 1 — pass on correct code. The test runs against your original source. If it fails here, something is wrong with the generated test itself. Rejected.
2. Phase 2 — fail on violated code. Quelltest injects a targeted violation (comments out the raise, weakens the Field bound, flips nullable=False). The test must fail. If it passes, it proved nothing. Rejected.

Only tests that survive both phases are written. This is the moat between Quelltest and every other test generation tool.

---

Use cases

Docstrings with Raises: / Returns:

def process_payment(amount: float, currency: str) -> PaymentResponse:
    """
    Process a payment.

    Args:
        amount: Payment amount in the specified currency.
        currency: ISO 4217 currency code.

    Returns:
        PaymentResponse with a unique transaction_id.

    Raises:
        ValueError: If amount is <= 0.
        ValueError: If currency is not supported.
    """

quell check src/payments.py --fix --no-llm
# → writes tests/test_payments.py with 2 verified MUST_RAISE tests

Generated test (simplified):

def test_process_payment_raises_valueerror_amount():
    """MUST_RAISE ValueError — auto-generated by Quelltest 0.6.9."""
    with pytest.raises(ValueError):
        process_payment(amount=0, currency="USD")

This test is verified: it passes on the original process_payment and fails when the raise ValueError is commented out.

---

Pydantic v2 Field constraints

from pydantic import BaseModel, Field
from typing import Literal

class PaymentRequest(BaseModel):
    amount: float = Field(gt=0, le=10_000)
    currency: Literal["USD", "EUR", "GBP", "INR"]
    idempotency_key: str = Field(min_length=8, max_length=64)

quell check src/models.py --fix --no-llm
# → BOUNDARY test for amount (>0, ≤10000)
# → ENUM_VALID test for currency (must be one of 4 values)
# → BOUNDARY test for idempotency_key (length 8–64)

---

PySpark schemas

from pyspark.sql.types import StructType, StructField, DoubleType, StringType

payment_schema = StructType([
    StructField("amount", DoubleType(), nullable=False),
    StructField("currency", StringType(), nullable=False),
    StructField("transaction_id", StringType(), nullable=True),
])

quell check src/schemas.py --fix --no-llm
# → NOT_NULL test for amount
# → NOT_NULL test for currency
# → TYPE_CHECK test for schema structure

---

Requirement coverage in CI

# Fail the pipeline if requirement coverage drops below 80%
quell check src/ --ci --threshold 0.80

# Post a gap report as a PR comment
quell pr 42 --comment

GitHub Actions workflow (generated by quell install --pr):

- name: Quelltest requirement coverage
  run: |
    pip install quelltest
    quell check src/ --ci --threshold 0.80

---

Reproduce a bug from plain English

quell reproduce "payment function accepts zero amount and does nothing"
# → finds the relevant function
# → generates a failing test that proves the bug
# → test is verified against the current (buggy) code

---

All commands

Command	LLM?	What it does
quell check src/	No	Scan for requirement gaps
quell check src/ --fix	No	Generate + verify tests for every gap
quell check src/ --fix --no-llm	Never	Same, rule engine only
quell check src/ --ci --threshold 0.8	No	Fail CI if coverage < 80%
quell pr <N> --comment	No	Post gap report as PR comment
quell score --badge	No	Print coverage score + SVG badge
quell reproduce "<bug>"	Yes	Turn bug description into failing test
quell prove src/file.py	No	Show requirement coverage for one file
quell install --pr	No	Write GitHub Actions workflow
quell auth login	—	Login for LLM features

---

How it works

Spec readers (docstring, Pydantic, PySpark)
        │
        ▼
  list[Requirement]
        │
        ▼
Coverage checker (AST scan of test files — no test execution)
        │
        ▼
Rule engine (deterministic, no LLM) → covers ~75% of requirements
LLM engine (fallback, only if configured) → covers complex cases
        │
        ▼
Verification engine — THE MOAT
  Phase 1: test passes on original code
  Phase 2: test fails on violated code
  Always restores source in finally block
        │
        ▼
Writer (libcst injection — preserves formatting)
  Backup before write. Validate CST. Restore on failure.

---

Constraint kinds

Kind	Detected from	Violation injected
MUST_RAISE	Raises: in docstring	Comments out the raise
MUST_RETURN	Returns: in docstring	Returns None instead
BOUNDARY	Field(gt=0, le=100)	Sets value outside bound
ENUM_VALID	Literal["a", "b"]	Passes "__invalid__"
NOT_NULL	nullable=False in StructField	Passes None
TYPE_CHECK	PySpark StructType	Wrong column type

---

Why not just use an LLM?

LLM-generated tests match current behavior, not stated requirements. If your code has a bug (accepts zero amounts), an LLM writing tests against the live code will write a test that passes amount=0 — because that's what the code currently does.

Quelltest reads the requirement (Raises: ValueError: If amount is <= 0) and generates a test that proves the requirement is enforced. If the code is buggy, the test fails. That's the point.

---

Comparison

	Quelltest	pytest-cov	Hypothesis	LLM test gen
Reads existing specs	✅	❌	❌	Partial
No LLM required	✅	✅	✅	❌
Verifies tests catch violations	✅	❌	❌	❌
Requirement-level coverage	✅	❌	❌	❌
Preserves code formatting	✅ (libcst)	—	—	❌
CI gate	✅	✅	❌	❌

---

Installation

pip install quelltest

Requires Python 3.11+. Works on Linux, macOS, Windows.

Optional extras

pip install quelltest[pyspark]   # PySpark schema scanning
pip install quelltest[mcp]       # MCP server for AI agents

---

Configuration

quell init   # adds [tool.quell] to pyproject.toml

[tool.quell]
llm_provider = "anthropic"        # "anthropic" | "openai" | "ollama" | "none"
llm_model    = "claude-sonnet-4-5"
enable_docstring = true
enable_types     = true
enable_mutations = false          # mutmut integration (opt-in)
auto_write       = false
score_threshold  = 0.0

LLM API key — only needed for quell reproduce and LLM fallback:

export ANTHROPIC_API_KEY=sk-ant-...
# or
export OPENAI_API_KEY=sk-...
# or login via browser (no key needed)
quell auth login

---

Python SDK

from quell import Quell

q = Quell()

# Scan for gaps (no LLM)
result = q.check("src/")
print(f"Score: {result.score:.0%} — {len(result.uncovered)} gaps")

# Reproduce a bug (uses LLM)
q.reproduce("payment accepts zero amount")

# Project score with badge
score = q.score()
print(f"Requirement coverage: {score.percentage}%")

Full SDK reference: quell.buildsbyshashank.tech/docs/sdk/overview

---

Development

git clone https://github.com/shashank7109/quelltest_lib.git
cd quelltest_lib
uv sync --dev

uv run pytest tests/ -v
uv run ruff check . --fix
uv run mypy quell/

# Run quelltest on itself
uv run quell check quell/ --no-llm

---

Links

• Docs: quell.buildsbyshashank.tech/docs
• Quickstart: quell.buildsbyshashank.tech/docs/quickstart
• Blog: quell.buildsbyshashank.tech/blog
• PyPI: pypi.org/project/quelltest
• Author: shashankbindal.me

---

License

MIT — see LICENSE