conformly 0.3.3

Generate valid & invalid test data from your typed schemas

conformly

Declarative test data generator for Python. Turns data models (now only dataclasses) and type constraints into valid fixtures and negative test cases.

conformly allows you to define your data schema once using standard Python dataclasses and Annotated constraints, and instantly generate rigorous test data. It replaces verbose factory patterns with a smart, schema-aware generator that supports both happy-path and edge-case testing.

Instead of writing separate factory classes or hardcoding test dictionaries, Conformly:

• Interprets constraints defined at the type level as an executable schema (length bounds, regex patterns, numeric ranges)
• Generates valid data strictly adhering to all constraints for happy-path testing
• Generates invalid data intelligently violating constraints for negative testing and fuzzing
• Bridges static typing and dynamic testing — your schema is the single source of truth

Key Features

• Typed Constraints as First-Class Objects
  Constraints in Conformly are explicit, typed entities bound to base Python types. They are interpreted as part of the schema and drive both valid and invalid data generation, rather than being treated as passive metadata.

• Schema-Driven Generation
  Your dataclass and its type annotations form a complete, executable data schema. Test data is derived directly from this schema — no factories, no duplicated validation logic, no hardcoded dictionaries.

• Systematic Negative Testing
  Invalid data is generated by intentionally violating constraints of a single, explicitly targeted field, while keeping the rest of the object valid. This produces minimal, meaningful negative cases suitable for API and validation tests.

• Guaranteed Happy-Path
  Valid generation strictly satisfies all declared constraints. Generated data is internally consistent and suitable for end-to-end tests, database seeding, and contract testing.

• Multiple Constraint Definition Styles
  Constraints can be defined using:

  • Annotated[T, Constraint(...)] (typed, reusable, recommended)
  • Annotated[T, "k=v"] shorthand (compact and convenient)
  • field(metadata={...}) for compatibility and gradual adoption

• Zero Boilerplate, Pure Python
  Works directly with standard dataclasses and typing.Annotated. No custom DSLs, no runtime code generation, no magic metaclasses. Lightweight, dependency-minimal, and fully type-checkable with mypy.

Install

pip install conformly
# or with uv
uv add conformly

Quickstart

Define a model:

from dataclasses import dataclass, field
from typing import Annotated, Optional
from conformly import case, cases
from conformly.constraints import MinLength, MaxLength, Pattern, GreaterOrEqual, LessOrEqual

Username = Annotated[
    str,
    MinLength(3),
    MaxLength(32),
]

Age = Annotated[
    int,
    GreaterOrEqual(18),
    LessOrEqual(120),
]

@dataclass
class User:
    username: Username
    email: Annotated[str, Pattern(r"^[^\s@]+@[^\s@]+\.[^\s@]+$")]
    age: Age
    bio: Optional[Annotated[str, MaxLength(160)]] = None

Generate valid data:

user = case(User, valid=True)
# -> {"username": "Abc", "email": "x@y.z", "age": 42, "bio": None}

Generate an invalid case for a specific field:

bad_user = case(User, valid=False, strategy="age")
# bad_user["age"] is outside 18..120 (either < 18 or > 120)
# All other field remain valid

Generate many cases:

items = cases(User, valid=True, count=10)

Use Cases

case(Model, ...) # single generated object
cases(Model, ...) # list of generated objects

strategy values:

• <field_name> - target specific field for invalidation (for nested fields using dot syntax "profile.name")
• "random" - choose a random field/constraint to violate
• "all" - (for cases) produce all minimal invalid variations for the model
• "first" -violate the first constrained field (for case) or take the first N constrained fields (for cases)

API Testing

# Valid payloads for happy-path tests
for _ in range(100):
    payload = case(CreateUserRequest, valid=True)
    response = client.post("/users", json=payload)
    assert response.status_code == 201

# Invalid payloads for error handling tests
invalid = case(CreateUserRequest, valid=False, strategy="age")
response = client.post("/users", json=invalid)
assert response.status_code == 400

# As option create all possible invalid cases for payload in one only line
invalid_payloads = cases(CreateUserRequest, valid=False, strategy="all")
for payload in invalid_payloads:
    response = client.post("/users", json=payload)
    assert response.status_code == 400

Database Seeding

# Generate realistic test data respecting schema constraints
products = cases(Product, valid=True, count=1000)
db.insert_many("products", products)

Fuzzing & Property-Based Testing

Conformly is not a replacement of Hypothesis, but a complementary tool for schema-driven testing and negative case generation.

# Generate random invalid data to stress-test validation
for _ in range(500):
    invalid = case(Model, valid=False, strategy="random")
    assert validate(invalid) is False  # Should always reject

Nested Models

Conformly supports nested models represented as tree structures (e.g. dataclasses containing other dataclasses).

Cyclic references between models are not supported

Constraints defined on nested fields are discovered recursively and can be used for both valid and invalid data generation.

Model Declaration

from dataclasses import dataclass
from typing import Annotated

from conformly.constraints import MinLength, GreaterOrEqual, Pattern

@dataclass
class Profile:
    email: Annotated[str, Pattern(r"^[^\s@]+@[^\s@]+\.[^\s@]+$")]
    phone: Annotated[str, Pattern(r"^\+[1-9]\d{1,14}$")]


@dataclass
class User:
    name: Annotated[str, MinLength(3)]
    age: Annotated[int, GreaterOrEqual(18)]
    profile: Profile

Generation Example

from conformly import case

valid_data = case(User, valid=True)
print(valid_data)
# {
#     "name": "validname",
#     "age": 25,
#     "profile": {
#            "email": "[e-mail]",
#            "phone": "[phone]"
#         }
# }

invalid_data_by_field = case(User, valid=False, strategy="profile.email")
print(invalid_data_by_field)
# {
#     "name": "validname",
#     "age": 25,
#     "profile": {
#            "email": "nonemailstring",
#            "phone": "[phone]"
#         }
# }

Supported Constraints

String

• MinLength(value: int) — minimum string length
• MaxLength(value: int) — maximum string length
• Pattern(regex: str) — regex pattern (must match)

Integer / Float Bounds

• GreaterThan(value: float | int) — strictly greater than
• GreaterOrEqual(value: float | int) — greater than or equal
• LessThan(value: float | int) — strictly less than
• LessOrEqual(value: float | int) — less than or equal

Closed-set (Finite Domain)

• OneOf(values: tuple[Any, ...]) — closed set of allowed values (canonical representation of closed-sets after parsing)

Supported closed-set sources:

• typing.Literal
• Enum

Boolean

• Basic boolean generation (no extra constraints)

Shorthand -> Constraint class mapping

• "gt" -> GreaterThan
• "ge" -> GreaterOrEqual
• "lt" -> LessThan
• "le" -> LessOrEqual
• "min_length" -> MinLength
• "max_length" -> MaxLength
• "pattern" -> Pattern

Defining Constraints

1) Annotated[..., Constraint(...)] (type-safe, recommended)

from typing import Annotated
from conformly.constraints import MinLength, GreaterOrEqual

username: Annotated[str, MinLength(3)]
age: Annotated[int, GreaterOrEqual(18)]

2) Annotated[..., "k=v"] (shorthand string syntax)

title: Annotated[str, "min_length=5", "max_length=200"]
views: Annotated[int, "ge=0"]
rating: Annotated[float, "ge=0", "le=5"]

3) field(metadata={...})

from dataclasses import field

sku: str = field(metadata={"pattern": r"^[A-Z0-9]{8}$"})
stock: int = field(metadata={"ge": 0})
price: float = field(metadata={"gt": 0})

All syntaxes are fully compatible - mix and match as needed.

Exception: OneOf (including Literal and Enum) defines closed set of allowed values and cannot be combined with other constraints.

Invalid Generation Contract (Important)

For case(Model, valid=False, strategy="<field>"):

• Exactly one field is targeted (the one specified by strategy).
• The generator will violate constraints for that field, making it invalid.
• If a field has multiple constraints, the violated constraint may be chosen by generator logic (not necessarily the one you expect).
• For numeric bounds, invalid values may violate the lower or upper bound (e.g., age > 120 or age < 18).
• For float bounds, invalid generation may produce inf when violating the upper boundary.

If you need deterministic control over which exact constraint to violate, that is not implemented in yet (see Roadmap).

Optional Fields and Defaults

• If a field is optional (Optional[T]), valid generation may produce None.
• If a field has a default value, valid generation returns the default.
• Invalid generation requires at least one constraint on the targeted field (raises ValueError otherwise).

Development

Install dependencies:

uv sync

Run tests:

uv run -m pytest -q

Run with coverage:

uv run -m pytest --cov=conformly --cov-report=term-missing

Build & check package:

uv build
uv run -m twine check dist/*

Roadmap

• Deterministic invalid generation - explicitly select which constraint to violate
• Better regex invalidation - guarantee that invalid strings don't match patterns
• More adapters - pydantic, TypedDict, attrs support
• More constraints and types - multitiple_of, list[T], dict[T] etc.
• Custom generators - allow per-field generator overrides

Changelog

See CHANGELOG.md for release notes and migration guidance.

License

MIT — see LICENSE file for details

Contributing

Contributions welcome! Please:

• Fork the repo
• Create a feature branch
• Add tests for new functionality
• Run uv run -m pytest and uv run -m ruff check .
• Submit a pull request