Nyctea

Polars-based data validation library with an extensible OOP validator architecture.

Features

Validator system

• Extensible: create custom parsers and checks by inheriting from base classes
• Type-safe: generic validator classes with runtime validation
• Discoverable: tag-based validator discovery and registration

Customizable pipeline

• Flexible: add, remove, or reorder validation phases
• Validated: strict dependency enforcement prevents invalid configurations
• Observable: built-in logging and metrics collection

Schema-centric API

• Intuitive: schema.validate(df, registry), the schema owns validation
• Pythonic: clean, object-oriented design
• Out-of-core: built on Polars LazyFrame, designed for larger-than-RAM data

Installation

pip install nyctea
# or with uv
uv add nyctea

Quick start

import polars as pl
from nyctea import Registry, SchemaModel, register_builtins

# Define schema
schema = SchemaModel.from_dict({
    "columns": {
        "name": {
            "dtype": "Utf8",
            "parsers": [{"name": "strip"}, {"name": "lower"}],
            "nullable": False,
        },
        "age": {
            "dtype": "Int64",
            "parsers": [{"name": "to_int"}],
            "checks": [{"name": "min_value", "args": {"min": 0}}],
            "nullable": False,
        },
    }
})

# Register built-in validators
registry = Registry()
register_builtins(registry)

# Load and validate data
df = pl.scan_csv("data.csv")
result = schema.validate(df, registry)

# Inspect results
print(result.report.summary())
print(result.data.collect())

Creating custom validators

Custom parser (OOP)

import polars as pl
from nyctea.validators.base import ValidatorMetadata
from nyctea.validators.column import ColumnParser


class TrimParser(ColumnParser):
    def __init__(self):
        super().__init__(ValidatorMetadata(
            name="trim",
            description="Remove whitespace",
            tags=["string", "cleaning"],
        ))

    def execute(self, column: pl.Expr, **kwargs) -> pl.Expr:
        return column.str.strip_chars()

    def validate_args(self, **kwargs) -> None:
        pass  # No arguments


registry.register_column_parser(TrimParser())

Custom check (functional)

import polars as pl
from nyctea.validators.decorators import ValidatorDecorator

decorators = ValidatorDecorator(registry)


@decorators.column_check(name="positive", tags=["numeric"])
def is_positive(column: pl.Expr) -> pl.Expr:
    return column > 0

Architecture

Validator[TInput, TOutput]
├── ColumnValidator[pl.Expr, pl.Expr]
│   ├── ColumnParser (transformations)
│   └── ColumnCheck (validations)
└── FrameValidator[pl.LazyFrame, pl.LazyFrame]
    ├── FrameParser (transformations)
    └── FrameCheck (validations)

Registry
├── column_parsers: ValidatorRegistry[ColumnParser]
├── column_checks: ValidatorRegistry[ColumnCheck]
├── frame_parsers: ValidatorRegistry[FrameParser]
└── frame_checks: ValidatorRegistry[FrameCheck]

ValidationPipeline
├── ColumnResolutionPhase (synonyms)
├── ColumnParsingPhase (transformations)
├── CoercionPhase (dtype coercion)
└── ColumnCheckPhase (checks, nullable enforcement)

Testing

uv run pytest tests/ -v
uv run pytest tests/ --cov=src/nyctea --cov-report=term --cov-report=html
uv run ruff check src/ tests/
uv run ty check src/nyctea

CI runs linting, type checking, and the test suite on Python 3.11 through 3.14 for every pull request.

Documentation

• Quickstart: getting started guide
• Features: schema syntax and validator capabilities
• Registry guide: registering and discovering validators
• API reference: full public API
• Breaking changes: migrating between versions

Contributing

Contributions are welcome. Open an issue before starting anything beyond a trivial fix; see docs/development/contributing.md for the full workflow (issue-first, branch and PR conventions, CI overview) and DEVELOPMENT.md for local setup.

License

MIT