Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

https://github.com/todofixthis/filters/actions/workflows/build.yml/badge.svg https://readthedocs.org/projects/filters/badge/?version=latest

Filters

The Filters library provides an easy and readable way to create complex data validation and processing pipelines, including:

  • Validating complex JSON structures in API requests or config files.

  • Parsing timestamps and converting to UTC.

  • Converting Unicode strings to NFC, normalising line endings and removing unprintable characters.

  • Decoding Base64, including URL-safe variants.

And much more!

The output from one filter can be piped into the input of another, enabling you to chain filters together to quickly and easily create complex data schemas and pipelines.

Philosophy

Filters applies the UNIX philosophy to data validation: do one thing well, and compose small tools together.

Each filter performs a single, focused task. Chain them using the | operator to build sophisticated validation pipelines that are easy to read and maintain.

Type-safe: Chain filters with | and a type checker infers the real output type all the way through — FilterRunner(...).cleaned_data reports Decimal for a chain ending in f.Decimal, not Any.

Opinionated: Makes deliberate choices to handle common issues automatically (Unicode normalisation, UTC conversion, etc.) so you write less boilerplate.

Quick Start

Install via pip:

pip install phx-filters

Create a validation schema:

import filters as f
from decimal import Decimal

# Define your schema
schema = f.FilterRunner(
    f.FilterMapper({
        "lat": f.Required | f.Decimal | f.Min(Decimal(-90)) | f.Max(Decimal(90)),
        "lon": f.Required | f.Decimal | f.Min(Decimal(-180)) | f.Max(Decimal(180)),
        "name": f.Required | f.Unicode | f.Strip,
    })
)

# Validate data
schema.apply({"lat": "42.36", "lon": "-71.06", "name": "  Boston  "})

if schema.is_valid():
    clean_data = schema.cleaned_data
    # clean_data = {
    #     "lat": Decimal("42.36"),
    #     "lon": Decimal("-71.06"),
    #     "name": "Boston"
    # }
else:
    errors = schema.errors
    # errors = {
    #     "lat": [
    #         {"code": "too_small", "message": "Value is too small (must be >= -90)."}
    #     ]
    # }

FilterRunner provides a familiar interface similar to Django forms, making it easy to integrate into web applications.

Examples

Validate API Request Data

When building APIs, you need to validate request payloads and handle errors gracefully. FilterRunner makes this straightforward:

from decimal import Decimal
import filters as f

# Define validation for a user registration endpoint
user_schema = f.FilterRunner(
    f.FilterMapper(
        {
            "email": f.Required | f.Unicode | f.Strip | f.MaxLength(254),
            "age": f.Required | f.Int | f.Min(13) | f.Max(120),
            "timezone": f.Decimal | f.Min(Decimal("-15")) | f.Max(Decimal("15")),
        },
        allow_extra_keys=False,
    )
)

# Validate incoming data
user_schema.apply(request_data)

if user_schema.is_valid():
    # Save to database
    user = User.create(**user_schema.cleaned_data)
else:
    # Return validation errors to client
    return {"errors": user_schema.errors}, 400

Parse Complex JSON Structures

Filters excels at validating nested data structures with complex constraints:

schema = f.FilterRunner(
    f.JsonDecode |
    f.FilterMapper(
        {
            "birthday": f.Date,
            "gender": f.CaseFold | f.Choice(choices={"f", "m", "n"}),
            "utcOffset": (
                f.Decimal |
                f.Min(Decimal("-15")) |
                f.Max(Decimal("15")) |
                f.Round(to_nearest="0.25")
            ),
        },
        allow_extra_keys=False,
        allow_missing_keys=False,
    )
)

result = schema.apply('{"birthday":"1879-03-14", "gender":"M", "utcOffset":"1"}')

Process Lists of Data

Use FilterRepeater to apply validation to every item in a collection:

# Clean a list of user-generated strings
schema = f.FilterRunner(
    f.FilterRepeater(f.Unicode | f.Strip | f.MaxLength(100))
)

result = schema.apply([
    "  some text  ",
    b"\xe2\x99\xaa unicode bytes ",
    "another string",
])

For more examples and detailed documentation, visit https://filters.readthedocs.io/

Features

  • Composable: Chain filters using the | operator

  • Type-safe: Full type hint support for IDE autocomplete and mypy

  • Familiar API: FilterRunner provides Django-form-like interface

  • Extensible: Create custom filters by extending BaseFilter

  • Battle-tested: Used in production applications for years

  • Well-documented: Comprehensive docs at https://filters.readthedocs.io/

Requirements

Filters is known to be compatible with the following Python versions:

  • 3.14

  • 3.13

  • 3.12

Installation

Install the latest stable version via pip:

pip install phx-filters

Extensions

The following extensions are available:

  • Django Filters: Adds filters designed to work with Django applications. To install:

    pip install phx-filters[django]
  • ISO Filters: Adds filters for interpreting standard codes and identifiers. To install:

    pip install phx-filters[iso]

Maintainers

To install the distribution for local development, some additional setup is required:

  1. Install uv (only needs to be done once).

  2. Run the following command to install additional dependencies:

    uv sync --group=dev
  3. Activate pre-commit hook:

    uv run autohooks activate --mode=pythonpath

Running Unit Tests and Type Checker

Run the tests for all supported versions of Python using tox:

uv run tox -p

If you just want to run unit tests in the current virtualenv (using pytest):

uv run pytest

If you just want to run type checking in the current virtualenv (using mypy):

uv run mypyc src test

Documentation

To build the documentation locally:

  1. Switch to the docs directory:

    cd docs
  2. Build the documentation:

    uv run make html

Releases

Steps to build releases are based on Packaging Python Projects Tutorial.

1. Build the Project

  1. Delete artefacts from previous builds, if applicable:

    rm dist/*
  2. Run the build:

    uv build
  3. The build artefacts will be located in the dist directory at the top level of the project.

2. Upload to PyPI

  1. One-time setup — install keyring and store your PyPI token:

    uv tool install keyring
    uv tool update-shell
    keyring set https://upload.pypi.org/legacy/ __token__
  2. Bump the version (updates pyproject.toml and re-locks uv.lock):

    uv version <version>
  3. Upload build artefacts to PyPI:

    uv publish --username __token__

3. Create GitHub Release

  1. Create an annotated tag and push to GitHub:

    git tag -a <version> -m "Release <version>"
    git push origin <version>

    <version> must match the version set in step 2 above.

  2. Go to the Releases page for the repo.

  3. Click Draft a new release.

  4. Select the tag that you created in step 1.

  5. Specify the title of the release (e.g., Filters v1.2.3).

  6. Write a description for the release. Make sure to include: - Credit for code contributed by community members. - Significant functionality that was added/changed/removed. - Any backwards-incompatible changes and/or migration instructions. - SHA256 hashes of the build artefacts.

  7. GPG-sign the description for the release (ASCII-armoured).

  8. Attach the build artefacts to the release.

  9. Click Publish release.

Metadata

Release files for phx-filters 4.0.0a1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for phx-filters 4.0.0a1
File Size Uploaded
phx_filters-4.0.0a1.tar.gz 593.1 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for phx-filters 4.0.0a1
File Interpreter ABI Platform
phx_filters-4.0.0a1-py3-none-any.whl Python 3 none any Details

Total release size: 645.2 kB

Release files / phx_filters-4.0.0a1.tar.gz

Download URL phx_filters-4.0.0a1.tar.gz
Size 593.1 kB
Tags Source
SHA-256 checksum
How to use checksums
f93113ab3810468772ba6aeb88a2241d90ac89f0f9e45aae6b40609a035e8389
BLAKE2b-256 checksum
How to use checksums
20af0784c103a2132871dd99ba325eefaf4397109760b7541e41604ad4cc1972
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / phx_filters-4.0.0a1-py3-none-any.whl

Download URL phx_filters-4.0.0a1-py3-none-any.whl
Size 52.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
4d244ddf05377d809c5446b4fe4ada913e54f4a457023ff647c6c1a8dc339a7b
BLAKE2b-256 checksum
How to use checksums
1752516f54721ed573a9b23b93cb33b43d3811c2d0f665a2a519b4bd9850af0f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release history Release notifications | RSS feed

This release

4.0.0a1 This release

2 release files

3.7.0

2 release files

3.6.0

2 release files

3.5.1

2 release files

3.5.0

2 release files

3.4.0

2 release files

3.3.0

2 release files

3.2.1

2 release files

3.2.0

2 release files

3.1.0

2 release files

3.0.1

2 release files

3.0.0

2 release files

2.0.2

2 release files

2.0.1

2 release files

2.0.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page