validater-data 0.1.0

A YAML-driven data validation library with HTML reports.

validater

validater is a YAML-driven data validation tool for checking datasets and producing an HTML report.

For a full step-by-step guide, see docs.md.

Supported Sources

validater currently supports:

• CSV files.
• Microsoft SQL Server.

Planned source support includes more relational databases, Excel workbooks, and Parquet files.

Install

pip install validater-data

For local development:

pip install -e ".[dev]"

To validate data from Microsoft SQL Server:

pip install "validater-data[mssql]"

Create rules.yaml

name: Validate Workouts
description: Checks required fields before ingestion

source:
  type: csv
  path: ./data/workouts.csv

checks:
  - column: title
    type: not_null
    severity: error

  - column: reps
    type: not_null
    severity: warn

For Microsoft SQL Server, use either table or query. Prefer connection_string_env so credentials are not committed to source control.

name: Validate Users

source:
  type: mssql
  connection_string_env: VALIDATER_MSSQL_URL
  schema: dbo
  table: users

checks:
  - column: user_id
    type: primary_key

  - column: email
    type: regex
    pattern: '^[\w\.-]+@[\w\.-]+\.\w+$'

Use filters when only part of a table should be validated. Multiple filters are combined with AND.

name: Validate Active Customers

source:
  type: mssql
  connection_string_env: VALIDATER_MSSQL_URL
  schema: dbo
  table: customers
  filters:
    - column: ACTIVE_FLAG
      operator: "="
      value: "Y"

    - column: COUNTRY
      operator: "in"
      value: ["US", "CA"]

    - column: DELETED_AT
      operator: "is_null"

checks:
  - column: customer_id
    type: primary_key

  - column: email
    type: regex
    pattern: '^[\w\.-]+@[\w\.-]+\.\w+$'

Supported filter operators are =, !=, >, >=, <, <=, in, not_in, is_null, and is_not_null.

name: Validate Recent Orders

source:
  type: mssql
  connection_string_env: VALIDATER_MSSQL_URL
  query: |
    SELECT order_id, customer_id, order_total, discount_total
    FROM dbo.orders
    WHERE created_at >= DATEADD(day, -1, SYSUTCDATETIME())

checks:
  - column: order_id
    type: primary_key

  - column: order_total
    type: column_dependency
    operator: ">="
    right_column: discount_total

filters are available for table-based MS SQL sources. For advanced filtering, joins, grouped AND/OR logic, or warehouse-specific SQL, use query.

Supported Checks

validater currently supports these checks:

• not_null: column values must not be null.
• regex: non-null column values must match a regex pattern.
• unique: a column, or set of columns, must be unique.
• unique_combination: alias for multi-column uniqueness.
• primary_key: a single column must be non-null and unique.
• composite_primary_key: multiple columns must be non-null and unique together.
• column_dependency: compare two columns row by row, such as max_value > min_value.

Example:

checks:
  - column: email
    type: regex
    pattern: '^[\w\.-]+@[\w\.-]+\.\w+$'

  - column: user_id
    type: primary_key

  - columns: [user_id, created_at]
    type: composite_primary_key

  - columns: [user_id, event_name]
    type: unique_combination

  - column: max_value
    type: column_dependency
    operator: ">"
    right_column: min_value

Run

validater /path/to/rules.yaml

validater exits with a non-zero status when an error severity check fails, so it can be used in CI pipelines. A report is written to reports/.

The misspelled command validaer is also installed as a compatibility alias.