pydata-constraints 1.0.1

The easiest way to validate your data streams in Python. Whether you have small JSON files or massive CSV dumps, this tool ensures your data isn't garbage.

PyData Constraints

PyData Constraints is the easiest way to validate your data streams in Python. Whether you have small JSON files or massive CSV dumps, this tool ensures your data isn't garbage.

---

🚀 Why PyData Constraints?

• Rule-Based: Define your rules in simple JSON/YAML files. No coding required.
• Universal: Works with JSON and CSV out of the box using efficient streaming.
• Developer Friendly: Written in pure Python with minimal dependencies.

⚡ Basic Use Case (At a glance)

Imagine you have a users.json file and you want to ensure all emails are valid.

1. Your Data (users.json):

[
  { "id": 1, "email": "[e-mail]" },
  { "id": 2, "email": "bob-has-no-domain" }
]

2. Your Rules (config.json):

{
  "sources": [
    {
      "service": "users",
      "type": "file",
      "path": "users.json",
      "format": "json"
    }
  ],
  "constraints": [
    {
      "type": "format",
      "id": "valid-email",
      "service": "users",
      "field": "email",
      "regex": "^[^\\s@]+@[^\\s@]+\\.[^\\s@]+$",
      "message": "Invalid email: {{email}}"
    }
  ]
}

3. Run and get results:

$ data-constraints validate --config config.json
[INFO] Validating data...
[valid-email] (format) Invalid email: bob-has-no-domain
Validation finished. Found 1 issues.

📦 Installation

To install via pip:

pip install pydata-constraints

This installs both the python package pydata_constraints and the CLI command data-constraints.

🧠 Basic Concepts

PyData Constraints works with three core files:

1. Data Files: Your actual data dumps in .json or .csv format.
2. Config File: A JSON/YAML file pointing the engine to your data and rules.
3. Constraints (Rules): The definitions of what is valid.

Rule Types at a Glance

• 📝 Format: Ensure strings look correct (e.g. Emails).
  • Example: "regex": "^[^\\s@]+@[^\\s@]+\\.[^\\s@]+$"
• 🆔 Unique: Ensure no duplicate IDs exist across a file.
  • Example: "field": "employee_id"
• 🔗 Foreign Key: Ensure referenced IDs actually exist in another file.
  • Example: order.userId must exist in users.id

📚 Documentation

For full documentation, guides and advanced use cases, please check the docs/ directory.

• Key Concepts: Easy-to-understand explanation of file types and constraints.
• User Guide: The comprehensive guide to using the CLI and defining rules.
• Integration Guide: How to integrate the engine programmatically in Python.
• Examples: Runnable examples, ranging from simple to e-commerce.

🛠️ Features

Feature	Description
Format Validation	Regex-based validation for strings (Emails, Phones, Codes).
Unique Validation	Ensure IDs and codes are unique across your dataset.
Foreign Keys	Validate relationships between different files (e.g. order.userId -> user.id).
Multiple Reporters	Output results to Console, JSON, or Markdown files.

🤝 Contributing

In PyData Constraints contributions, bug reports, and feature requests are welcome. If you have ideas, just launch your PRs!

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.