cleancore 1.0.2

Zero-overhead data observability with row-level immutability

CleanCore

Stop shipping unobserved data.

cleancore is a zero-dependency, high-performance tool to inject row-level immutability and audit trails into your Python pipelines.

Think of it like Git for your Data Rows.

---

Why CleanCore?

Data pipelines often fail silently. cleancore automates the observability phase by tracking every mutation and flagging schema drifts before they break your production models.

Features:

• Audit Trail: Captures row-level changes (Old -> New)
• Schema Sentinel: Flags type drifts (e.g., int -> str) and null regressions
• Big Data Engine: Chunk-based processing (10k batches) to prevent memory crashes
• Zero Config: Works with plain Python Lists, Generators, and Pandas

---

Installation

pip install cleancore

---

Quick Start

from cleancore import audit_trail, ProvenaLogger

@audit_trail(rule_id="MASK_PII")
def clean_step(data):
    for row in data:
        row['email'] = "***@***"
    return data

my_data = [{"id": 1, "email": "[e-mail]"}]

with ProvenaLogger("Production_Pipeline") as logger:
    processed = clean_step(my_data, provena_logger=logger)

---

Schema Sentinel (Type Drift)

CleanCore catches silent killers in your data types:

• [WARN] age: int -> str (Unexpected type swap)
• [WARN] price: float -> NoneType (Null regression)

---

Contributing

CleanCore is open-source! GitHub Repository: https://github.com/Sidra-009/cleancore-python-library

---

License

MIT License

---

Author: Sidra Saqlain GitHub: @Sidra-009