lightningclean 1.2.1

Blazing fast hardware-accelerated tabular firewall and regulated sanitization engine.

LightningClean

Hardware-Accelerated Tabular Firewall and Regulated Low-Latency Data Sanitization Engine.

LightningClean is an enterprise-grade high-performance Python package built with a native C++ backend designed to sanitize massive tabular datasets at bare-metal speeds. By utilizing hardware-level AVX-512/AVX2 SIMD vectorization and breaking Python's execution limits via OpenMP multi-core multithreading, it isolates and rectifies structural data anomalies seamlessly without memory copies.

---

Key Architectural Capabilities

1. CPUID Dynamic Dispatcher: Automatically detects host microprocessors at runtime to deploy tailored hardware vectors seamlessly.
2. True Zero-Copy Memory Linkage: Shares direct RAM memory data pointer tracks across Python/C++ loops to preserve system memory space.
3. Shield Mode Page Isolation: Wraps executions inside sandboxed memory barriers to capture segmentation faults safely without process termination.
4. Deterministic Audit Control: Locks parallel reduction variances to deliver strict bit-for-bit mathematical reproducibility across compliance audits.
5. PII Masking Engine: Performs regex scans during string array extraction to obscure protected data structures natively.

---

Installation

Standard Production Core

pip install lightningclean

Full Enterprise Web Extra

pip install "lightningclean[web]"

---

Operational Code Example

import pandas as pd
import lightningclean as lc

# Load a massive contaminated analytical database
df = pd.read_csv("unstable_enterprise_dataset.csv")

# 1-Line Execution Pass with Compliance Constraints Enabled
clean_df = lc.clean(
    df, 
    shield=True, 
    deterministic=True,      # Bit-exact audit reproducibility
    pii_mode=True,           # Automatic masking of protected rows
    numa_aware=True          # Pins tasks directly across physical hardware CPU sockets
)

# Extract structured summaries instantly
report = clean_df.attrs["shield_report"]
print(f"Sanitized: {report['cleaned_count']} | Quarantined: {report['corrupted_count']}")