dd-parser-cleaner 0.2.0

A private, local LLM-powered data dictionary parser and entity mapper with automated cleaning.

dd-parser-cleaner

A private, local LLM-powered data dictionary parser, entity mapper, and case-preserving data transformation pipeline. This package crawls raw dataset structures, classifies attributes through local inference, and executes deterministic data cleaning routines while protecting case variations and file formats.

🚀 Quick Start

1. Installation

This repository uses uv for environment management and dependency tracking.

# Clone the repository and sync the local virtual environment
cd dd_parser_cleaner
uv sync

# Install package command entry points in editable mode
uv pip install -e .

2. Run the Pipeline CLI

# Step 1: Ingest layout and classify data dictionary entities
classify-entities --workspace . --config config.yaml

# Step 2: Validate signatures and execute geographic scrubbing routines
clean-dataset --workspace . --config config.yaml

3. Run Automated Tests (pytest)

All integration tests run inside an isolated folder context to ensure production directories are never contaminated.

uv run pytest -v tests/

---

📑 Core Documentation Index

For deep-dive architectural specifications, path coordinator details, and pipeline handshake rules, please consult the full documentation suite:

• 👉 USER_GUIDE.md: Exhaustive manual detailing hardened engineering constraints, file-routing abstraction schemas, data transformation algorithms, and structural testing profiles.