A private, local LLM-powered data dictionary parser and entity mapper with automated cleaning.
Project description
dd-parser-cleaner
A lightweight framework for documenting and automating dataset preparation with AI-driven metadata discovery.
What it does
- Builds a documented cleaning workflow for raw datasets.
- Creates structured metadata and transformation guidance for later featurization.
- Supports repeatable, deterministic dataset preparation in local environments.
Real-world examples
For complete, real-world migration examples, see:
- SBA dataset: https://github.com/rajivsam/kmds_migration/blob/main/sba_migration/documents/sba_development_example_full_doc.md
- Olist dataset: https://github.com/rajivsam/kmds_migration/blob/main/olist_migration/documents/olist_development_example_full_doc.md
These examples show how the tool is applied to cross-sectional datasets. An example for longitudinal/panel datasets is coming soon.
Why it matters
- Saves time by automating metadata discovery and documentation.
- Keeps data cleaning transparent and audit-ready.
- Makes downstream featurization easier because the cleaning process is already documented.
Quick start
Install
pip install dd-parser-cleaner
Initialize a workspace
uv run init-workspace ./my_project
uv run bootstrap-config ./my_project
Discover package features from Python
from dd_parser_cleaner import get_package_info
info = get_package_info()
print(info)
Run the core workflow
classify-entities
uv run clean-dataset --action full --workspace ./my_project
Where to look next
USER_GUIDE.mdfor usage detailsdocuments/for methodology and internal design notestests/notebooks/for example notebook workflows
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file dd_parser_cleaner-0.7.0.tar.gz.
File metadata
- Download URL: dd_parser_cleaner-0.7.0.tar.gz
- Upload date:
- Size: 1.6 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.9.26 {"installer":{"name":"uv","version":"0.9.26","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
e86573ce42fbd4c6f4d3d9c39f8ef724e70b23ad057b6bb4dfa4b567f039d972
|
|
| MD5 |
d41415632197faec7ae0932ca38a693b
|
|
| BLAKE2b-256 |
f20c95b81b46a17670209a49d5eaec838d255d24a87565b194ff59f4dd83b9ba
|
File details
Details for the file dd_parser_cleaner-0.7.0-py3-none-any.whl.
File metadata
- Download URL: dd_parser_cleaner-0.7.0-py3-none-any.whl
- Upload date:
- Size: 61.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.9.26 {"installer":{"name":"uv","version":"0.9.26","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
56cf02ea0b9e339d67434f6a65c7a46b9db9904ab6ff6b3bc11de88123ea9710
|
|
| MD5 |
596351b52c070ba01573b9a432c068e9
|
|
| BLAKE2b-256 |
f17708db104511d8bcc2a9d3a2a6bd816359c6e2bd4c0645675a533edd766bbc
|
