mapmonkey 1.1.1

Fuzzy column mapping between source and target schemas: suggest, validate, persist, and apply column + value mappings.

mapmonkey

Fuzzy column mapping between source and target schemas. Source has First Name, target needs first_name; two files have Customer ID vs customer_id vs CustomerID vs cust_id. mapmonkey suggests the mapping with confidence scores, lets you save and reuse it, validates it against real schemas, and applies the renames (and value rewrites) to your data.

Part of the monkey toolkit. MIT licensed.

Install

pip install mapmonkey

Depends on cleanmonkey (invisible character / whitespace normalization) and PyYAML.

Quick start

from mapmonkey import suggest, apply_map, save_mapping, load_mapping

source = ["First Name", "Last Name", "Customer ID", "qty"]
target = ["first_name", "last_name", "customer_id", "quantity"]

suggestion = suggest(source, target)
for m in suggestion.matches:
    print(f"{m.source} -> {m.target}  ({m.confidence:.2f}, {m.reason})")
# First Name -> first_name   (0.95, normalized)
# Customer ID -> customer_id (0.95, normalized)
# qty -> quantity            (0.95, abbreviation)

mapping = suggestion.to_mapping()
save_mapping(mapping, "customers.yaml")        # reuse it later

rows = [{"First Name": "Ann", "qty": "3"}]
apply_map(rows, load_mapping("customers.yaml"))
# [{'first_name': 'Ann', 'quantity': '3'}]

What it does

1. Auto-suggest mappings — suggest(source, target) returns matches with confidence scores in [0, 1] and a reason (exact, normalized, abbreviation, fuzzy).
2. Normalization engine — normalize("CustomerID") == "customer id". Collapses case, separators, camelCase, digit boundaries and abbreviations.
3. Abbreviation dictionary — safe defaults (qty->quantity, dob->date of birth, ...). Risky short words (min, long, st) are opt-in via build_table(aggressive=True); extend with build_table({"abbr": "expansion"}).
4. Mapping persistence — save_mapping / load_mapping as YAML or JSON, chosen by file extension. Round-trips are lossless.
5. Validation — validate(mapping, source_columns=, target_columns=) reports unmapped, stale, and colliding columns.
6. Apply — apply_map(data, mapping) renames columns on list[dict] records or dict[str, list] columnar data; unmapped columns pass through.
7. Value mapping — detect_value_map(["M", "F"]) == {"M": "Male", "F": "Female"}; apply_value_map(values, vmap) rewrites cells, keeping unknown values.

CLI

mapmonkey suggest  --source a.csv --target b.csv --out map.yaml
mapmonkey apply    --map map.yaml --input a.csv --output renamed.csv
mapmonkey validate --map map.yaml --source a.csv --target b.csv

suggest also accepts --source-cols "a,b,c" instead of a file, a --threshold, and --interactive to confirm each match.

Using with AI assistants

See SKILL.md for an LLM-consumable quick reference (decision tree, worked examples, troubleshooting). See LIMITATIONS.md for deliberate design tradeoffs before "fixing" surprising behaviour.

Development & review

See CONTRIBUTING.md for the testing philosophy and the competitive multi-model review process. The release decision is rubric-based: RELEASE_READINESS.md defines the gates and score, and python scripts/readiness.py computes it (history in REVIEW_HISTORY.md).

Scope

In scope: column-name matching, fuzzy matching, abbreviation expansion, mapping persistence, value mapping. Out of scope: data transformation beyond renaming and type conversion (use typemonkey).