typemonkey

Column type inference and type-aware cleaning for messy tabular data. Infer whether a column is an integer, float, currency, percentage, boolean, date, or free-text string — then clean it to that type. Numbers buried in currency symbols, thousands separators, European decimal commas, accounting parentheses, and percent signs come out as plain Python numbers; a column's worth of yes/Y/1/true come out as bool; twenty-plus spellings of "null" collapse to None.

Part of the monkey toolkit. Delegates date detection to datemonkey and value normalisation to cleanmonkey — it does not reinvent either.

Install

pip install typemonkey

Quick start

from typemonkey import infer_type, clean_numeric, clean_boolean, clean_column

profile = infer_type(["$1,234.56", "$2,000.00", "$3.50"])
profile.type          # TypeName.CURRENCY
profile.confidence    # 1.0
profile.locale        # "us"

clean_numeric(["$1,234.56", "(50)", "12%", "N/A"]).values
# [1234.56, -50, 0.12, None]      # parens = negative, 12% = 0.12, N/A = null

clean_numeric(["1.234,56", "3,50"], locale="eu").values
# [1234.56, 3.5]                  # European decimal comma

clean_boolean(["yes", "NO", "1", "0", "maybe"]).values
# [True, False, True, False, None]   # "maybe" recorded in .failures

clean_column(["01234", "07090", "02139"]).values
# ['01234', '07090', '02139']     # zero-padded IDs preserved as strings

Every entry point returns a typed dataclass (ColumnProfile, CleanResult), not a dict. CleanResult.failures lists (index, original) for non-null values that didn't parse, so "missing" is never confused with "empty".

ColumnProfile.needs_attention is a triage hint for human/LLM review: True only when typemonkey can name an ambiguity (attention_reasons), such as a DD/MM-ambiguous date that still reports confidence == 1.0, a type that only just cleared the threshold, or a string that was almost a real type — the cases plain confidence can't express.

What it recognises

• Numbers — int, float, with thousands separators, apostrophe/space grouping, leading +/-, accounting (parentheses) negatives.
• Currency — $ € £ ¥ ₹ ... symbols and ISO codes (USD, EUR, ...).
• Percentages — "12%", "8 %" → 0.12, 0.08 (or keep as 12, 8).
• Booleans — true/false, t/f, yes/no, y/n, on/off, 1/0.
• Dates — via datemonkey (ISO, US/EU slash and dash, ambiguity reporting).
• Nulls — 20+ spellings (N/A, #N/A, null, none, -, unknown, …).
• Preserve-as-string — zero-padded IDs, Zip+4, phone numbers.
• Locale — US 1,234.56 vs European 1.234,56, auto-detected per column.

CLI

printf '$1,234.56\n$2,000.00\n$3.50\n' | typemonkey profile      # JSON report
printf '12%%\n8 %%\nN/A\n'             | typemonkey clean         # cleaned values
typemonkey clean --type integer column.txt

typemonkey profile prints a JSON ColumnProfile; typemonkey clean prints one cleaned value per line (blank for nulls) and exits non-zero if any non-null value failed to parse.

Using with AI assistants

See SKILL.md for an LLM-oriented quick reference (decision table, worked examples, anti-patterns).

Deliberate tradeoffs

Some behaviour is intentional and might look like a bug — bare 5-digit numbers aren't treated as zips, all-0/1 columns are integers not booleans, Excel serials report numeric. See LIMITATIONS.md for the rationale and escape hatch on each.

License

MIT