infermap 0.2.0

Inference-driven schema mapping engine

infermap

Inference-driven schema mapping engine.
Map messy source columns to a known target schema — accurately, explainably, with zero config.

📖 Wiki · 🌐 Docs · 🧪 Examples · 💬 Discussions · 🐛 Issues

---

infermap is a schema-mapping engine. Give it any two field collections (CSVs, DataFrames, database tables, in-memory records) and it figures out which source field corresponds to which target field, with confidence scores and human-readable reasoning. Available as a Python package on PyPI and a TypeScript package on npm, with mapping decisions verified bit-for-bit by a shared golden-test parity suite.

Table of contents

• Install
• Quick start
• How it works
• Features
• Which package should I use?
• Custom scorers
• CLI examples
• Config reference
• Documentation
• License

Install

Python

pip install infermap

Optional database extras:

pip install infermap[postgres]   # psycopg2-binary
pip install infermap[mysql]      # mysql-connector-python
pip install infermap[duckdb]     # duckdb
pip install infermap[all]        # all extras

TypeScript / Next.js

npm install infermap

Zero runtime dependencies in the core entrypoint. Compatible with Next.js Server Components, Route Handlers, Server Actions, and the Edge Runtime out of the box. See the package README for the full reference.

Quick start

Python

import infermap

# Map a CRM export CSV to a canonical customer schema
result = infermap.map("crm_export.csv", "canonical_customers.csv")

for m in result.mappings:
    print(f"{m.source} -> {m.target}  ({m.confidence:.0%})")
# fname -> first_name  (97%)
# lname -> last_name   (95%)
# email_addr -> email  (91%)

# Apply mappings to rename DataFrame columns
import polars as pl
df = pl.read_csv("crm_export.csv")
renamed = result.apply(df)

# Save mappings to a reusable config file
result.to_config("my_mapping.yaml")

# Reload later — no re-inference needed
saved = infermap.from_config("my_mapping.yaml")

TypeScript

import { map } from "infermap";

const crm = [
  { fname: "John", lname: "Doe", email_addr: "[e-mail]" },
  { fname: "Jane", lname: "Smith", email_addr: "[e-mail]" },
];

const canonical = [
  { first_name: "", last_name: "", email: "" },
];

const result = map({ records: crm }, { records: canonical });

for (const m of result.mappings) {
  console.log(`${m.source} → ${m.target}  (${m.confidence.toFixed(2)})`);
}
// fname       → first_name  (0.44)
// lname       → last_name   (0.48)
// email_addr  → email       (0.69)

For Next.js, drop it directly into a Route Handler — works on Edge Runtime with zero config:

// app/api/infer/route.ts
import { map } from "infermap";
export const runtime = "edge";

export async function POST(req: Request) {
  const { sourceCsv, targetCsv } = await req.json();
  const result = map({ csvText: sourceCsv }, { csvText: targetCsv });
  return Response.json(result);
}

How it works

Each field pair runs through a pipeline of 6 scorers. Each scorer returns a score in [0.0, 1.0] or abstains (None/null). The engine combines scores via weighted average (requiring at least 2 contributors), then uses the Hungarian algorithm for optimal one-to-one assignment.

Scorer	Weight	What it detects
ExactScorer	1.0	Case-insensitive exact name match
AliasScorer	0.95	Known field aliases (fname ↔ first_name, tel ↔ phone)
PatternTypeScorer	0.7	Semantic type from sample values — email, date_iso, phone, uuid, url, zip, currency
ProfileScorer	0.5	Statistical profile similarity — dtype, null rate, unique rate, length, cardinality
FuzzyNameScorer	0.4	Jaro-Winkler similarity on normalized field names
LLMScorer	0.8	Pluggable LLM-backed scorer (stubbed by default)

Read the full architecture →

Features

	Python	TypeScript
6 built-in scorers	✅	✅
Hungarian assignment	✅ (scipy)	✅ (vendored)
Custom scorers	@infermap.scorer	defineScorer()
In-memory data	Polars, Pandas, list[dict]	Array<Record>
File providers	CSV, Parquet, XLSX	CSV, JSON
Schema definition files	YAML + JSON	JSON
Database providers	SQLite, Postgres, DuckDB	SQLite, Postgres, DuckDB
Engine config	YAML	JSON
Saved mapping format	YAML	JSON
CLI	✅ (Typer)	✅ (node:util)
Apply to DataFrame	✅	❌ (CSV rewrite via CLI)
Edge-runtime compatible	❌	✅
Zero runtime deps	n/a	✅

Full feature parity matrix →

Which package should I use?

If you are…	Use
Building a Python data pipeline or notebook	Python
Building a Next.js app, Node service, or browser tool	TypeScript
Running mapping in a serverless edge function	TypeScript (zero Node built-ins)
Doing ad-hoc CSV exploration on the command line	Python CLI has more features; TS CLI is leaner
Both — Python backend + Next.js admin UI	Both — outputs are interoperable via the JSON config format

Custom scorers

Python

import infermap
from infermap.types import FieldInfo, ScorerResult

@infermap.scorer("prefix_scorer", weight=0.8)
def prefix_scorer(source: FieldInfo, target: FieldInfo) -> ScorerResult | None:
    if source.name[:3].lower() != target.name[:3].lower():
        return None
    return ScorerResult(score=0.85, reasoning=f"Shared prefix '{source.name[:3]}'")

from infermap.engine import MapEngine
from infermap.scorers import default_scorers

engine = MapEngine(scorers=[*default_scorers(), prefix_scorer])

TypeScript

import { MapEngine, defaultScorers, defineScorer, makeScorerResult } from "infermap";

const prefixScorer = defineScorer(
  "prefix_scorer",
  (source, target) => {
    if (source.name.slice(0, 3).toLowerCase() !== target.name.slice(0, 3).toLowerCase()) {
      return null;
    }
    return makeScorerResult(0.85, `Shared prefix '${source.name.slice(0, 3)}'`);
  },
  0.8 // weight
);

const engine = new MapEngine({
  scorers: [...defaultScorers(), prefixScorer],
});

CLI examples

The CLI works the same way in both packages:

# Map two files and print a report
infermap map crm_export.csv canonical_customers.csv

# Map and save the config (Python: --save, TS: -o)
infermap map crm_export.csv canonical_customers.csv -o mapping.json

# Apply a saved mapping to rename columns
infermap apply crm_export.csv --config mapping.json --output renamed.csv

# Inspect the schema of a file or DB table
infermap inspect crm_export.csv
infermap inspect "sqlite:///mydb.db" --table customers

# Validate a saved config against a source
infermap validate crm_export.csv --config mapping.json --required email,id --strict

Config reference

Both packages accept an engine config (scorer weight overrides + alias extensions). Python uses YAML, TypeScript uses JSON; the shape is identical.

# Python: infermap.yaml
scorers:
  LLMScorer:
    enabled: false
  FuzzyNameScorer:
    weight: 0.3
aliases:
  order_id:
    - order_num
    - ord_no

// TypeScript: infermap.config.json
{
  "scorers": {
    "LLMScorer":       { "enabled": false },
    "FuzzyNameScorer": { "weight": 0.3 }
  },
  "aliases": {
    "order_id": ["order_num", "ord_no"]
  }
}

See infermap.yaml.example for a full annotated reference.

Documentation

• 📖 Wiki — full reference for both languages
  • Getting Started
  • Python API
  • TypeScript API
  • Python vs TypeScript — migration guide
  • Scorers
  • Architecture
  • FAQ
• 🌐 Documentation site
• 🧪 Examples
  • Python examples — 7 numbered scripts + sample data
  • TypeScript examples — basic mapping, Next.js Edge Runtime, custom scorer, SQLite, save/reuse
• 📓 Open in Colab — Python notebook
• 💬 GitHub Discussions
• 🐛 Issue tracker

License

MIT