deidkit 0.1.0

Schema-driven pseudonymization for clinical/tabular datasets: synthetic names, interval-preserving date shifting, multi-stage free-text PII detection, and before/after audit exports.

deidkit

Schema-driven de-identification for clinical & tabular data. Point it at your tables; get back a usable, pseudonymized copy plus a full audit of every change.

It replaces names with realistic fakes (in columns and inside free text), shifts dates while preserving every interval, swaps IDs/phones/emails for stable surrogates, and sends anything uncertain to a human-review sheet instead of guessing. It is deterministic (same input + same secret ⇒ same output) and re-runnable.

---

Contents

1. Install
2. Quick start
3. Inputs — what you feed it
4. What it does to each column
5. Outputs — what you get back
6. Intermediate outputs — inspect before you trust
7. Training it — the review → learn loop
8. Tuning recall & precision
9. Reading from a database (Fabric / Azure SQL)
10. Database connection parameters — exactly what to fill in
11. What to look out for
12. Full option reference (CLI + Python)
13. Troubleshooting

---

1. Install

pip install deidkit

One command, no square brackets, nothing to pick. The database drivers needed to read your Azure SQL / Fabric source (SQLAlchemy + pyodbc) are bundled in, so a plain pip install deidkit is ready to connect.

Heads-up — not on PyPI yet. Until the package is published (see Publishing), install it straight from the repository:

pip install git+https://github.com/<your-org>/deidkit.git
# or, from a local clone:  pip install .

Both give you the identical deidkit command and library.

One system-level thing is required once, on the machine that will connect: the Microsoft ODBC Driver 18 for SQL Server — the actual database driver that pyodbc talks to:

# macOS
brew tap microsoft/mssql-release https://github.com/microsoft/homebrew-mssql-release
brew install msodbcsql18
# Linux (Debian/Ubuntu): install Microsoft's "msodbcsql18" package
# Windows: install the "ODBC Driver 18 for SQL Server" MSI

Requires Python 3.9+. Confirm it:

deidkit --version

---

2. Quick start

CLI — one line:

deidkit run data/

Reads every table in data/, de-identifies it, and writes two folders: data_deid/ (ship this) and data_deid-PRIVATE/ (keep this — see §5).

Python — one call:

import deidkit as dk

result = dk.deidentify("data/", out="deid_out", secret_file="secret.key")
print(result.summary)                      # counts of what changed
clean = result.tables["tbl_notes"]         # or write to files via out=

The recommended run (pins a secret, uses the schema-tuned policy):

deidkit init-secret --out secret.key       # do this once, keep the file private
deidkit run data/ deid_out \
    --policy examples/policy.meridian.yaml \
    --secret-file secret.key

The data/ above is a folder of table files, used here just to show the shape of a run. Your data is in the Fabric / Azure SQL database, not a folder — for that path (what you connect to and how), jump to §9 and §10; it's a few lines that read straight from the SQL endpoint.

CLI or code — same engine. Everything below is shown for both interfaces. The rule of thumb: every CLI --flag is a field on the Python Policy object (e.g. --mode balanced ⇄ Policy(mode="balanced"), --ignore blood_type ⇄ Policy(ignore={"blood_type"})). Pick whichever fits your workflow — a one-off delivery (CLI) or a pipeline/notebook (code).

---

3. Inputs — what you feed it

Your source: the Fabric / Azure SQL database

For this project the input is your live database — the Azure SQL Database mirrored into Microsoft Fabric. deidkit connects to the Fabric SQL analytics endpoint (read-only), reads the six tables directly, de-identifies them, and writes the clean copy out. It never writes back to your database.

To connect, we need a few connection details from you (the data owner) — the SQL endpoint, the database name, and how to authenticate. Exactly what to send, and how the connection works, is spelled out in §9 (how it runs) and §10 (the connection details you provide). Start there.

Or: a data export as files

If instead you hand over a data extract as files (one table per file in a folder), deidkit reads those too — .csv, .tsv, or .json. One file = one table, and the filename becomes the table name. CSVs are read as text (so IDs and date formats aren't mangled) and tolerate non-UTF-8 exports.

deidkit run data/            # a folder of table files

Two things that matter about your columns

• patient_id is the anchor. By default entity_key = patient_id. Every date for a patient is shifted by the same offset, and every real value maps to the same surrogate — across all tables — so joins and time-intervals survive. If your patient column has a different name, set --entity-key <col>. All six Meridian tables share patient_id, so this works out of the box.
• Column names drive the plan. deidkit recognises clinical column names in English and Spanish (e.g. note_full, result_interpretation, report_text, clinical_justification_pbs) and classifies each one automatically. You can override any decision (§12). An optional data-dictionary JSON (--schema dict.json) makes the classification authoritative instead of heuristic.

---

4. What it does to each column

Every column gets exactly one strategy. Most are chosen automatically; you can force any of them with a rule.

strategy	effect	typical columns
passthrough	left untouched (the safe default)	sex, hemoglobin, diagnosis, insurer
date_shift	move by the per-patient offset — intervals preserved	note_datetime, encounter_start/end, birth_date
freetext	detector scrubs names/IDs/phones/emails/dates inside the text	note_full, report_text, result_interpretation
synthetic_name	a name value → realistic fake name	a patient_name / provider_name column
identifier	an ID value → stable shape-preserving surrogate	a raw cédula / document number
age_band	bucket an age (e.g. 10-yr bands, 90+)	age_years
generalize_year	reduce a date to its year
redact	blank the value
drop	remove the column entirely

Preview the full plan before running anything with deidkit plan (§6).

---

5. Outputs — what you get back

A run always produces two folders, kept separate on purpose so that zipping the deliverable can never leak the key:

deid_out/                         ← SHIP THIS — de-identified data only
    tbl_patient.csv
    tbl_encounter.csv
    tbl_notes.csv
    tbl_lab.csv
    tbl_imaging.csv
    tbl_medication.csv

deid_out-PRIVATE/                 ← NEVER SHARE — the re-identification key
    deidkit-secret.key            ← the secret (only if it was auto-generated)
    mapping.private.json          ← value → surrogate map (reverses/extends the run)
    deid_audit.PRIVATE.xlsx       ← before/after audit workbook (see §6)
    deid_audit.PRIVATE_freetext_changes.csv
    review.csv                    ← uncertain detections for a human (see §7)

• deid_out/ — the pseudonymized tables, same structure as the input. This is the only thing you deliver.
• deid_out-PRIVATE/ — the secret + mapping + audit. Whoever holds these can re-identify. Store them somewhere else and never include them in a delivery.

Output format defaults to CSV; use --format parquet|tsv|json (CLI) or fmt= (Python) to change it. Reading from a DB and want the result written back to a DB? See run-db in §9.

---

6. Intermediate outputs — inspect before you trust

Don't run blind. deidkit gives you three ways to look between input and final output.

deidkit plan — the decision for every column (changes nothing)

deidkit plan data/

table            column                           strategy         decided_by
----------------------------------------------------------------------------------------
tbl_notes        note_full                        freetext         heuristic:freetext  *
tbl_lab          result_interpretation            freetext         heuristic:freetext  *
tbl_lab          order_datetime                   date_shift       heuristic:date  *
tbl_patient      sex                              passthrough      default

Columns marked * will change. If something is wrong, fix it with --ignore, --only, --name-column, or a rule, then re-run plan until it's right.

In code: deid.plan_dataframe(df, "tbl_notes") returns the same {column: PlanItem} decision for one table.

deidkit scan — what the free-text detector sees on a snippet

deidkit scan --text "Paciente Juan Pérez, CC 1.032.456.789, tel [phone]"

accepted  type         conf  detectors              text
--------------------------------------------------------------------------------
REPLACE   PERSON       0.99  context+gazetteer      'Juan Pérez'
REPLACE   NATIONAL_ID  0.99  regex                  '1.032.456.789'
REPLACE   PHONE        0.99  regex                  '300 123 4567'

REPLACE = will be scrubbed. review = queued for a human instead.

In code: dk.Detector("es", "conservative").detect(text) returns the same detections.

The audit workbook — evidence of every change

deid_out-PRIVATE/deid_audit.PRIVATE.xlsx has:

sheet	what it shows
Summary	counts by entity type and column
Field plan	the strategy chosen for every column, and why
Free-text changes	every before → after edit inside notes/reports, and which detector fired
Structured changes	before → after for structured columns (dates, IDs, names)
Mapping	the private value → surrogate map
Review queue	detections that were not auto-replaced

Verify intervals survived (they should, exactly):

import pandas as pd
enc = pd.read_csv("deid_out/tbl_encounter.csv")
los = pd.to_datetime(enc["encounter_end"]) - pd.to_datetime(enc["encounter_start"])
print(los.describe())   # identical to the source — only absolute dates moved

---

7. Training it — the review → learn loop

deidkit deliberately does not guess on ambiguous names. Instead it puts them in review.csv, and you teach it. This gets better every pass, is fully auditable, and uses no black-box model.

# 1. Run with a learn directory. Uncertain detections go to review.csv.
deidkit run data/ deid_out --learn-dir learned/

# 2. A human opens deid_out-PRIVATE/review.csv and sets the first column:
#       y  = real PII      (learn it as a name  → higher recall next time)
#       n  = false positive (learn it as a stopword → higher precision next time)

# 3. Fold those decisions into the learned dictionaries.
deidkit learn --review deid_out-PRIVATE/review.csv --learn-dir learned/

# 4. Re-run. Confirmed names are now caught; rejected ones are suppressed.
deidkit run data/ deid_out --learn-dir learned/

learned/ accumulates learned_names.txt and learned_stoplist.txt. Point every future run at the same --learn-dir and it auto-loads them.

In code: pass the learned files onto the policy — Policy(extra_name_files=["learned/learned_names.txt"], extra_stoplist_files=["learned/learned_stoplist.txt"]).

Two stronger levers (deterministic, near-100% recall)

• Known-PHI columns — if you already know a row's real name or ID (from a structured column), force-scrub those exact strings from that row's free text. No guessing:

  # CLI
  deidkit run data/ deid_out \
      --known-name-column patient_name \
      --known-id-column patient_id
  

  # code
  dk.Policy(known_name_columns=["patient_name"], known_id_columns=["patient_id"])
  

• Extend the name lists — feed your own gazetteer / stoplist:

  # CLI
  deidkit run data/ deid_out \
      --extra-name-file my_surnames.txt \
      --extra-stoplist-file never_redact.txt
  

  # code
  dk.Policy(extra_name_files=["my_surnames.txt"],
            extra_stoplist_files=["never_redact.txt"])
  

---

8. Tuning recall & precision

Lever	What it does
--mode conservative (default)	replace only clear names; queue the rest to review.csv
--mode balanced / aggressive	replace more eagerly (higher recall, more to review)
--spacy-model es_core_news_lg	add a statistical NER second opinion (needs deidkit[ner])
--id-checksums DNI,NIE	catch bare national IDs (no cue word) with near-zero false positives
--name-style tagged|placeholder|token|realistic	how fakes look — tagged = "Juan Gómez [SINTÉTICO]" (obvious); realistic = unmarked
--no-medical-vocab	(advanced) stop treating ~21k drug/disease terms as safe non-names

Start conservative, work the review queue down, and turn on the NER stage only if you need more recall — the audit is how you prove the result.

In code, the same knobs are Policy fields:

dk.Policy(mode="balanced", spacy_model="es_core_news_lg",
          id_checksums=["DNI", "NIE"], name_style="tagged")

---

9. Reading from a database (Fabric / Azure SQL)

How we get access to your system

We do not log into your Azure portal or touch your production database. The flow is the standard Fabric mirror share:

1. You (the data owner) mirror the Azure SQL Database into a Microsoft Fabric workspace and share the mirrored database with us as read-only — grant Read + Read all SQL analytics endpoint data, nothing else. (This is the setup in the onboarding document; no write or admin access is ever needed.)
2. You send us four connection details (listed in §10) — the SQL analytics endpoint, the database name, the authentication method, and the workspace URL.
3. We connect to that read-only SQL endpoint from our side using those details, read the six tables, de-identify them, and write out the clean copy. deidkit never writes back to your database — it only reads.

So the only thing you provide is read-only access plus those four details; the de-identification runs entirely on our side against the shared endpoint.

Then it runs — two patterns

The database drivers ship with deidkit (§1). Pick based on where you want the output.

Pattern 1 — Database → files (recommended, simplest)

Read the tables from Fabric into memory, de-identify them together, write clean files. Python only:

import deidkit as dk
from deidkit.io import read_sql_table

CONN = (
    "mssql+pyodbc://@<sql-endpoint>.datawarehouse.fabric.microsoft.com/<database>"
    "?driver=ODBC+Driver+18+for+SQL+Server&authentication=ActiveDirectoryInteractive"
)

TABLES = ["tbl_patient", "tbl_encounter", "tbl_notes",
          "tbl_lab", "tbl_imaging", "tbl_medication"]

# Read all tables so they share one map + one set of per-patient date offsets
tables = {t: read_sql_table(CONN, t) for t in TABLES}

result = dk.deidentify(
    tables,
    out="deid_out",
    secret_file="secret.key",
    lang="es",
    entity_key="patient_id",
    known_id_columns=["patient_id"],
)
print(result.summary)
# -> deid_out/*.csv  (ship)   +   deid_out-PRIVATE/  (keep)

Pattern 2 — Database → database (run-db / deidentify_database)

Write the de-identified result into a separate output database that mirrors the source structure (same tables, columns, types). You must provision the output DB yourself — deidkit refuses to run without it and never edits the source.

# CLI
deidkit run-db \
    --source "mssql+pyodbc://@<sql-endpoint>.datawarehouse.fabric.microsoft.com/<database>?driver=ODBC+Driver+18+for+SQL+Server&authentication=ActiveDirectoryInteractive" \
    --out-db "postgresql://deid_user:PASSWORD@your-host:5432/deid_db" \
    --policy examples/policy.meridian.yaml \
    --secret-file secret.key

# code
import deidkit as dk

dk.deidentify_database(
    source="mssql+pyodbc://@<sql-endpoint>.datawarehouse.fabric.microsoft.com/<database>"
           "?driver=ODBC+Driver+18+for+SQL+Server&authentication=ActiveDirectoryInteractive",
    out_db="postgresql://deid_user:PASSWORD@your-host:5432/deid_db",
    secret_file="secret.key",
    lang="es",
    entity_key="patient_id",
    known_id_columns=["patient_id"],
)

• --out-db / out_db is required and must differ from the source.
• A column whose de-identified value no longer fits the source type (e.g. an age turned into a band) is widened to text; every other column keeps its structure.
• The secret + mapping + audit go to ./deidkit-db-PRIVATE/ (never the output DB).

---

10. Database connection parameters — exactly what to fill in

What you (the data owner) send us

After sharing the mirrored database read-only, send us these four items (they are exactly what the Fabric "share" step produces):

#	Item	Example	Used for
1	SQL Analytics Endpoint (the host)	abc123....datawarehouse.fabric.microsoft.com	connecting
2	Database name	ProductionMirror	connecting
3	Authentication method — Microsoft Entra ID or SQL Login	Microsoft Entra ID	connecting
4	Fabric Workspace URL	https://app.fabric.microsoft.com/groups/...	locating & confirming the shared database

Items 1–3 go into the connection string; item 4 is how we find and verify the shared item in Fabric (it is not part of the connection string).

How we assemble the connection string

Connections are SQLAlchemy URLs. For the Fabric source, the shape is:

mssql+pyodbc://@<HOST>/<DATABASE>?driver=<DRIVER>&authentication=<AUTH>

Placeholder	Filled from	Example
<HOST>	item 1 — the SQL analytics endpoint you send us	abc123....datawarehouse.fabric.microsoft.com
<DATABASE>	item 2 — the database name you send us	ProductionMirror
<AUTH>	item 3 — your authentication method	ActiveDirectoryInteractive
<DRIVER>	installed on our machine (§1), not sent by you	ODBC+Driver+18+for+SQL+Server

Note the @ before <HOST>: ...pyodbc://@abc123.... With Entra ID auth there is no username/password in the URL — the @ with an empty user is intentional. Spaces in the driver name are written as + (URL-encoded).

Authentication options for <AUTH>:

• Microsoft Entra ID, interactive (opens a browser to sign in) — best for a person running it manually:

  ...?driver=ODBC+Driver+18+for+SQL+Server&authentication=ActiveDirectoryInteractive
  

• Microsoft Entra ID, service principal (unattended jobs / servers):

  mssql+pyodbc://<CLIENT_ID>:<CLIENT_SECRET>@<HOST>/<DATABASE>?driver=ODBC+Driver+18+for+SQL+Server&authentication=ActiveDirectoryServicePrincipal
  

• SQL login (only if they gave you a SQL username/password instead of Entra):

  mssql+pyodbc://<USER>:<PASSWORD>@<HOST>/<DATABASE>?driver=ODBC+Driver+18+for+SQL+Server
  

Install the ODBC driver (<DRIVER> above must match what’s installed):

# macOS
brew tap microsoft/mssql-release https://github.com/microsoft/homebrew-mssql-release
brew install msodbcsql18

# Debian/Ubuntu — follow Microsoft’s "ODBC Driver 18 for SQL Server" apt instructions
# Windows — install the "ODBC Driver 18 for SQL Server" MSI

The output DB URL (Pattern 2 only) is any SQLAlchemy URL for a database you created and can write to:

Target	URL shape
PostgreSQL	postgresql://user:password@host:5432/deid_db
Azure SQL / SQL Server	mssql+pyodbc://user:password@host/deid_db?driver=ODBC+Driver+18+for+SQL+Server
Local SQLite (quick test)	sqlite:///deid_local.db

Quick connectivity check before a full run:

from sqlalchemy import create_engine, text
CONN = "mssql+pyodbc://@<HOST>/<DATABASE>?driver=ODBC+Driver+18+for+SQL+Server&authentication=ActiveDirectoryInteractive"
with create_engine(CONN).connect() as c:
    print(c.execute(text("SELECT TOP 1 patient_id FROM tbl_patient")).fetchone())

---

11. What to look out for

• Is patient_id a raw ID or already hashed? By default it's passed through (it's the join key). If it's actually the cédula / national ID, that's a leak — add FieldRule("patient_id", "identifier") (or uncomment the rule in examples/policy.meridian.yaml). Surrogating it still preserves joins.
• Filenames & paths can hide PHI. Columns like result_file_name and image_relative_uri often embed a name or MRN. They pass through by default — redact them if you don't need them.
• Keep the secret and mapping private. They are the re-identification key. Never put *-PRIVATE/ in a delivery. Losing the secret means you can't re-run consistently; leaking it undoes the de-identification.
• Determinism needs the same secret. Pass the same --secret-file every time, or every run produces different surrogates.
• No detector is perfect. Pseudonymization reduces but does not eliminate re-identification risk (per NIST SP 800-188 / HIPAA expert-determination). Review the audit, consider quasi-identifier combinations (age + stratum + location …), and treat outputs accordingly.

---

12. Full option reference (CLI + Python)

CLI commands

Command	Purpose
deidkit init-secret --out secret.key	generate a keying secret (chmod 600)
deidkit plan <input>	show the strategy for every column; change nothing
deidkit run <input> [out]	de-identify files → files
deidkit run-db --source <url> --out-db <url>	de-identify a DB → a separate DB
deidkit scan --text "…"	show detections on a snippet (tuning/debug)
deidkit learn --review review.csv --learn-dir learned/	fold review decisions into learned dicts
deidkit map --mapping mapping.private.json	dump the private value→surrogate map

Run deidkit <command> --help for any command's flags.

CLI flags shared by run, run-db, plan, scan

Flag	Meaning
--policy FILE	YAML policy file; CLI flags override it
--lang {es,en,multi}	language for names/detection (multi = ES+EN)
--mode {conservative,balanced,aggressive}	free-text detection aggressiveness
--name-style {tagged,placeholder,token,realistic}	how fake names look (default tagged = obviously synthetic)
--entity-key COL	column that groups date shifts (default patient_id)
--date-max-days N	date-shift window in days (default 365; intervals preserved regardless)
--ignore a,b,c	never transform these columns
--only a,b	transform only these columns
--name-column COL	force a column to synthetic-name treatment (repeatable)
--known-name-column COL	column holding a known name to force-scrub from that row’s text (repeatable)
--known-id-column COL	column holding a known ID to force-scrub from that row’s text (repeatable)
--known-name "NAME"	a known name (e.g. clinic staff) scrubbed from all text (repeatable)
--id-checksums LIST	checksum-validated bare-ID detection: DNI,NIE (ES), CPF,CNPJ,CNS (BR), NHS (UK), CURP,RFC (MX), SSN, or ALL
--spacy-model NAME	enable the spaCy NER stage (e.g. es_core_news_lg)
--names-token	use [PERSON_x] tokens instead of realistic fake names
--no-date-text	do not shift date strings found inside free text
--no-medical-vocab	do not load the bundled medical safe-vocabulary
--extra-name-file FILE / --extra-stoplist-file FILE	merge extra gazetteer / stoplist files (repeatable)
--learn-dir DIR	auto-load learned_names.txt + learned_stoplist.txt from a previous learn

run-specific flags

Flag	Meaning
input / out (positional)	input path/glob/dir, and output dir (default <input>_deid)
--secret-file FILE / --secret STR	the keying secret (or set DEIDKIT_SECRET)
--schema FILE	data-dictionary JSON (auto-detected from the input folder if omitted)
--mapping FILE	private mapping JSON (default <out>-PRIVATE/mapping.private.json)
--report FILE	audit workbook path (default <out>-PRIVATE/deid_audit.PRIVATE.xlsx)
--format {csv,parquet,tsv,json}	output format (default csv)

run-db-specific flags

Flag	Meaning
--source URL	source DB SQLAlchemy URL (read-only) — required
--out-db URL	output DB SQLAlchemy URL you created — required, must differ from source
--tables a,b,c	limit to these tables (default: all)
--out-schema NAME	schema/namespace in the output DB
--if-exists {fail,replace,append}	behaviour if an output table exists (default fail)
--private-dir DIR	where secret+mapping+audit go (default ./deidkit-db-PRIVATE)

Python Policy fields

entity_key (default "patient_id"), date_max_days (365), lang ("es"), mode ("conservative"), name_style ("tagged"), shift_dates_in_text (True), only, ignore, rules (list[FieldRule]), spacy_model, known_name_columns, known_id_columns, extra_known_names, known_token_min_len (5), id_checksums, use_medical_vocab (True), extra_name_files, extra_stoplist_files.

FieldRule(column, strategy, table=None, options={}) — column and table accept * globs; the first matching rule wins.

Python functions

import deidkit as dk

# one call: files / DataFrame / dict of DataFrames -> Result
dk.deidentify(data, out=None, *, policy=None, schema=None, secret=None,
              secret_file=None, mapping=None, report=None, private_dir=None,
              fmt=None, table="table", **policy_kwargs)

# one DataFrame in, one clean DataFrame out
dk.deidentify_dataframe(df, table="table", **kwargs)

# database -> a separate database (mirrors source structure)
dk.deidentify_database(source, out_db, *, tables=None, schema=None,
                       out_schema=None, if_exists="fail", policy=None,
                       secret=None, secret_file=None, mapping=None,
                       private_dir=None, report=None, **policy_kwargs)

deidentify(...) returns a Result with .tables, .table (the single DataFrame when one went in), .summary, .out, .report_path, .mapping_path, .private_dir, and .secret (set only when a secret was auto-generated — save it to stay reproducible).

Full control — drive many tables through one shared map + one set of per-patient date offsets, and write the audit yourself:

import deidkit as dk

policy = dk.Policy(lang="es", mode="conservative", entity_key="patient_id",
                   known_id_columns=["patient_id"], ignore={"blood_type"})

deid = dk.Deidentifier(
    policy,
    secret_file="secret.key",                              # or secret="…", or $DEIDKIT_SECRET
    mapping_path="deid_out-PRIVATE/mapping.private.json",  # load+update for consistency
    # dict_index=dk.load_dictionary("dictionary.json"),   # optional data dictionary
)

tables = dk.io.read_tables("data/")          # {name: DataFrame}
clean = deid.run_dataset(tables)             # {name: DataFrame}, all sharing state
# or a single table:  deid.run_table(df, table="tbl_notes")

deid.save_mapping()
deid.write_report("deid_out-PRIVATE/deid_audit.PRIVATE.xlsx")
deid.write_review("deid_out-PRIVATE/review.csv")
print(deid.summary())

---

13. Troubleshooting

Symptom	Fix
ModuleNotFoundError: pyodbc / SQLAlchemy	Reinstall: pip install --force-reinstall deidkit (the DB drivers ship with it)
Data source name not found / driver error	Install ODBC Driver 18 for SQL Server and make the driver= value match exactly (§10)
Browser never opens for ActiveDirectoryInteractive	Run from an interactive session; for servers use ActiveDirectoryServicePrincipal
cannot reach the output database (run-db)	Create the empty output DB first and pass a reachable --out-db URL
A column was changed that shouldn’t be	Add it to --ignore, or add a FieldRule(..., "passthrough"); confirm with deidkit plan
A free-text name slipped through	Try --mode balanced, enable --spacy-model, or add the name via the review→learn loop (§7)
Non-UTF-8 CSV export	Handled automatically (deidkit falls back to latin-1/cp1252)
Different output each run	You’re not pinning the secret — pass the same --secret-file every time (§2)

---

Publishing to PyPI

deidkit is not on PyPI yet, so pip install deidkit will only work once you publish it under your own PyPI account. The package already builds cleanly into a wheel (verified). To publish:

# 0. one-time: make a PyPI account at https://pypi.org and create an API token.
#    Also edit the [project.urls] in pyproject.toml (they still say "your-org").

pip install build twine          # the standard build + upload tools

python -m build                  # produces dist/deidkit-0.1.0-py3-none-any.whl (+ .tar.gz)
python -m twine check dist/*     # validates the package metadata

# optional dry run on the test index first:
python -m twine upload --repository testpypi dist/*

# the real publish (asks for your PyPI token):
python -m twine upload dist/*

After that first upload, pip install deidkit works for everyone, and the name deidkit is yours (it is currently unclaimed on PyPI). Bump version in pyproject.toml for each subsequent release — PyPI refuses to overwrite an existing version.

---

License

MIT.