cloakdata 2.0.0

Fast data anonymization with Polars

🔐 CloakData — Data Anonymizer

A flexible and extensible data anonymization library built on Polars. Designed for privacy, compliance, and testing with minimal overhead.

---

🧾 What’s New (2.0.0)

• 🔧 Improved all masking and transformation methods for consistency and safety.
• ✨ Standardized method signatures to return pl.Expr for better composability.
• 🗓️ Added round_date to round dates to month or year start.
• 🛡️ Improved parameter handling (defaults, null-safety, predictable behavior).
• 🧪 Refactored and validated tests to ensure stability across changes.
• 📚 Improved documentation and moved detailed examples into examples/.

---

✨ Features

• 🔒 Masking: full, partial, emails, phone numbers.
• 🔄 Replacement: static values, dictionaries, substrings.
• 🔢 Sequential IDs: numeric or alphabetical.
• ✂️ Truncation & initials extraction.
• 📊 Generalization: ages into ranges, dates into month/year.
• 🎲 Randomization: choices, digits, shuffling.
• 📅 Date offsetting with reproducible seeds.
• 🧩 Conditional rules — multi-rules, nested (all/any/not), logical groups (and/or).
• ⚡ Built on Polars → fast & scalable.

---

⚙️ How it works

1. Load your dataset into a Polars DataFrame.
2. Define anonymization rules in a simple JSON config.
3. Call anonymize(df, config) → get a safe anonymized DataFrame.

---

🧪 Example Config

{
  "columns": {
    "name": { "method": "initials_only" },
    "email": { "method": "mask_email" },
    "phone": { "method": "mask_number" },
    "cpf": {
      "method": "replace_with_random_digits",
      "params": { "digits": 11 }
    },
    "status": {
      "method": "replace_exact",
      "params": { "mapping": { "active": "A", "inactive": "I" } }
    },
    "id_seq": { "method": "sequential_numeric", "params": { "prefix": "ID" } },
    "ref_code": { "method": "sequential_alpha", "params": { "prefix": "REF" } },
    "comments": { "method": "truncate", "params": { "length": 5 } },
    "age": { "method": "generalize_age" },
    "birth_date": { "method": "generalize_date", "params": { "mode": "month_year" } },
    "state": { "method": "random_choice", "params": { "choices": ["SP","RJ","MG","BA"] } },
    "last_access": { "method": "date_offset", "params": { "min_days": -2, "max_days": 2 } },
    "feedback": { "method": "shuffle" }
  }
}

---

🧠 Conditional Rules

Apply transformations only when conditions are met.

Single condition

"cpf": {
  "method": "replace_with_random_digits",
  "params": { "digits": 11 },
  "condition": {
    "column": "status",
    "operator": "equals",
    "value": "active"
  }
}

Multiple rules per column

"city": [
  { "method": "replace_with_value", "params": { "value": "X" } },
  {
    "method": "mask_partial",
    "params": { "visible_start": 1, "visible_end": 1 },
    "condition": { "column": "country", "operator": "equals", "value": "BR" }
  }
]

Nested conditions

"age": {
  "method": "generalize_age",
  "condition": {
    "all": [
      { "column": "country", "operator": "equals", "value": "BR" },
      { "any": [
          { "column": "status", "operator": "equals", "value": "active" },
          { "column": "status", "operator": "equals", "value": "archived" }
        ]
      }
    ]
  }
}

Operators supported: equals, not_equals, in, not_in, gt, gte, lt, lte, contains, not_contains Groups: all, any, not Logical: and, or

---

🔍 Example Input → Output

Input DataFrame:

name	email	age	status
Alice Smith	[e-mail]	25	active
Bob Jones	[e-mail]	42	inactive

Config:

{
  "columns": {
    "name": { "method": "initials_only" },
    "email": { "method": "mask_email" },
    "age": { "method": "generalize_age" },
    "cpf": {
      "method": "replace_with_random_digits",
      "params": { "digits": 8 },
      "condition": {
        "column": "status",
        "operator": "equals",
        "value": "active"
      }
    }
  }
}

Output DataFrame:

name	email	age	cpf
A.S.	[e-mail]	20-29	48291034
B.J.	[e-mail]	40-49	(null)

---

🧩 Examples

Runnable, self-contained scripts are in the examples/ folder.

• Masking: masking/full_mask.py, masking/mask_email.py
• Replacement: replacement/replace_with_value.py
• Generalization: generalization/generalize_age.py
• Dates: dates/date_offset.py
• Randomization: randomization/random_choice.py
• Utilities: utilities/coalesce_cols.py

---

📊 Supported Methods

Method	Description	Example Input → Output
full_mask	Fixed mask or literal; supports char, len, mask_literal, match_length, preserve_nulls.	12345 → ***** / XXXXXXXX / REDACTED
mask_email	Masks local part; supports mask, fallback_domain, preserve_nulls.	[e-mail] → [e-mail]
mask_number	Keep first N digits, then mask the rest (configurable keep, mask, len, preserve_nulls)	123456789 → 123***** 98765 + keep=2, mask="X" → 98XXX 42 + keep=2, len=4, mask="#" → 42####
mask_partial	Partial masking with configurable visibility	abcdef → a****f (visible_start=1, visible_end=1)
replace_with_value	Replace entire column with a static value (dtype preserved). Optionally keep nulls with preserve_nulls=True. Requires value.	["a", None, "b"] + value="X" → "X","X","X" • preserve_nulls=True → "X", None, "X" • value=123 → 123,123,123
replace_exact	Replace values that exactly match keys in a mapping. Values not in the mapping are unchanged. Dtype is inferred from replacements (no forced Utf8).	["a","b","c"] + {"a":"X"} → ["X","b","c"] • [1,2,3] + {1:99,3:-1} → [99,2,-1] • [True,False] + {True:False} → [False,False]
replace_by_contains	Replace values when they contain given substrings. Literal by default; first match wins; nulls preserved. Options: mapping, substr+replacement, case_sensitive, use_regex.	["foo","bar","baz"] + {"ba":"X"} → ["foo","X","X"] • case_sensitive=False: "Hello" + {"hello":"X"} → "X" • use_regex=True: {"\\d{3}":"HIT"} on "id=123" → "HIT"
replace_with_random_digits	Replace values with randomly generated digit strings (fixed length)	11111 → 80239
sequential_numeric	Sequential numeric pseudonyms with optional prefix (prefix=None → raw integers, default "val")	["Alice","Bob","Alice"] → ["val 1","val 2","val 1"]
sequential_alpha	Sequential alphabetic pseudonyms with optional prefix; duplicates get the same label; order by first appearance	["Alice","Bob","Alice"] → ["val A","val B","val A"]
truncate	Truncates strings to a maximum length (nulls preserved unless configured)	"Porto Alegre" → "Port"
initials_only	Convert names to initials	John Doe → J.D.
generalize_age	Group ages into ranges	25 → 20-29
generalize_date	Generalize dates/datetimes by reducing granularity (year, month, quarter, semester, week, date, datetime)	2025-07-20 → 2025-07 ; 2025-07-20 → 2025-Q3
generalize_number_range	Bucket numeric values into fixed-size ranges (e.g. 0–9, 10–19)	42 → 40-49
random_choice	Replace values with a deterministic pseudo-random choice from a fixed set (null-safe)	São Paulo → X / Y (with seed)
shuffle	Shuffle column values (row order preserved)	[A, B, C] → [B, C, A]
date_offset	Apply a deterministic pseudo-random day offset within a configurable range (null-safe)	2025-07-20 → 2025-07-18
coalesce_cols	Return the first non-null value from a list of columns, respecting priority order	(None, Y) → Y
round_number	Round numeric values to a configurable number of decimal places	3.14159 (digits=2) → 3.14
round_date	Round dates down to the start of a month or year	2025-07-29 → 2025-07-01

---

📂 Project Structure

src/
 └── cloakdata/           # Core library
tests/                    # Test suite (pytest + Polars)
examples/                 # Sample CSVs & configs
README.md                 # Project docs
pyproject.toml            # Build system (uv/hatch)

---

⚡ Installation

pip install cloakdata

Or with uv:

uv add cloakdata

---

🚀 Quickstart

import polars as pl
from cloakdata import anonymize

df = pl.DataFrame({
    "name": ["Alice Smith", "Bob Jones"],
    "email": ["[e-mail]", "[e-mail]"],
    "age": [25, 42]
})

config = {
    "columns": {
        "name": { "method": "initials_only" },
        "email": { "method": "mask_email" },
        "age": { "method": "generalize_age" }
    }
}

out = anonymize(df, config)
print(out)

---

🛠️ Development

git clone https://github.com/youruser/cloakdata
cd cloakdata
uv sync
pre-commit install
pytest -v

---

🔮 Roadmap

• Regex-based redaction
• Hashing strategies (SHA256, bcrypt)
• Parallel processing for large datasets

---

🤝 Contributing

We love contributions! See CONTRIBUTING.md for setup, coding standards, how to add a new anonymization method, tests and the PR checklist.

📄 Notice

See NOTICE for attribution details.

📜 License

MIT © Jeferson Peter