Anonymizer tool for datasets such CSV files
Project description
Anonymizer tool for datasets such CSV files.
To generate fake data, you can choose between two excelent generators:
Install
Using pip:
pip install datanonymizerUsing mimesis instead of the default Faker:
pip install datanonymizer[mimesis]Or from source:
git clone https://github.com/fgmacedo/datanonymizer
cd datanonymizer
python setup.py installUsage
Pass your data through stdin and get it back anonymized on stdout.
cat input_file.csv | datanonymizer >output_file.csvUsing a config file to declare conversions and generators for the required fields:
cat input_file.csv | datanonymizer --config ./dataset_anon_config.yml >output_file.csvPlease see examples folder for a small demo:
cat examples/small.csv | python -m datanonymizer -i --config examples/small_faker.yml --seed my_seed >examples/small_anonymized_using_faker.csvOptional arguments:
-h, --help show this help message and exit
-l LANGUAGE, --language LANGUAGE
Language used by the Generator
-di DELIMITER_INPUT, --delimiter_input DELIMITER_INPUT
CSV delimiter
-do DELIMITER_OUTPUT, --delimiter_output DELIMITER_OUTPUT
CSV delimiter
-i, --ignore_errors Continue on errors
--head HEAD Outputs only the first <HEAD> lines
-g {faker,mimesis}, --generator {faker,mimesis}
Generator library to be used for fake data
--seed SEED Seed for the pseudo random generator providers
--config CONFIG Configuration fileConfig file
You’l need a configuration file to setup transformations for each dataset.
This file is a simple yaml where you can configure fields.
Field names should match the column name declared into the CSV input file.
---
fields:
Task ID:
omit: true
Location:
conversions:
- fn: coords_to_h3
kwargs:
resolution: 8
Client Address:
conversions:
- fn: has_value
rename: has_address
Company Name:
generator:
provider: business.company
rename: company
Invoice ID:
generator:
provider: person.identifier
kwargs:
mask: "#######"
rename: invoiceGenerators
The generatos clause depends of the library you choose to provide fake data.
You can use any generator available at the generic API from Faker or mimesis.
For example, if you wanna mimic data with company names:
Faker
--- fields: Company Name: generator: provider: companyMimesis
--- fields: Company Name: generator: provider: business.company
But you can replace the real names by names of fruits (using Mimesis) or any other provider:
---
fields:
Company Name:
generator:
provider: food.fruitOr generate random integers to replace real IDs:
Faker
--- fields: ID: generator: provider: pyint kwargs: min_value: 1 max_value: 15_000_000Mimesis
--- fields: ID: generator: provider: person.identifier kwargs: mask:"#######"
Conversions
You can apply any pre-configured conversion functions available.
coords_to_h3
has_value
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
