Skip to main content

A utility to obfuscate and mask elements in XML and JSON files

Project description

#XML AND JSON MASKING

For detailed information please visit = https://sonra.io/2019/04/01/paranoid-masking-anonymizing-and-obfuscating-pii-in-xml-and-json-data/

##About

Paranoid is data masking and obfuscation command line tool for XML and JSON file formats. Paranoid is best used in combination with Flexter. Flexter is Sonra's XML converter for complex XML and JSON based on industry data standards such as ACORD, HL7, FHIR, NDC, XBRL, FpML etc. It converts XML to any relational database, Hadoop formats (ORC, Parquet, Avro, Hive, Impala), or text (TSV, CSV).

##Features

  • Works with one or more XML/JSON document(s). If input path points to a directory - processes its content recursively. Auto detects the format of each file.
  • Masks all the elements/attributes in the XML/JSON document(s) by default while preserving the exact structure of the file(s).
  • Can also mask only specific elements (by provided path/XPath) in XML/JSON document(s).
  • Universal: runs on both: Python 2.6+ or 3.6+
  • Offline tool - runs locally on your system. No data gets transferred anywhere.
  • Open source - anyone can examine what it does to make sure the data can't be successfully de-encoded back after leaving The Sausage Machine. Any contributions are welcome!
  • Easy installation - can download script itself or use pip

##Advanced Features

  • Custom built Parser - simple parser that does only what needs to be done. Removes the overhead of using external libs. It's fast. It doesn't validate documents so can work with some rough edged ones …to some extent.
  • Smart Buffering - easy on memory (redefinable buffer to use, 2MB default) but at the same time works with huge files (eg. 10GB). Works with them even if all the content is lumped into a single line 💪! *Masking Statistics - Provides stats for number of xml tags and number of tags masked in during the operation which one can store in a log file too.

##Architecture Architecture

##Installation

###pip install PARANOID

#Instructions

usage: paranoid [-h] -i INPUT [-b BYTESIZE] -o OUTPUTDIR

Data Masking

optional parameters:

-h, --help shows this help message and exit

-i INPUT Input Directory Name / File Name

-b BYTESIZE (re)Defines buffer size (2MB default)

-o OUTPUTDIR Output Directory Name

paranoid -h

Usage

##Usage Examples

Mask Single File

paranoid -i <input filename> -o <output directory name>

SingleFile

Mask all XML and JSON files in a Directory

paranoid -i <directory name> -o <output directory name>

MultipleFile

Change Buffer size

paranoid -i <File or directory name> -o <output directory name> -b buffersize

BufferLimit

That's the way to ingest big fat one liners as it analyses your file by streamig it byte by byte, buffer by buffer.

Mask Certain Tags

paranoid -i <input filename> -o <output directory name> -l xpath separated by ,

SpecificTags

Generate Log File

paranoid -i <input filename> -o <output directory name> -L Log File Location

Generate Log File

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for PARANOID, version 1.1.5
Filename, size & hash File type Python version Upload date
PARANOID-1.1.5-py2.py3-none-any.whl (9.6 kB) View hashes Wheel py2.py3

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page