Skip to main content

Metasyn disclosure control

Python package Project Status: WIP – Initial development is in progress, but there has not yet been a stable, usable release suitable for the public.

A privacy plugin for metasyn, based on statistical disclosure control (SDC) rules of thumb as found in the following documents:

  • The SDC handbook of the Secure Data group in the UK
  • The EU-funded Data Without Boundaries document on Guidelines for output checking (pdf)
  • Statistics Netherlands' statistical disclosure control guidelines for microdata output

Producing synthetic data with metasyn is already a great first step towards protecting privacy, but it doesn't adhere to official standards. For example, fitting a uniform distribution will disclose the lowest and highest values in the dataset, which may be a privacy issue in particularly sensitive data. This plugin solves these kinds of problems.

Installing the plugin

To install the package with pip, run the following:

pip install metasyn-disclosure

For the development, installed the package directly through git with the following command:

pip install git+https://github.com/sodascience/metasyn-disclosure-control.git

Usage

Basic usage for our built-in titanic dataset is as follows:

from metasyncontrib.disclosure import DisclosurePrivacy
from metasyn.distribution import FakerDistribution

from metasyn import MetaFrame, VarSpec, demo_dataframe

df = demo_dataframe("titanic")

spec = [
    VarSpec(name="PassengerId", unique=True),
    VarSpec(name="Name", distribution=FakerDistribution("name")),
]

mf = MetaFrame.fit_dataframe(
    df=df,
    var_specs=spec,
    privacy=DisclosurePrivacy(),
)

mf.synthesize(5)
shape: (5, 13)
┌─────────────┬────────────────────┬────────┬──────┬───┬────────────┬────────────┬─────────────────────┬────────┐
│ PassengerId ┆ Name               ┆ Sex    ┆ Age  ┆ … ┆ Birthday   ┆ Board time ┆ Married since       ┆ all_NA │
│ ---         ┆ ---                ┆ ---    ┆ ---  ┆   ┆ ---        ┆ ---        ┆ ---                 ┆ ---    │
│ i64         ┆ str                ┆ cat    ┆ i64  ┆   ┆ date       ┆ time       ┆ datetime[μs]        ┆ f32    │
╞═════════════╪════════════════════╪════════╪══════╪═══╪════════════╪════════════╪═════════════════════╪════════╡
│ 0           ┆ Benjamin Cox       ┆ female ┆ 27   ┆ … ┆ 1931-12-01 ┆ 14:33:06   ┆ 2022-07-30 02:16:37 ┆ null   │
│ 1           ┆ Mr. David Robinson ┆ female ┆ null ┆ … ┆ 1906-02-18 ┆ null       ┆ 2022-08-03 13:09:19 ┆ null   │
│ 2           ┆ Randy Mosley       ┆ male   ┆ 24   ┆ … ┆ 1933-01-06 ┆ 15:52:54   ┆ 2022-07-18 18:52:05 ┆ null   │
│ 3           ┆ Vincent Maddox     ┆ female ┆ 24   ┆ … ┆ 1937-02-10 ┆ 16:58:30   ┆ 2022-07-23 20:29:49 ┆ null   │
│ 4           ┆ Kristin Holland    ┆ male   ┆ 17   ┆ … ┆ 1939-12-09 ┆ 18:07:45   ┆ 2022-08-05 02:41:51 ┆ null   │
└─────────────┴────────────────────┴────────┴──────┴───┴────────────┴────────────┴─────────────────────┴────────┘

Implementation details

The rules of thumb, roughly, are:

  • at least 10 units
  • at least 10 degrees of freedom
  • no group disclosure
  • no dominance

For most distributions, we implemented micro-aggregation. This technique pre-averages a sorted version of the data, which then supplied to the original fitting mechanism. The idea is that during this pre-averaging step, we ensure that the rules of thumb are followed, so that the fitting method doesn't need to do anything in particular. While from a statistical point of view, we are losing more information than we probably need, it should ensure the safety of the data.

Contributing

You can contribute to this metasyn plugin by giving feedback in the "Issues" tab, or by creating a pull request.

To create a pull request:

  1. Fork the Project
  2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
  3. Commit your Changes (git commit -m 'Add some AmazingFeature')
  4. Push to the Branch (git push origin feature/AmazingFeature)
  5. Open a Pull Request

Contact

This is a project by the ODISSEI Social Data Science (SoDa) team. Do you have questions, suggestions, or remarks on the technical implementation? File an issue in the issue tracker or feel free to contact Raoul Schram or Erik-Jan van Kesteren.

SoDa logo

Metadata

Release files for metasyn-disclosure 0.3.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for metasyn-disclosure 0.3.3
File Size Uploaded Provenance Actions
metasyn_disclosure-0.3.3.tar.gz 447.2 kB Without provenance Details for metasyn_disclosure-0.3.3.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for metasyn-disclosure 0.3.3
File Interpreter ABI Platform Actions
metasyn_disclosure-0.3.3-py3-none-any.whl Python 3 none any Details for metasyn_disclosure-0.3.3-py3-none-any.whl

Total release size: 468.1 kB

Release files / metasyn_disclosure-0.3.3.tar.gz

Download URL metasyn_disclosure-0.3.3.tar.gz
Size 447.2 kB
Tags Source
SHA-256 checksum
How to use checksums
6206eb79eb6de7a327e6490fb8b95b4dd618db369f3c43098742a267229a47db
BLAKE2b-256 checksum
How to use checksums
c3f22455d0dec48012486a020e9476c7836f7f34a1471d99a921a1efd2b58bc6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / metasyn_disclosure-0.3.3-py3-none-any.whl

Download URL metasyn_disclosure-0.3.3-py3-none-any.whl
Size 20.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
7f906a76698a4b3a436088cd0304487301a56488c7a7ea5de6945961adc50988
BLAKE2b-256 checksum
How to use checksums
424e217c07d05f7b46d6b447d4b07f9146f57261ca59b889409ce6827a076461
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for metasyn-disclosure 0.3.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page