kanonym4text K-anonymity guerantee for texts
K Anonymity for text (or in short - kanonym4text) is an open‑sourced library that receives a corpus from the user (dataframe) and returns a K-anonymized corpus with additional information about the anonymization process performed. kanon4txet is designed to be easily utilized, to guarantee anonymization at a certain level pre-defined by the user (k) while still preserving some of the text utilization properties. This repo and package are part of our Y-Data data science final project, and we would love to hear your feedback and learn from it!
Overview
In this project, we aim to apply data science techniques to anonymize textual data while preserving their utility. K-Anonymity is a technique used to ensure that an individual in a dataset cannot be identified by linking their attributes to external information, by forcing each row to be identical to k-1 other rows The anonymized data can be used for various purposes like data sharing, research, and analysis without compromising privacy. We plan on creating a novel algorithm for k-anonymity. Specifically, we address the case of unstructured data items, such as texts. Using various NLP techniques, from classical to modern DL-based solutions, and testing the utility of the anonymized data. We have tested the library on two main datasets:
- Amazon dataset,
- Enron emails dataset
We show it is able to generate anonymized corpora in both cases.
Package High-Level Algorithm:
- Data Preprocessing: tokenization, lemmatization, and stop word removal.
- K-Anonymization: Generalization and Reduction.
- Evaluation: Utilization is evaluated by embedding distance and semantic scores.
- Visualization: dataset properties prior to and post anonymization will be visualized (optional)
Getting Started
You can get started with kanonym4text immediately by installing it with pip:
download package
pip install kanonym4text
Step 1 - creat a data frame from your corpus
The code receives a data frame containing the corpus in the following format: "txt" - column with the texts (default column name)
Step 2 - import the following and initializethe object
Use gensim word embedding model for you choice (default - glove-twitter-25)
from kanonym4text import Kanonym
kan = Kanonym()
Step 3 - read the df:
import pandas as pd
df = pd.read_csv('YOUR_FILE.csv')
Step 4 - run the anonymization process on your corpus
dfa, dist = kan.anonymize(df, k=2)
Running instructions: The main function is called anonymize.
It's input parameters are:
df- Input Dataframek- kcol- The column in df that holds the text to anonymize. Default - txtnum_stop- Number of stop word to use. Default - 1000num_jobs- Number of CPUs to utilize. Default - 1. All CPUs - -1.verbose- Output text level. Default - 0.
The function outputs are:
A tuple with these items:
- A dataframe - the same df the user sent with additional columns:
- general_txt - text after "generalization"
- anon_txt_history - changes performed on text during annonymization process:
[] - replaced
{} - Lemmatize
() - protected word (stop-word)- anon_txt - resulted anonymized text
- neighbors - indeces of k neigbors (Bow anonymized)
- An averaged cosine distance of sentence embedding for the documents before and after the anonymizaion
Running Example
use the following link to run some examples of the package on your own dataframe:
or use the following code:
from kanonym4text import Kanonym
import pandas as pd
df = pd.read_csv('YOUR_FILE.csv')
kan = Kanonym('glove-twitter-25') # creating an object from class
dfa, dist = kan.anonymize(df, k=k, n_jobs=-1, plot=True)
Support
Create a Bug Report
If you see an error message or run into an issue, please create a bug report. This effort is valued and helps all users.
Submit a Feature Request
If you have an idea, or you're missing a capability that would make development easier and more robust, please Submit a feature request.
If a similar feature request already exists, don't forget to leave a "+1". If you add some more information such as your thoughts and vision about the feature, your comments will be embraced warmly :)
Contributing
Kanon4txt is an open-source project. We are committed to a fully transparent development process and highly appreciate any contributions. Whether you are helping us fix bugs, proposing new features, improving our documentation, or spreading the word - we would love to have you!
Metadata
Release files for kanonym4text 0.2.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| kanonym4text-0.2.7.tar.gz | 40.5 kB | Without provenance | Details for kanonym4text-0.2.7.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| kanonym4text-0.2.7-py3-none-any.whl | Python 3 | none | any | Details for kanonym4text-0.2.7-py3-none-any.whl |
Total release size: 81.1 kB
Release files / kanonym4text-0.2.7.tar.gz
| Download URL | kanonym4text-0.2.7.tar.gz |
|---|---|
| Size | 40.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6d07f44c258f5cfb534d5c056b885f57394d4bb038bbbc28c72aa0870836ae3b
|
|
BLAKE2b-256 checksum How to use checksums |
eb0f1c0e25ae4c1a40ddab6c04874d619b8199b8425642e1d35cd01399c70fa6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.8.10
|
Release files / kanonym4text-0.2.7-py3-none-any.whl
| Download URL | kanonym4text-0.2.7-py3-none-any.whl |
|---|---|
| Size | 40.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
053cf8a4ae714cae323626d262d79594fde4b98407e80d4a29cecca61c04dd58
|
|
BLAKE2b-256 checksum How to use checksums |
a014b2a2a79541dbe8532825228dd450646315f13e7319d6971ab34a7007272f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.8.10
|
Security for kanonym4text 0.2.7
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.