text-mallet 0.1.3

Toolkit for Obfuscating Text Through Transformations.

A package for applying transformations to text for creating derived formats, intended for preventing reconstruction of the original data. This can be applied, for instance, to sensitive training data to prevent its reconstruction while preserving some usefulness of the text in pre-training on primarily encoder-based tasks such as classification and retrieval tasks. Such obfuscated representations are therefore intended to accompany non-sensitive, non-obfuscated data in the pre-training mix where the text's usage would otherwise risk infringing on copyright.

The package natively supports text processing in English and German.

Install

pip install text-mallet

# For obfuscating English, download the English model you want to use 
# The default is the "lg" i.e. large SpaCy model:
python -m spacy download en_core_web_lg

# Or alternatively, download the model for German
python -m spacy download de_core_news_lg

Strategic Obfuscation

The central aim of text obfuscation using this package is to prevent reconstructability. Doing so involves eroding various aspects of text, such as:

• Word Forms: The exact sequence of characters.
• Syntactic Features: Morpho-syntactic token properties.
• Meanings: Core semantic associations.
• Grammatical Relations: Hierarchical sentence dependency tree structure.
• Sequence Information: Flat linear sequence boundaries.

Each layer contributes to the discoverability and reconstructibility of the source text. This package provides four main approaches to obfuscation: POS filtering, mutual information filtering, hierarchical scrambling, and bag-of-words. For a more detailed overview of each method, please see the documentation.

Why Obfuscate Text?

When training models for text generation, we typically need all of the content and style of the original, fluent text. However, there are many tasks such as classification, semantic similarity scoring, topic modelling, and so on, where the original text may not be required in its original form to help model performance. There is typically a trove of public-domain data that can be used for model training, but there are still many questions around the usage of copyright-protected data in training. This package offers a route to preserve some of the value of copyrighted texts while hindering their reconstruction, whether that be through training-data reconstruction or model outputs.

The creation of transformed texts that are thus no longer consumable by humans, but still useful for training on specific tasks.

Impact on Performance

While training on transformed text formats can introduce minor performance drops compared to raw text baselines, text-mallett is explicitly designed to accompany non-obfuscated, public-domain collections. Introducing these anonymised, non-human-consumable formats adds proprietary text signals into your pipeline without exposing the original text to model parameters.

Basic Obfuscation

There are multiple general obfuscation approaches to choose from, separated into three general categories:

• Structural Obfuscation (bag-of-words, hierarchical)
• Part-of-Speech Filtering
• Mutual-Information Filtering

Let's start with an example of bag-of-words scrambling.

from tmallet import TMallet

# 1. Define the Obfuscation Configuration
algorithm = "scramble-BoW"
config = {
    "level": "document",
}

# 2. Define Sample Text
sample = "Leipzig is the most populous city in the German state of Saxony. The city has a population of 633,592 residents as of 31 December 2025. It is the eighth-largest city in Germany and is part of the Central German Metropolitan Region. Leipzig is located about 150 km (90 mi) southwest of Berlin, in the southernmost part of the North German Plain (the Leipzig Bay), at the confluence of the White Elster and its tributaries Pleiße and Parthe."

# 3. Load Text Mallet and Obfuscate
tmallet = TMallet(lang="en", prefer_gpu=True)
tmallet.load_obfuscator(algorithm, config)

obfuscated_text_sample = tmallet.obfuscate(sample)
print(obfuscated_text_sample)

Output

southernmost 150 eighth-largest most the (90 Central located Bay), Parthe. mi) in city of German Germany part population the is Plain of populous and as Leipzig the German the the km White its Metropolitan Berlin, and Leipzig in the confluence 2025. of of city It Elster is Region. December tributaries The state Saxony. (the southwest the residents the of city Leipzig German is in is Pleiße of a has at part about of 31 and North 633,592

Obfuscate based on an approximation of 'word importance' Mutual information measures how much information context tells you about a word. Words which are both rare and context-dependant tend to be important to the meaning of a text. We can apply a filter to set upper or lower bounds on such an MI score, filering at the word level.

from tmallet import TMallet

algorithm = "shannon", 
config = {
    "threshold": 9.5,
    "bound": "lower",
    "replacement_mechanism": "default",
    "max_context_length": 128,
}

tmallet = TMallet(lang=lang, prefer_gpu=True)
tmallet.load_obfuscator(algorithm, config)

text = "Data obfuscation is the process of modifying sensitive data in such a way that it is of no or little value to unauthorized intruders while still being usable by software or authorized personnel. Data masking can also be referred as anonymization, or tokenization, depending on different context."
obfuscated = tmallet.obfuscate(text)
print(obfuscated)

Output

_ _ is the _ of _ _ _ in _ a _ that it is of no or little _ to _ _ _ _ _ _ by software or _ _. _ _ can also be referred as _, or _, _ on different _.

If we obfuscate too strongly using mutual information, we'll end up with obfuscated sentences like:

, . . - ., or . the / . the . - –, . -

That's, well, probably not very useful. Ideally, we can find a balance between the obfuscation of some words and inclusion of others. Using the bound as lower instead of upper will preserve those words which tend to be more meaningful in a text. Running the above example again with the lower bound of 9, we get an obfuscation which preserves core semantic value while make the text more difficult to reconstruct:

Data obfuscation _ _ process _ modifying sensitive data _ such _ way _ _ _ _ _ _ _ value _ unauthorized intruders while still being usable _ _ _ authorized personnel _ Data masking _ _ _ _ _ anonymization _ _ tokenization _ depending _ _ context _

Here's an overview of an approximation of pointwise word-level mutual information, i.e. PMI(word; context), over 12,000 tokens taken from 10 random texts in the FineWeb-Edu dataset, for instance.

Contributions

Contributions are always welcome, this package was primarily developed in conjunction with carrying out research on the potential value of such obfuscation formats in encoder pre-training. Therefore, the package has many ways it could be improved upon and there are likely opportunities for efficiency improvements in particular.

Acknowledgements

Part of this work was conducted within the CORAL project funded by the German Federal Ministry of Research, Technology, and Space (BMFTR) under the grant number 16IS24077A. Responsibility for the content of this publication lies with the authors.