better-profanity-rs

better_profanity with a Rust core: the same censored text for every input, about 2,100 times faster on a 10-word message and 10,000 to 12,000 times faster on texts of 100 to 700 words (benchmarks).

better_profanity is downloaded about two million times a month, although its last release, 0.7.0, dates from November 2020. It is slow: contains_profanity takes 9 ms for a 10-word message and 0.9 s for a 700-word article on an M1 Pro with CPython 3.13. The word list is a Python list of 916 objects, and each word of the text, alone and joined with the words after it, is compared with every one of them by a method that tries the leetspeak variants letter by letter. better-profanity-rs turns the word list into a trie whose edges are sets of characters (a, @, * and 4 for an a) and runs the original scan, step for step, in Rust. Words in leetspeak, phrases split by separators (f.u.c.k, H4nd j0b), whitelists, custom word lists and the quirks of the original give the same output.

The Python package is better-profanity-rs; the Rust crate is better-profanity. This is an independent project, not affiliated with the author of better_profanity.

Python

pip install better-profanity-rs

Change the import and keep the rest of your code:

from better_profanity_rs import profanity  # was: from better_profanity import profanity

profanity.censor("That wh0re gave m3 a very good H4nd j0b, dude.")
# 'That **** gave m3 a very good ****, dude.'
profanity.contains_profanity("he is a m0th3rf*cker")
# True
profanity.load_censor_words(["happy", "jolly"], whitelist_words=["jolly"])
profanity.censor("Happy and jolly days", "-")
# '---- and jolly days'

Profanity, load_censor_words, load_censor_words_from_file, add_censor_words, censor and contains_profanity take the same arguments as in better_profanity 0.7.0 and return the same results or raise the same exceptions. The attributes CENSOR_WORDSET, CHARS_MAPPING, MAX_NUMBER_COMBINATIONS and ALLOWED_CHARACTERS can be read and changed as before, and the modules better_profanity, constants, utils and varying_string are those of 0.7.0.

Code you do not control

When a library imports better_profanity itself, call install() before importing it:

import better_profanity_rs

better_profanity_rs.install()  # `import better_profanity` now returns better_profanity_rs

import my_web_app

install() covers better_profanity and its modules better_profanity, constants, utils and varying_string. Modules that imported better_profanity before the call keep the original.

Rust

[dependencies]
better-profanity = "0.1"

use better_profanity::Profanity;

let profanity = Profanity::new();
assert_eq!(
    profanity.censor("That wh0re gave m3 a very good H@nD j0b.", "*"),
    "That **** gave m3 a very good ****."
);

let mut custom = Profanity::with_words(["happy", "jolly"]);
custom.add_censor_words(["merry christmas"]);
assert!(custom.contains_profanity("Merry Christmas!"));

Profanity follows the Python class; hide_swear_words, WordSet and CharSet expose the scan for other word sets, separator sets and lowercasing rules. The crate lowercases with Rust's Unicode tables, which can differ from those of a given Python for characters added to Unicode recently; the Python package always uses the running interpreter's str.lower().

Compatibility

The tests compare better-profanity-rs with better_profanity 0.7.0 and fail on any difference in output or exception: the original's own test suite, hand-picked cases, dictionary words in leetspeak inside texts and joined by separators, custom word lists and whitelists, invalid arguments, every kind of attribute change listed below, subclasses, pickling, threads and property-based tests, on CPython 3.9 to 3.14. A separate script compares 285,074 texts, among them the civil_comments test split and the Jigsaw toxic comments (50,532 of the texts are censored), and 28,183 of them again with a custom word list and whitelist, on CPython 3.9, 3.11, 3.13 and 3.14: no output differs (see BENCHMARKS.md).

This includes the original's quirks. For example, add_censor_words does not raise MAX_NUMBER_COMBINATIONS, and when it is 1, only every other word is joined to the next one. After Profanity(["happy", "jolly"]).add_censor_words(["merry christmas"]), both packages censor "m3rry christmas" and "a b m3rry christmas" but not "a m3rry christmas".

better_profanity's own code, bundled in the package, runs whenever the Rust core could behave differently, so the output stays the same and only the speed-up is lost:

• the text or censor_char is not an exact str (a str subclass, for example), or contains lone surrogates;
• CENSOR_WORDSET is not an exact list or tuple of VaryingString objects and strings, or a CHARS_MAPPING value used by a word has a variant that is not exactly one character (better_profanity also accepts longer and empty variants);
• MAX_NUMBER_COMBINATIONS is not an int, or the call is made so close to the recursion limit that the original, which recurses MAX_NUMBER_COMBINATIONS calls deep, could reach it (within about twice MAX_NUMBER_COMBINATIONS + 20 frames);
• ALLOWED_CHARACTERS was replaced by another set, or holds something other than strings;
• a subclass or a patch replaces _get_next_words, _update_next_words_indices, _get_start_index_of_next_word, _get_next_word_and_end_index, get_replacement_for_swear_word, any_next_words_form_swear_word or VaryingString.__eq__, or turns one of the attributes into a property.

The original runs in the same stack frame as in better_profanity, so a RecursionError is raised exactly where better_profanity raises it. Changes in place are seen. CENSOR_WORDSET is compiled again when its items change: each call checks that the list still holds the same objects, which takes about 1 µs for the default list. ALLOWED_CHARACTERS is a set subclass that notes its changes. Lowercasing uses the interpreter's str.lower(), so, as with better_profanity, results follow the Unicode version of the Python running them.

Known differences:

• The private attributes of a VaryingString (_char_combos, _min_len, _max_len) are read when the word list is compiled; changing them later on an object already in CENSOR_WORDSET is not seen.
• ALLOWED_CHARACTERS is a set subclass; changes made through set itself, as in set.add(ALLOWED_CHARACTERS, "-"), are not seen.
• Near the recursion limit, frames are counted with sys._getframe. CPython 3.11 and older also count nested C calls toward the limit; the check allows one per Python frame, so a call made deep inside C-level recursion (a recursive __repr__, for example) could still raise where better_profanity would not, or the reverse. On CPython 3.11, a call made within two frames of the limit raises RecursionError as better_profanity does, but the message can name a different operation.
• A compiled word list keeps the VaryingString objects of CENSOR_WORDSET alive until the list changes or the Profanity object is deleted. An attribute added to one of them that refers back to the Profanity object keeps that object alive.
• Free-threaded CPython builds are not supported: the extension module asks to keep the GIL enabled.
• __version__ and name are those of better_profanity_rs.

Development

cargo test -p better-profanity
cd bindings/python
uv sync --group dev
uv run pytest
cd benchmarks && uv run --group bench python parity.py   # all corpora

License

MIT, the same licence as better_profanity, whose code, word list and character list this project includes. See NOTICE.