📖 Description

BadWords cleans up user-generated content. Unlike plain keyword matching it normalizes text first — NFKC, case folding, confusable characters, homoglyphs and cyrillic/latin transliteration — and then looks the result up in a dictionary of 20330 entries across 25 languages (16981 of them distinct), with optional fuzzy, phrase and substring matching on top.

Architecture: the engine is Rust. Python gets it through PyO3 with the GIL released around every call, JavaScript through WebAssembly, and Rust projects can depend on the core directly. All three share one set of word lists and one language registry.

📦 Installation

Requirements

• Python: 3.10+
• Rust (if you use the crate): 1.78+

PyPI

pip install badwords-py

From GitHub

pip install git+https://github.com/FlacSy/badwords.git

Cargo / npm

cargo add badwords-core
npm install badwords-wasm

⚡ Quick Start

from badwords import ProfanityFilter

p = ProfanityFilter()
p.init(languages=["en", "ru"])   # or p.init() for all 25

p.is_profane("hello world")      # False
p.is_profane("sonofabitch")      # True

# Censoring keeps everything that is not part of a match,
# including the punctuation attached to the word.
p.censor("hey shit, ok")         # "hey ****, ok"

# find() says what matched, where, and in which language.
for m in p.find("what a shitty, damn mess"):
    print(m.matched_text, m.start, m.end, m.word, m.language, m.kind)

Your own words and a whitelist

p.add_words(["spam_link_v1", "scam_bot_99"])
p.add_whitelist(["assessment"])   # never reported, whatever the rules say

p.word_count()        # entries in the dictionary
p.contains_word("x")  # after normalization

Many texts at once

One call into Rust instead of many, with the GIL released for the whole batch:

p.is_profane_many(["hello", "sonofabitch", "fine"])   # [False, True, False]
p.find_many(texts)                                    # list[list[Match]]
p.censor_many(texts, "*")                             # list[str]

🎛 Options

Every call takes an Options; without one the filter's default is used. Each field is off by default, because each detector trades false negatives for false positives.

from badwords import Options, ProfanityFilter

p = ProfanityFilter()
p.init(["en"], options=Options(collapse_repeats=True))  # default for this filter

p.is_profane("text", Options(match_threshold=0.9))      # just this call

Options.aggressive() turns on every detector at threshold 0.9 in substring mode. It is a starting point for measurement, not a recommended default — see the false-positive table below.

Each match is a frozen dataclass:

🧩 Evasion Detection

Normalization is always on: hеllo with a Cyrillic е, decorative Unicode, diacritics and mixed scripts all fold to the same form before lookup. The rest is opt-in:

from badwords import Options, ProfanityFilter

p = ProfanityFilter()
p.init(["en"])

strict = Options(split_on_punctuation=True, collapse_repeats=True, leetspeak=True)

p.is_profane("shiiit")             # False
p.is_profane("shiiit", strict)     # True — collapse_repeats
p.is_profane("you.shit", strict)   # True — split_on_punctuation
p.is_profane("wh0re", strict)      # True — leetspeak

Some leet spellings are in the word lists as literal entries — sh1t and a55 ship in en — so they match without leetspeak at all. The flag is for the spellings nobody wrote down.

What each detector costs

Measured against 73302 clean words from /usr/share/dict/american-english with all 25 languages loaded. cargo run --release -p badwords-core --bin fp_report --features substring reproduces the table, and tests/false_positives.rs fails the build if the budgets regress.

The three character-level detectors are free of false positives on that corpus; fuzzy matching and substring mode are not. Substring is more expensive here than it will be for most users, because a short entry in one language occurs inside ordinary words of another — with English alone the rate at the default length is 0.150%.

⏱ Benchmarks

en + ru loaded, default options (exact matching), release builds.

Rust — criterion, make bench-rust:

Python — pytest-benchmark medians, make bench-python:

Two things the numbers show. A match is cheaper than clean text, because matching stops at the first hit while clean text is normalized to the end. And the batch API is worth using: five texts in one call cost 15.5 µs against 21.2 µs one at a time, since the GIL is released once instead of five times.

vs glin-profanity

Rule-based mode, en+ru, same texts through each library's own API. Run: make bench-compare (needs pip install glin-profanity). BadWords is timed over 50,000 iterations, glin-profanity over 1,000.

Roughly 300–550×, which is what a Rust core buys against a pure-Python one.

ML mode

The optional model is XLM-RoBERTa, INT8-quantized, run through ONNX Runtime — no torch on the inference path. 100 iterations each, same machine.

Cost scales with length, not just count: these are short texts, and a 400-character comment filling the 128-token window costs several times more — the held-out evaluation, on comments up to 400 characters, averages about 48 ms per text. Batch what you can.

Model timings are indicative. Repeated runs on this machine varied by up to 2x with whatever else it was doing, while the ratio to glin's transformer held at roughly 4x across every run. Rule-based numbers are far steadier, within about 15%.

A model call is three orders of magnitude more expensive than a rule call — see ML scoring for what to do about that.

🤖 ML scoring

pip install "badwords-py[ml]"     # onnxruntime + transformers, no torch

The rules answer "is this word in the dictionary". The model answers "how toxic is this text, and in what way" — seven independent probabilities, not a single verdict:

from badwords.ml import ToxicityPredictor

predictor = ToxicityPredictor()
scores = predictor.predict_scores("you are a worthless idiot")

scores.toxicity       # 0.97
scores["insult"]      # 0.92
scores.strongest()    # ("toxicity", 0.97)
scores.above(0.5)     # [("toxicity", 0.97), ("insult", 0.92)]
scores.as_dict()      # every axis

The axis names come from the model's own config, so a model you train yourself with a different label set stays usable.

Quality

Measured on 10,000 held-out rows — civil_comments and toxic_conversations test splits, filtered against every row training touched. make ml-evaluate reproduces it. AUC is threshold-free; "best F1" comes with the threshold that reaches it, because one threshold does not fit seven axes of very different rarity.

Measured on the INT8 model, the one the release ships: quantizing costs about 0.3 points of AUC against fp32.

Read AUC and average precision together. Every axis ranks well, but the rare ones — threat is 1% of rows — have far lower average precision, which is what you feel in practice: at a useful precision their recall is modest. Treat toxicity as the axis to decide on and the narrow ones as the reason to show ("blocked, reads as a threat"), not as triggers of their own.

severe_toxicity has no row in the test set where a majority of annotators agreed, so there is nothing to score it against at all.

The narrow axes are supervised by English civil_comments alone; overall toxicity is supervised by every source. Rebalancing the training mix towards Russian lifted toxicity (0.906 → 0.923 F1) and cost the narrow axes several points — English rows in the mix were halved to make room.

Quality in Russian

English and Russian are measured separately, because they behave differently. 1000 rows each: the English ones from the held-out split above, the Russian ones from AlexSham/Toxic_Russian_Comments.

Two things are worth reading off this. The rules behave differently per language — 50% recall in Russian against 27% in English, because Russian profanity is lexical and a dictionary catches it, while English toxicity is more often built from ordinary words. And the model's thresholds now sit close together (0.25 and 0.15), which one default can serve; an earlier English-heavy training mix put them at 0.30 and 0.70, and no single default could.

The Russian figure is in-domain: the model trained on other rows of that same corpus. Read it as "good on Russian of this kind", not as a cross-corpus guarantee.

Neither half replaces the other. The dictionary cannot see toxicity built out of ordinary words, which is most of it in English; the model has no notion of the project-specific words you add to the dictionary yourself, and no interest in matching them exactly. That is what HybridFilter is for.

Rules first, model second

from badwords.ml import HybridFilter

hybrid = HybridFilter(languages=["en", "ru"])

result = hybrid.check("you are a worthless waste of oxygen")
result.is_profane     # True
result.decided_by     # "model" — no dictionary entry to go on
result.scores         # every axis
result.rule_score     # what the rules made of it

A certain dictionary hit answers immediately and never reaches the model; everything else does, text the rules found nothing in included. Treating "the rules saw nothing" as "clean" is what makes a hybrid score worse than the model it wraps.

The model is downloaded on first use, not on construction — download_model() warms the cache deliberately, and BADWORDS_ML_PATH points at your own.

From Rust

[dependencies]
badwords-ml = "3"

use badwords_ml::ToxicityModel;

let model = ToxicityModel::open_located()?;   // BADWORDS_ML_PATH or the shared cache
let scores = model.predict("you are an idiot")?;

scores.toxicity();        // 0.94
scores.get("insult");     // Some(0.93)
scores.above(0.5);        // [("toxicity", 0.94), ("insult", 0.93)]

HybridFilter is there too, with the same semantics as the Python one. The crate reads the same model directory the Python package downloads, so the two languages score identically — the only difference is batch padding, worth a few hundredths on an INT8 model.

Requires Rust 1.88 (ort's own minimum); badwords-core still builds on 1.78.

🌍 Supported Languages

25 languages, keyed by ISO 639-1 where one exists. Entry counts as shipped:

p.available_languages()   # every language that could be loaded
p.loaded_languages()      # the ones actually loaded, as canonical codes
p.load_languages(["de"])  # add to what is already loaded
p.unload_languages(["de"])

Pre-3.0 codes

The old codes still work as aliases: cz→cs, du→nl, gr→el, po→pt, sp→es, tu→tr, ua→uk.

Four of them collide with a real ISO 639-1 language and emit a DeprecationWarning:

🚀 Full Integration Example

from badwords import Options, ProfanityFilter

MODERATION = Options(
    split_on_punctuation=True,
    collapse_repeats=True,
    leetspeak=True,
)


def monitor_chat() -> None:
    p = ProfanityFilter()
    p.init(["en", "ru", "de"], options=MODERATION)

    # Project-specific words, and words that must never be flagged.
    p.add_words(["spam_link_v1", "scam_bot_99"])
    p.add_whitelist(["assessment"])

    user_input = "Hey! Check out this cr4p"

    matches = p.find(user_input)
    if matches:
        reason = ", ".join(f"{m.word} ({m.kind})" for m in matches)
        print(f"Message blocked: {reason}")
        print("Shown instead:", p.censor(user_input))
    else:
        print("Message accepted")


if __name__ == "__main__":
    monitor_chat()

🦀 Rust API (badwords-core)

Published on crates.io:

[dependencies]
badwords-core = "3"

use badwords_core::{Options, ProfanityFilter};

let mut filter = ProfanityFilter::builder()
    .embedded()                    // word lists compiled into the crate
    .languages(["en", "ru"])       // or .all_languages()
    .build()?;

let opts = Options::new();

filter.is_profane("hello world", opts);        // false
filter.censor("hey shit, ok", '*', opts);      // "hey ****, ok"

for m in filter.find("what a shitty, damn mess", opts) {
    println!("{:?} at {}..{} ({:?})", m.matched_text, m.start, m.end, m.kind);
}

filter.add_words(&["custombad"]);
filter.add_whitelist(&["assessment"]);

find_first short-circuits on the first match, and find_into reuses caller-owned Scratch and output buffers so a hot loop allocates nothing.

Crate features

Examples: cargo run --example rust_basic, rust_chat_moderation, rust_specific_languages.

🌐 WebAssembly (JavaScript / TypeScript)

The same Rust code compiled for the browser and Node.js, with English and Russian built in.

npm install badwords-wasm

import init, { ProfanityFilter } from 'badwords-wasm';

await init();                       // browser build only
const filter = new ProfanityFilter();

filter.isProfane('hello');          // false
filter.censor('hey shit, ok', '*'); // "hey ****, ok"
filter.find('what a shitty mess');  // [{ word, matchedText, start, end, language, score, kind }]

filter.addWords(['spam_link']);
filter.addWhitelist(['assessment']);

Options are a plain object, reusable and checked — an unknown key is an error rather than a silently disabled detector:

filter.isProfane('sh1t', { leetspeak: true, collapseRepeats: true });
filter.setOptions({ matchThreshold: 0.9 });   // default for later calls

Match and MatchOptions are real TypeScript interfaces in the generated .d.ts. filterText, isBad and getLanguages still work and are marked @deprecated.

Optional languages (npm)

npm install badwords-wasm @badwords/languages

import de from '@badwords/languages/de';
import uk from '@badwords/languages/uk';

filter.addWords(de);
filter.addWords(uk);

Available: cs, da, de, el, en, es, es_419, fi, fr, hu, id, it, ja, ko, nl, no, pl, pt, pt_br, ro, ru, sv, th, tr, uk, plus the pre-3.0 codes as aliases. See @badwords/languages.

Build from source

cargo install wasm-pack
make wasm          # browser  -> rust/badwords-wasm/pkg-web/
make wasm-nodejs   # Node.js  -> rust/badwords-wasm/pkg-node/

Examples: examples/wasm/browser/, examples/wasm/node/ (JavaScript and TypeScript).

🔀 Migrating from 2.x

Everything from 2.x still works and still behaves identically — the deprecated paths are checked against 8736 recorded 2.3.1 responses. They warn, so the move can be gradual.

Two behaviour changes worth knowing about:

• Censoring keeps punctuation. censor("hey shit, ok") returns "hey ****, ok"; 2.x returned "hey ***** ok" because it replaced the whole whitespace-delimited token. filter_text keeps the old behaviour.
• Multi-word entries match now. They could not before, so phrases in the shipped lists never fired. Phrase matching needs two or more consecutive words, so it cannot produce a single-word false positive; turn it off with Options(phrases=False).

🔧 Building from source

Requires Rust, Python and maturin.

python -m venv .venv && source .venv/bin/activate
pip install maturin
make develop        # debug build, installed into the venv
make test           # rust + python + wasm
make bench          # criterion + pytest-benchmark

make develop mirrors rust/badwords-core/resources/ into python/badwords/resource/ first; make check-resources verifies the mirror, and CI fails if it drifts.

📚 Documentation

Full documentation (Python, Rust, JavaScript) with examples and API reference: badwords.flacsy.dev (EN / RU).

🤝 Contributing

See CONTRIBUTING.md. Word-list changes have their own checklist there — every entry needs a source, and the false-positive budget is enforced by CI.

📄 License

Distributed under the MIT License. See LICENSE for more information.

Word lists incorporate material from LDNOOBW (CC BY 4.0) and washyourmouthoutwithsoap (MIT); see NOTICE and rust/badwords-core/resources/words/SOURCES.md.