openmoderate 1.1.3

See every upload. Keep only what's safe.
Open-source, self-hosted moderation for images and video: nudity, violence, weapons, hate symbols, drugs and more.

Documentation · Benchmarks · Discussions · Changelog · API reference

_{Every result shown is real output on openly licensed photos.}

Quick start

pip install openmoderate

from openmoderate import Moderator

mod = Moderator()                      # or Moderator(preset="accurate")
result = mod.analyze("upload.jpg")     # path, URL, bytes, file upload or PIL image

result.is_safe        # False
result.violations     # [Label(name='weapons', confidence=0.96, parent='violence', threshold=0.5)]

Or from the terminal:

openmoderate scan ./uploads            # images and videos, recursively

Why openmoderate

• An open alternative to cloud moderation APIs. Comparable to AWS Rekognition's DetectModerationLabels, but it runs inside your app: no cloud account, no per-image fees, and no image ever leaves your servers.
• Real machine learning, no ML expertise needed. A CLIP deep-learning model does zero-shot classification, so it works out of the box. Fine-tune it on your own labelled images when you need more.
• Measured, not claimed. Accuracy comes from a public benchmark of 475 hand-reviewed photos with a held-out half and cross-validation, and the limits are published next to the numbers.
• Your policy, in plain text. Change thresholds, switch categories off, or add new ones like gambling with a few words of YAML. No training data required.
• Images and video. Videos get timestamped segments ("weapons at 0:03–0:06"), with an early stop for fast rejection.
• Drops into your stack. One line protects uploads in FastAPI, Flask or Django; a REST API and Docker image serve every other language.
• Hardened for untrusted input. SSRF-safe URL fetching, size and decoding limits, pickle-free model files, and confidence scores hidden from end users by default.
• Choose speed or accuracy. Three model sizes, from 6 ms per image to the most accurate.

Install

Python 3.9–3.14. The model weights (~600 MB) download once on first use; after that everything runs offline, on CPU, NVIDIA GPUs or Apple Silicon. On CPU-only servers, see the install notes to save ~2 GB.

Usage

from openmoderate import Moderator

mod = Moderator(thresholds={"weapons": 0.3}, exclude=["alcohol", "tobacco"])
results = mod.analyze_batch(["a.jpg", "b.jpg", "c.jpg"])     # faster for many images

# policy.yaml  →  Moderator("policy.yaml")
preset: accurate
thresholds: {weapons: 0.3}
disabled: [alcohol, tobacco]
custom_categories:
  - name: gambling
    prompts: ["a casino slot machine", "people gambling at a poker table"]
    negatives: ["people playing a board game"]   # benign look-alikes

# FastAPI: violating images get a 422 before your endpoint runs
@app.post("/avatar")
async def avatar(file: ModeratedFile): ...

# Flask
@app.post("/avatar")
@moderate_uploads("avatar")
def avatar(): ...

# Django: works in models, forms, the admin and DRF serializers
avatar = models.ImageField(validators=[ModerationValidator()])

result = Moderator().analyze_video("clip.mp4", fps=1, stop_on_violation=True)
result.segments   # [VideoSegment(name='weapons', start=3.0, end=6.0, max_confidence=0.99, frames=3)]

docker run -p 8000:8000 -e OPENMODERATE_API_KEY=change-me ghcr.io/karammm/openmoderate
curl -H "X-API-Key: change-me" -F file=@upload.jpg localhost:8000/v1/moderate

openmoderate scan ./uploads --json                 # one JSON object per file
openmoderate scan ./uploads --fail-on-violation    # exit code 1 if anything is flagged
openmoderate scan clip.mp4 --fps 2 --preset accurate

openmoderate train ./dataset -o my_head.omh        # dataset/<label>/*.jpg + dataset/safe/
openmoderate evaluate ./holdout --head my_head.omh

mod = Moderator(head="my_head.omh")

How it works

trim borders → embed image (CLIP) → compare with category prompts → calibrated score → your threshold

openmoderate is built on OpenCLIP, an open implementation of CLIP: a deep-learning model with a Vision Transformer image encoder and a text encoder, trained on LAION-2B (about 2 billion image–text pairs). CLIP places images and sentences in the same vector space, so each category is described in plain words ("a photo of a gun", "a nazi swastika flag"). An image is flagged when it matches a category's prompts clearly better than everyday safe prompts and known look-alikes such as kitchen knives, national flags and medicine.

This is zero-shot classification: no training data is needed, and a new category is a few lines of text. With labelled images, openmoderate train adds supervised learning on top: a class-balanced logistic-regression head on frozen CLIP embeddings.

How it works, in depth →

Accuracy

Measured on 475 openly licensed photos from Wikimedia Commons, each reviewed by eye, of which 215 are safe, including deliberate look-alikes. Reproduce it with python benchmark/run.py --preset <name> --split all.

Every model is calibrated with one rule: the highest recall with at most ~5% of safe images flagged. Because a single tuning/test split proved unreliable at this size, the last row scores each half of the images with the calibration chosen on the other half; it is the fairest estimate of real-world performance. Precision on the default model's held-out half is 100% for alcohol, hate symbols, violence and weapons, and 82–89% for drugs and tobacco. Method, licences and full results →

Quality

Every push is checked in CI with:

• Unit and integration tests on Python 3.9, 3.10, 3.11, 3.12, 3.13 and 3.14, plus a test against the real model.
• Lint, formatting and type checks (ruff, mypy); the package ships type information (py.typed).
• The Docker image built and smoke-tested end to end, moderating a real image and a real video inside the container.
• A strict docs build, so broken links and docstrings fail the build.

Releases are automatic: when a version bump passes CI on main, it is tagged and published to PyPI (through trusted publishing, with no stored tokens), GitHub Releases and the GitHub container registry.

Community

• Questions and ideas: GitHub Discussions
• False positives, missed content, bugs: open an issue. Please describe harmful images in words and never attach them.
• Contributing: see CONTRIBUTING.md for setup, tests, the benchmark and releases. Better prompts and benchmark images are especially welcome.
• Security: please report vulnerabilities privately, as described in SECURITY.md.
• Code of conduct: CODE_OF_CONDUCT.md
• Sister project: SafeType.js moderates text as people type.

Responsible use

openmoderate is meant to help keep platforms safe. Do not use it to surveil individuals or to profile people by protected characteristics. Moderation models make mistakes, so give users a way to appeal automated decisions.

License

Apache 2.0 © openmoderate contributors. Benchmark images keep their own licences (see benchmark/manifest.csv).