Argus Curator

Image curation and moderation: score, de-duplicate, and tag images by face, apparent age, scene and garment, then filter on those tags with a reason recorded for every exclusion. Use it on its own to audit or moderate an image collection, or as the LoRA data-prep front-end of the Argus suite, captioned downstream by argus-lens.

Two ways to use it:

• Standalone moderation and dataset auditing. Scan a folder or call the HTTP API, review the per-image report, and keep, drop or quarantine images on criteria you choose. Nothing has to be trained or captioned. See Standalone: moderation and dataset auditing.
• The curation stage of the Argus suite, described below.

argus-curator is the curation stage of the Argus suite. Upstream, argus-quarry acquires provenance-clean images; curator decides which images, of whom, at what quality belong in a LoRA training set; argus-lens then decides what's in each image. Curator and lens share one TargetProfile, so a manifest written here is captioned downstream with no remapping.

   argus-quarry               argus-curator (:8101)      argus-lens (:8100)      imogen / kohya
   ─ download   ─┐            ─ scan + score ─┐          ─ caption ─┐            ─ train ─
   ─ provenance ─┤  images    ─ face-cluster ─┤ manifest ─ buckets ─┤  dataset   ─ LoRA  ─
   ─ verify ─────┴─────────►  ─ select ───────┴────────► (identity/ ─┴────────►  ───────►
                              ─ export ───────►          wardrobe/…)

Standalone: moderation and dataset auditing

Everything curator records about an image is usable without the rest of the suite. A scan never modifies the source folder; it produces a per-image record (--json, --csv, or GET /scan/{scan_id}), and an export applies filters to it, giving every excluded image a keep_reason. That record doubles as an audit trail: you can show which images were removed and on what grounds.

A typical moderation pass is scan, review, act:

# 1. Scan and report only: nothing is copied, moved or deleted.
argus-curator scan /data/inbox --mivolo --scenes --garments --csv report.csv --json scan.json

# 2. Review report.csv: one row per image with every tag and its confidence.

# 3. Keep only images whose primary face reads as clearly adult. --min-score 0,
#    --include-rejected and --keep-similar stop quality and duplicates from
#    removing anything, so age is the only criterion. Images where no age could
#    be estimated (no face found, or too blurry to pass the hard filters that
#    face detection runs behind) are excluded as `face-attributes-missing`: the
#    filter fails closed rather than letting unknowns through.
argus-curator scan /data/inbox --mivolo --min-age 25 \
    --min-score 0 --include-rejected --keep-similar \
    --copy-to /data/approved --csv decisions.csv

decisions.csv then lists every image with keep and its keep_reason (face-age-out-of-range, face-attributes-missing, garment-excluded, …), which is the list to send for human review or to quarantine. The same scans and filters are available over HTTP (POST /scan/folder, GET /scan/{scan_id}, POST /export), so curator can sit in front of an upload or moderation queue.

Limits — read before relying on it for moderation:

• Content checks need their servers, and both miss things. NudeNet detects exposed body parts only, and misses stylised, partial or unusual poses. SingGuard judges a whole policy (violence, gore, self-harm, …) but gives a verdict without a confidence, so nothing is "near the threshold" to review. Set NSFW thresholds conservatively and have a person review flagged and borderline images.
• Age and demographics are model estimates from appearance, not facts about anyone. Age estimates are typically off by a few years, more so from the face alone (MiVOLO runs face-only here), so filter with a margin above any legal threshold (--min-age 25 rather than 18) and send borderline images to a person.
• Only the primary face of each image is filtered on; other faces are recorded per face (faces[].attributes) but do not exclude the image.
• Scene and garment models make mistakes too; see the measured accuracy in Scene tags and Garments.

Curator is a filter and an audit trail to put in front of human review, not a replacement for it.

Why not FiftyOne / fastdup / Immich?

It's LoRA-native, identity-aware, and caption-integrated — the reason the suite exists.

The shared TargetProfile (the moat)

Both services speak the same taxonomy. This single schema is the contract:

from argus_curator import TargetProfile

TargetProfile(
    target_style="photo",        # "photo" | "anime"
    target_backend="sdxl",       # "sdxl" | "flux-dev-1" | ...
    checkpoint=None,
    target_category="identity",  # identity | wardrobe | pose_composition | setting
)

The curator uses it to weight scoring and label exports; argus-lens inherits it verbatim.

Installation

pip install argus-curator                 # engine only (Pillow, numpy, ImageHash, pydantic)
pip install "argus-curator[server,cli]"   # FastAPI server + CLI
pip install "argus-curator[faces]"        # + InsightFace identity clustering (CPU onnxruntime)
pip install "argus-curator[gpu]"          # + onnxruntime-gpu for GPU face detection
pip install "argus-curator[all]"          # everything

System libraries for the face stack (Ubuntu/Debian):

sudo apt install -y libgl1 libglib2.0-0 libgomp1

Usage

Python

from argus_curator import scan_folder, TargetProfile, ScanConfig, FaceConfig

summary = scan_folder(
    "/data/images",
    profile=TargetProfile(target_category="identity"),
    cfg=ScanConfig(min_short_side=512, blur_threshold=100.0, cluster_distance=10),
    faces_cfg=FaceConfig(enabled=True, model="buffalo_l", cluster_eps=0.5),
)

print(summary.passed, "passed,", summary.duplicates, "near-dupes")
for fc in summary.face_clusters:
    print(fc.cluster_id, fc.size, "faces, rep:", fc.representative_rel_path)

CLI

# Report only — see the score distribution before committing to a threshold
argus-curator scan /data/images --csv report.csv

# Identity curation with face clustering, copy 0.65+ keepers (structure preserved)
argus-curator scan /data/images \
    --target-category identity --faces --device cuda \
    --min-score 0.65 --copy-to /data/curated

# Export only specific identities, capped to a diverse 200
argus-curator scan /data/images --faces \
    --face-clusters face_1,face_2 --max-keep 200 --copy-to /data/curated

# Pick a pose-balanced subset (head-on + 3/4 only, drop side profiles)
argus-curator scan /data/images --faces \
    --pose frontal,three_quarter --copy-to /data/curated

# Predict attributes per face (FairFace: race/gender/age bracket; MiVOLO: age in years/gender),
# then keep adults up to 35 with a confident race prediction
argus-curator scan /data/images --fairface --mivolo \
    --race east_asian --min-attr-conf 0.6 --min-age 18 --max-age 35 --copy-to /data/curated

# Which detectors are available?
argus-curator detectors

Face attributes (FairFace + MiVOLO)

Two optional models read each detected face; either switch implies --faces.

Race labels: white, black, latino_hispanic, east_asian, southeast_asian, indian, middle_eastern. Age brackets: 0-2, 3-9, 10-19, …, 60-69, 70+.

With both on, MiVOLO wins gender and age (it is far stronger at both) and FairFace alone supplies race; the bracket is then derived from MiVOLO's years, so --age filters work with either model. gender_source / age_source say which model produced each value. Results land per face (faces[].attributes), on the primary face (primary_face_attributes, also in the CSV report), and pooled per identity cluster (face_clusters[].attributes): probabilities are averaged and ages take the median over every member face, so one odd frame cannot flip or drag an identity's label.

MiVOLO is built to read a face and a person crop; there is no person detector here, so it runs face-only (a supported mode, slightly less accurate than MiVOLO's published face+body numbers).

These are a model's guess from appearance, not facts about anyone; expect errors on ambiguous faces, mixed heritage, heavy makeup or poor lighting, and check *_conf before relying on a label.

Filters (all on the primary face): --race, --gender, --age (brackets), --min-age / --max-age (years, MiVOLO). --min-attr-conf / min_attribute_conf additionally requires a filtered label's confidence to reach the floor (a MiVOLO age has none, so it is not held to it). Excluded images say why in keep_reason: face-<attr>-excluded, face-<attr>-low-confidence, face-<attr>-missing, face-age-out-of-range, face-age-years-missing, or face-attributes-missing.

Models are not on PyPI; each is converted to ONNX once, after which inference runs on the [faces] onnxruntime stack with no torch:

# MiVOLO: downloads the pinned checkpoint (commit + sha256 checked)
pip install "argus-curator[mivolo-convert]"
argus-curator mivolo-convert          # -> ~/.cache/argus_curator/models/mivolo_v2_d1_384.onnx

# FairFace: download fairface_alldata_20191111.pt from the FairFace repo first
pip install "argus-curator[fairface-convert]"
argus-curator fairface-convert fairface_alldata_20191111.pt
                                      # -> ~/.cache/argus_curator/models/fairface_res34_race7.onnx

Point elsewhere with --fairface-model / --mivolo-model or ARGUS_CURATOR_FAIRFACE_MODEL / ARGUS_CURATOR_MIVOLO_MODEL. Model paths are server configuration, never part of a request; a server without a requested model refuses the scan with 400 before scoring anything, and GET /detectors reports fairface / mivolo.

Scene tags (Places365)

--scenes (or "scenes": {"enabled": true} on the scan request) runs a Places365-CNN (Zhou et al., TPAMI 2017) on every passing image and records its top-k scene categories (365 of them: beach, bedroom, gas_station, forest_path, …) with their softmax probabilities, plus an indoor/outdoor call. environment_conf is the probability mass on categories of that environment, not upstream's top-10 vote. Results land on results[].scene and in the CSV report.

Top-1 accuracy on Places365 is about 55% (top-5 about 85%), and neighbouring categories overlap (lagoon / beach / coast), which is why filters match any of the recorded top-k rather than only the first:

• --scene beach,coast / scenes: keep images with one of these among their top-k at probability ≥ --min-scene-conf / min_scene_conf (default 0.1, so a label lingering in the tail at 3% does not count). Unknown labels are rejected with 422 rather than silently matching nothing.
• --environment outdoor / scene_environments: indoor or outdoor, held to the same floor.
• --diversify-scenes / diversify_scenes (with --max-keep): the diversity cap also spreads the selection across backdrops, so an identity set is not two hundred bathroom selfies.

Excluded images say why: scene-excluded, scene-low-confidence, scene-environment-excluded, scene-environment-low-confidence, or scene-missing.

The weights are converted once to ONNX, like the face models:

pip install "argus-curator[places365-convert]"   # torch, only for this step
argus-curator places365-convert                  # resnet18 (default) or --arch resnet50
# -> ~/.cache/argus_curator/models/places365_resnet18.onnx

Upstream serves the checkpoints over plain HTTP, so each is pinned by sha256. ResNet-50 is twice the size for about one point of top-1 (55.6% vs 54.5%). Point elsewhere with --places365-model / ARGUS_CURATOR_PLACES365_MODEL.

Garments (Fashionpedia)

--garments (or "garments": {"enabled": true}) runs a YOLOS-small detector fine-tuned on Fashionpedia (valentinafevu/yolos-fashionpedia, MIT; dataset CC BY 4.0) on every passing image. Each detection records its garment type (27 classes: dress, pants, shoe, bag_wallet, hat, …), probability, box, whether the box touches the frame edge (likely cut off), and a dominant colour from the box's centre. Results land on results[].garments ([] = ran, found nothing) and in the CSV as label:conf:#colour, with * marking a cut-off garment.

Measured on 300 Fashionpedia validation images (CPU, ~0.3 s/image):

Fashionpedia's 19 garment parts (sleeve, pocket, zipper, …) scored AP50 0.105 and are not recorded; neither are its 294 fine-grained attributes, which no published checkpoint predicts. Detections are recorded from 0.3 so reviewers see the doubtful ones; filters count them from --min-garment-conf (0.5). A floor below what the scan recorded cannot be honoured, so the server refuses it with 400; the CLI instead records from the lower floor in the first place.

• --garment dress,skirt / garments: wearing at least one of these.
• --exclude-garment bag_wallet,umbrella / exclude_garments: none of these. Exclusion wants recall, so consider a lower --min-garment-conf for it.
• --uncropped / garments_uncropped: the matching garment is not cut off.
• --diversify-outfits / diversify_outfits (with --max-keep): the diversity cap also spreads the selection across outfits (garment types and their colours), so an identity LoRA does not learn one outfit as part of the person.

Exclusions say why: garment-not-found, garment-low-confidence (present, below the floor), garment-excluded, garment-cropped, garments-missing.

pip install "argus-curator[garments-convert]"   # torch + transformers, only for this step
argus-curator garments-convert                  # pinned commit + sha256
# -> ~/.cache/argus_curator/models/yolos_fashionpedia.onnx

Point elsewhere with --garments-model / ARGUS_CURATOR_GARMENTS_MODEL.

Nudity detection (NudeNet)

Curator can check every scanned image with NudeNet, a detector for 18 body-part classes (FEMALE_BREAST_EXPOSED, BUTTOCKS_COVERED, FACE_FEMALE, …). It talks to a NudeNet server over HTTP; curator never imports NudeNet, so it gains no dependency and stays MIT. NudeNet itself is AGPL-3.0, so running its server is a choice for whoever deploys it.

Configure the server in the environment (or .env), never per request:

ARGUS_CURATOR_NUDENET_URL=http://192.168.1.50:8080   # its origin; /infer is appended
ARGUS_CURATOR_NUDENET_TIMEOUT=30                     # seconds per image (default 30)
ARGUS_CURATOR_NUDENET_WORKERS=4                      # concurrent requests (default 4)

Any host running ghcr.io/notai-tech/nudenet works, including one elsewhere on your LAN. docker compose up starts one alongside curator (pinned by digest) and uses it unless .env says otherwise.

It is opt-out. When a URL is configured, every scan checks its images, including ones that failed the quality filters, unless the scan opts out (--no-nsfw, or "nsfw": {"enabled": false}). --nsfw / "enabled": true makes the check mandatory: the scan is refused if no server is configured. Set the URL empty to switch the check off for a deployment. The scan summary records whether it ran (nsfw_config.enabled). Images are sent to that server, so keep it on infrastructure you control.

Each image records every detection, the highest score per class (nsfw.labels), and an exposed score: the highest score among the five exposed-nudity classes (BUTTOCKS_EXPOSED, FEMALE_BREAST_EXPOSED, FEMALE_GENITALIA_EXPOSED, MALE_GENITALIA_EXPOSED, ANUS_EXPOSED). Faces and covered body parts are detected too but never count towards it. The CSV report has nsfw_exposed_score and nsfw_labels.

Filters drop an image when a score is at or above its threshold:

# Total: drop anything whose exposed score is 0.5 or more
argus-curator scan /data/inbox --nsfw-threshold 0.5 --copy-to /data/clean

# Per class, alongside or instead of the total
argus-curator scan /data/inbox \
    --nsfw-label FEMALE_BREAST_EXPOSED=0.4,MALE_GENITALIA_EXPOSED=0.3,BELLY_EXPOSED=0.8 \
    --copy-to /data/clean

Over HTTP these are nsfw_threshold, nsfw_label_thresholds ({"CLASS": score}) and nsfw_exposed_labels (which classes the total is taken over). The server reports nothing under 0.2, so thresholds must be 0.2–1. An image that was never checked, or whose check failed, is excluded whenever an NSFW filter is set (nsfw-unchecked): the filter fails closed. Other reasons are nsfw-exposed and nsfw-<class> (e.g. nsfw-belly-exposed). Exporting with NSFW thresholds from a scan that did not run NudeNet is refused (400) rather than excluding everything. GET /detectors reports nudenet when a URL is configured.

Policy check (SingGuard)

Where NudeNet scores nudity per body part, a SingGuard server judges each image against a whole moderation policy and answers with a verdict and the rule that fired. Its default rules:

It gives no scores, so it complements NudeNet rather than replacing it. It runs a vision language model: about a second per image in fast mode, several in fast-slow (which also writes why a rule fired). Configure it like NudeNet:

ARGUS_CURATOR_SINGGUARD_URL=http://192.168.1.198:9150   # its origin; /api/guard/check is appended
ARGUS_CURATOR_SINGGUARD_THINKING=fast                    # or fast-slow, for explanations
ARGUS_CURATOR_SINGGUARD_POLICY_FILE=/etc/curator/policy.txt  # optional: replaces the default rules
ARGUS_CURATOR_SINGGUARD_TIMEOUT=90                       # seconds per image (default 90)
ARGUS_CURATOR_SINGGUARD_WORKERS=2                        # concurrent requests (default 2)

It is opt-out in the same way (--no-guard, "guard": {"enabled": false}; --guard makes it mandatory). The default rule A flags any sexual content, adult nudity included. For an adult dataset, write a policy file whose sexual-content rule covers only apparent minors and non-consent, and whose Safe rule names consensual adult nudity. Each image records guard.safe, the rules that fired, the explanation, the model and the policy (default, or custom-<sha256 prefix> of the file). The CSV report has guard_safe and guard_rules.

# Drop anything judged unsafe
argus-curator scan /data/inbox --guard-unsafe --copy-to /data/clean

# Drop only crime and unethical-behaviour verdicts, by letter or full name
argus-curator scan /data/inbox --guard-rules B,C --copy-to /data/clean

Over HTTP these are guard_unsafe and guard_rules. As with NudeNet, an unchecked image is excluded whenever a SingGuard filter is set (guard-unchecked); other reasons are guard-unsafe and guard-rule-<rule> (e.g. guard-rule-c), or guard-rule-unknown for an unsafe verdict that names no rule. GET /detectors reports singguard.

HTTP server (:8101, peer to argus-lens)

pip install "argus-curator[server,faces]"
argus-curator serve --cors --port 8101 --source-root /data/images --export-root /data/out

Request-supplied paths are treated as untrusted: scan folders resolve under --source-root (env: ARGUS_CURATOR_SCAN_ROOT / CURATOR_SOURCE_PATH) and export destinations under --export-root (ARGUS_CURATOR_EXPORT_ROOT / CURATOR_EXPORT_PATH). Traversal escapes are refused, and scan/export refuse outright when their root is not configured. Both roots are needed to export: the destination resolves under the export root, and the scan being exported must itself lie under the source root — a scan_id is data from a store the CLI also writes to, so its folder is re-checked rather than trusted (the same check applies to /thumb?scan_id=). The destructive mode: "move" is rejected unless the server is started with --allow-move (CURATOR_ALLOW_MOVE=1). --cors allows only the localhost dev frontend (:3000); allow-list other origins with --cors-origin <origin> (CURATOR_CORS_ORIGINS), or use --cors-any for a credential-less wildcard on public demos. A literal * in the allow-list is treated as --cors-any (wildcard without credentials), never as origin reflection.

CORS is not a write boundary — POST /upload sends multipart/form-data, which browsers deliver with no preflight — so state-changing requests are additionally gated on Origin: it must be absent (curl, the CLI, server-to-server), same-origin, or on the allow-list, else 403. --cors-any therefore permits anonymous reads from anywhere but no cross-site writes; name an origin with --cors-origin to let it write.

The */stream variants emit event: progress frames ({phase, done, total}) while the work runs, then a single event: complete frame carrying the same payload the non-streaming endpoint returns (or event: error).

POST /scan/folder body:

{
  "folder": "shoots/2026-07",         // relative to the server's --source-root ("" = the root)
  "target_profile": { "target_style": "photo", "target_category": "identity" },
  "config": { "min_short_side": 512, "max_aspect_ratio": 3.0, "blur_threshold": 100.0,
              "cluster_distance": 10, "max_workers": 4 },
  "faces":  { "enabled": true, "model": "buffalo_l", "min_det_score": 0.5, "cluster_eps": 0.5 }
}

POST /export body:

{
  "scan_id": "...",                 // or inline "selection": ["rel_path", ...]
  "dest": "training-set-v1",        // relative to the server's --export-root
  "mode": "copy",                   // "copy" | "symlink" | "move" (move needs --allow-move)
  "preserve_structure": true,
  "min_score": 0.6, "include_rejected": false, "keep_similar": false,
  "face_clusters": ["face_2"],      // optional: export only these identities
  "face_poses": ["frontal", "three_quarter"],  // optional: export only these head poses
  "face_races": null, "face_genders": null,    // optional: attribute filters on the primary face
  "face_ages": null,                // FairFace brackets ("20-29", ...)
  "min_age_years": 18, "max_age_years": 35,     // optional: MiVOLO age range
  "min_attribute_conf": 0.6,        // filtered attributes must be at least this confident
  "scenes": ["beach", "coast"],     // optional: Places365 scenes among the image's top-k
  "scene_environments": ["outdoor"],
  "min_scene_conf": 0.1, "diversify_scenes": false,
  "garments": ["dress"], "exclude_garments": ["bag_wallet"],  // optional: Fashionpedia
  "min_garment_conf": 0.5, "garments_uncropped": false, "diversify_outfits": true,
  "write_manifest": true,
  "caption_url": null               // optional: POST manifest to argus-lens
}

Docker

docker compose up --build

Bind-mounts a dataset into /data/images, an output dir into /data/out, and persists the scan cache + InsightFace model downloads across rebuilds. A converted FairFace / MiVOLO model under ~/.cache/argus_curator/models/ is picked up through the same cache mount.

Handoff to argus-lens

Export writes a JSONL manifest (one row per exported image — rows exist only for files whose transfer actually succeeded):

{ "manifest_version": "2.1", "rel_path": "...", "abs_path": "...",
  "exported_path": "...", "exported_abs_path": "...", "target_profile": { ... },
  "primary_face_cluster": "face_2", "primary_face_pose": "three_quarter",
  "score": 0.87, "similar_group": 3 }

The row shape is published in the wire schema as ManifestRow. exported_path is the path actually written under the export root — consumers must use it rather than re-deriving a location from rel_path. Flattened exports (preserve_structure: false) de-collide duplicate basenames with a short hash suffix, so the two can differ; collisions are detected case-insensitively (and Unicode-normalised) so the result is safe on case-insensitive destination filesystems, and the export fails loudly rather than overwrite if a unique name cannot be generated. The same rel_path -> exported_path mapping is returned as exported_paths on the export response, so it is available even with write_manifest: false. Note that each export call plans in isolation: re-exporting into the same destination overwrites files (and rewrites the manifest).

exported_abs_path is that file's absolute location, and exported_abs_paths the matching mapping on the export response. Prefer them: joining the export root onto exported_path means reconstructing the server's layout rule somewhere that does not own it, and the de-collision suffix above is not reproducible by a client anyway. A manifest is also frequently read somewhere other than the root it was written into — argus-lens receives it as an upload — and nothing in a row names that root.

abs_path is where the image can be read, which is not always where it came from: mode: "move" deletes the source during transfer, so for a moved export abs_path equals exported_abs_path. Use rel_path for the original location within the scan.

GET /health reports manifest_version alongside the package version, so a consumer can check the contract it is about to speak — and refuse an unknown major — rather than inferring it from which fields a response happens to carry. The two versions move independently.

argus-lens batch-captions this manifest — categories are already shared, so no remapping. Set caption_url on the export request to POST it straight to lens for a one-click curate→caption run.

How scoring works (per image)

1. Hard filters — min short side, max aspect ratio, blur (Laplacian-edge variance floor).
2. Composite score — target-aware weighted blend of sharpness / resolution / artifact, plus a small composition bonus that depends on target_category (e.g. identity rewards a single centred face; setting rewards wide framing).
3. Face-count fit — with [faces], identity targets penalise 0 or 2+ faces; other categories are progressively more tolerant.
4. Near-duplicate dedup — pHash clustering keeps the highest-scoring representative and reports the rest (never silently dropped).
5. Selection (at export) — score threshold + optional diversity cap (max_keep / diversity_weight) + optional face-cluster filter. Every excluded image carries a keep_reason.

State

Scans are cached on disk (keyed by scan_id, default ~/.cache/argus_curator/scans, override with CURATOR_CACHE_DIR). This is what makes paginated GET /scan/{id} and export-by-id work without recomputing.

Related projects

• argus-quarry — provenance-first acquisition of public-domain / CC0 portraits (the suite's input stage).
• argus-lens — intent-aware, multi-model captioning (consumes the manifest this package exports).
• argus-studio — the suite's Next.js web UI (its /curate view drives this server).

License

MIT — matches the rest of the Argus suite.