Exordium

Collection of Preprocessing Functions and Deep Learning Methods for Multimodal Feature Extraction

GitHub Release PyPI CI codecov Docs
Python PyTorch Code style: ruff License

Exordium is a comprehensive toolkit for multimodal feature extraction across audio, video, and text modalities. It provides preprocessing functions, utility tools, and deep learning wrappers for processing and analyzing multimodal data.

Features

Audio

Functionality Model / Method Output
I/O load, save, resample waveform
Spectral features MFCC, Mel-spectrogram (with pre-emphasis) spectrogram
Low-level descriptors OpenSMILE — eGeMAPSv02 88-d vector
Audio–language embeddings CLAP (laion/larger_clap_music_and_speech) 512-d vector
Speech representations Wav2Vec2 (base-960h / emotion-iemocap) (T, 768)
Speech representations WavLM (microsoft/wavlm-base/base+/large) (T, 768/1024) per layer
Speech emotion features emotion2vec+ (emotion2vec_plus_seed) (T, 768)

Video

Face Detection & Tracking

Functionality Model / Method Output
Face detection YOLOv8-Face (arnabdhar/YOLOv8-Face-Detection) bounding boxes
Face detection + keypoints YOLO11-pose (yolo11n/s-pose_widerface) bounding boxes + 5-pt keypoints
Multi-face tracking IoU-based tracker track IDs across frames
Face-ID tracking AdaFace (IResNet-18/50/101, CVPR 2022) + IoU gating track IDs with identity recovery
Protagonist extraction IoU tracklets + AdaFace global clustering (agglomerative, FaceClusterTracker) single protagonist track

Face Analysis

Functionality Model / Method Output
Dense facial landmarks MediaPipe FaceMesh (face_landmarker.task) 478 × (x, y)
Iris landmarks MediaPipe Iris 71 eye pts + 5 iris pts, EAR, diameters
Head pose 6DRepNet (300W-LP + AFLW2000) yaw, pitch, roll (degrees)
Gaze estimation L2CS-Net (ResNet-50, MPIIFaceGaze) pitch, yaw (radians)
Gaze estimation UniGaze (ViT-based) pitch, yaw (radians)
Eye blink detection BlinkDenseNet121 (DenseNet-121) per-eye open/closed probability
Facial action units OpenGraphAU (Swin-T backbone) 41-dim AU intensity vector

Deep Visual Features

Functionality Model / Method Output
Video features Swin Transformer (tiny/small/base) 768-d / 768-d / 1024-d
Face identity embeddings AdaFace (IResNet-18/50/101, CVPR 2022) 512-d L2-normalised
Face appearance features FAb-Net 256-d
Vision–language embeddings CLIP (ViT-H/14, laion2B) 1024-d
Self-supervised visual features DINOv2 (small/base/large/giant) 384 / 768 / 1024 / 1536-d
Facial expression features EmotiEffNet (EfficientNet-B0/B2, AffectNet) 1280-d / 1408-d
Universal face features FaRL (CLIP ViT-B/16, LAION-Face 20M, CVPR 2022) 512-d
Facial video features MARLIN (ViT, 16-frame clips, CVPR 2023) 384 / 768 / 1024-d

Text

Functionality Model / Method Output
Speech-to-text Whisper (OpenAI) transcript
Word-level timestamps Whisper + torchaudio MMS_FA or whisperX forced alignment word (start, end, score)
Transcript re-alignment fuzzy search of known transcript in word stream (rapidfuzz) segment (start, end, score)
Annotation validation SpeechAlignmentPipeline facade — transcribe, align, evaluate, re-segment metrics + accept/recut/drop decisions
Transcript metrics TranscriptEvaluator — raw + Whisper-normalized WER/CER, rapidfuzz similarity, xml-roberta semantic cosine TranscriptMetrics + is_acceptable
Contextual embeddings BERT (bert-base-uncased) (T, 768)
Contextual embeddings RoBERTa (roberta-large) (T, 1024)
Multilingual embeddings XML-RoBERTa (xlm-roberta-base) (T, 768)

Utilities

  • Device management — GPU/CPU selection via get_torch_device
  • Caching — @load_or_create decorator (safetensors, npy, pkl, fdet, vdet, track)
  • Normalization — global, per-feature, sliding-window
  • Padding — fixed-length sequence padding and masking
  • Loss functions — Bell, ecl1 losses
  • Concurrency — thread- and process-pool helpers

Installation

Requires uv. The video extras include unigaze, which pins timm==0.3.2 (broken with modern PyTorch). uv's override-dependencies in pyproject.toml silently upgrades it to timm>=1.0. Plain pip has no equivalent override mechanism and will fail to resolve this conflict.

uv pip install exordium          # base only
uv pip install exordium[all]     # all optional dependencies
uv pip install exordium[audio]   # audio extras only
uv pip install exordium[video]   # video extras only
uv pip install exordium[text]    # text extras only (includes whisperX)

Install uv if you don't have it yet:

curl -LsSf https://astral.sh/uv/install.sh | sh

Extras

Extra Dependencies
audio OpenSMILE, torchaudio — audio feature extraction
text transformers, torchaudio, rapidfuzz, whisperX — text, speech & alignment
video MediaPipe, Ultralytics, blinklinmult, unigaze, timm — face & video models
all all previously described extras

Development

git clone https://github.com/fodorad/exordium
cd exordium
uv pip install -e ".[all,dev]"
make check   # lint + type-check + test + docs

Documentation

Related Projects

EmotionLinMulT (202X)

Efficient, transformer-based, multi-task emotion detection system.

BlinkLinMulT (2023)

Transformer-based eye blink detection and eye state recognition across 7 public benchmark databases.

PersonalityLinMulT (2022)

LinMulT trained for Big Five personality trait estimation and sentiment analysis.

LinMulT

General-purpose multimodal transformer with linear-complexity attention mechanisms.

Contact

Ádám Fodor — adamfodor.com · fodorad201@gmail.com

Project links

Data verified by PyPI on Jul 14, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2.11.4)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for fodorad from gravatar.com fodorad

Credits

Author: fodorad

GitHub Statistics

Data verified by PyPI on Jul 14, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.13

Provides Extra

all audio dev docs text video

Tags

audio detection multimodal preprocess text tracking utils video

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

exordium-2.9.0.tar.gz (40.4 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

exordium-2.9.0-py3-none-any.whl (200.5 kB view details)

Uploaded Python 3

File details

Details for the file exordium-2.9.0.tar.gz.

File metadata

  • Download URL: exordium-2.9.0.tar.gz
  • Upload date:
  • Size: 40.4 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for exordium-2.9.0.tar.gz
Algorithm Hash digest
SHA256 7a8221fca9f44a770d70f9e55020d973f36372517961aa21671ea35447303bb7
MD5 0b8fa87119caf540fe966ca10dee74bb
BLAKE2b-256 dff2f4710dc28f428bef9a08f5d41b69f8c4103e7f3c3cf10b8fd855b818ee4d

See more details on using hashes here.

Provenance

The following attestation bundles were made for exordium-2.9.0.tar.gz:

Publisher: cd.yml on fodorad/exordium

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file exordium-2.9.0-py3-none-any.whl.

File metadata

  • Download URL: exordium-2.9.0-py3-none-any.whl
  • Upload date:
  • Size: 200.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for exordium-2.9.0-py3-none-any.whl
Algorithm Hash digest
SHA256 4326113c68e6a51e775c57f767ce1b1d4bf8cea2442f53d767abe0528d34701f
MD5 fff1d8d3f9ca7bfefd68dbfb7ae4bdc5
BLAKE2b-256 dbd1c69292b83d7dd73f08db7924b2c26efe862e8120c30824e39e8444c71b11

See more details on using hashes here.

Provenance

The following attestation bundles were made for exordium-2.9.0-py3-none-any.whl:

Publisher: cd.yml on fodorad/exordium

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

2.11.4

2 files

2.11.3

2 files

2.11.2

2 files

2.11.1

2 files

2.11.0

2 files

2.10.1

2 files

2.10.0

2 files

2.9.1

2 files

This release

2.9.0 This release

2 files

2.8.0

2 files

2.7.0

2 files

2.6.2

2 files

2.6.1

2 files

2.6.0

2 files

2.5.0

2 files

2.4.1

2 files

2.4.0

2 files

2.3.1

2 files

2.3.0

2 files

2.2.0

2 files

2.1.0

2 files

2.0.0

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

1.3.0

2 files

1.2.5

2 files

1.2.4

2 files

1.2.0

2 files

1.1.1

2 files

1.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page