Exordium

Collection of Preprocessing Functions and Deep Learning Methods for Multimodal Feature Extraction

GitHub Release PyPI CI codecov Docs
Python PyTorch Code style: ruff License

Exordium is a comprehensive toolkit for multimodal feature extraction across audio, video, and text modalities. It provides preprocessing functions, utility tools, and deep learning wrappers for processing and analyzing multimodal data.

Features

Audio

Functionality Model / Method Output
I/O load, save, resample waveform
Spectral features MFCC, Mel-spectrogram (with pre-emphasis) spectrogram
Low-level descriptors OpenSMILE — eGeMAPSv02 88-d vector
Audio–language embeddings CLAP (laion/larger_clap_music_and_speech) 512-d vector
Speech representations Wav2Vec2 (base-960h / emotion-iemocap) (T, 768)
Speech representations WavLM (microsoft/wavlm-base/base+/large) (T, 768/1024) per layer
Speech emotion features emotion2vec+ (emotion2vec_plus_seed) (T, 768)

Video

Face Detection & Tracking

Functionality Model / Method Output
Face detection YOLOv8-Face (arnabdhar/YOLOv8-Face-Detection) bounding boxes
Face detection + keypoints YOLO11-pose (yolo11n/s-pose_widerface) bounding boxes + 5-pt keypoints
Multi-face tracking IoU-based tracker track IDs across frames
Face-ID tracking AdaFace (IResNet-18/50/101, CVPR 2022) + IoU gating track IDs with identity recovery
Protagonist extraction IoU tracklets + AdaFace global clustering (agglomerative, FaceClusterTracker) single protagonist track

Face Analysis

Functionality Model / Method Output
Dense facial landmarks MediaPipe FaceMesh (face_landmarker.task) 478 × (x, y)
Iris landmarks MediaPipe Iris 71 eye pts + 5 iris pts, EAR, diameters
Head pose 6DRepNet (300W-LP + AFLW2000) yaw, pitch, roll (degrees)
Gaze estimation L2CS-Net (ResNet-50, MPIIFaceGaze) pitch, yaw (radians)
Facial action units OpenGraphAU (Swin-T backbone) 41-dim AU intensity vector

Deep Visual Features

Functionality Model / Method Output
Video features Swin Transformer (tiny/small/base) 768-d / 768-d / 1024-d
Face identity embeddings AdaFace (IResNet-18/50/101, CVPR 2022) 512-d L2-normalised
Face appearance features FAb-Net 256-d
Vision–language embeddings CLIP (ViT-H/14, laion2B) 1024-d
Self-supervised visual features DINOv2 (small/base/large/giant) 384 / 768 / 1024 / 1536-d
Facial expression features EmotiEffNet (EfficientNet-B0/B2, AffectNet) 1280-d / 1408-d
Universal face features FaRL (CLIP ViT-B/16, LAION-Face 20M, CVPR 2022) 512-d
Facial video features MARLIN (ViT, 16-frame clips, CVPR 2023) 384 / 768 / 1024-d

Text

Functionality Model / Method Output
Speech-to-text Whisper (OpenAI) transcript
Word-level timestamps Whisper + torchaudio MMS_FA or whisperX forced alignment word (start, end, score)
Transcript re-alignment fuzzy search of known transcript in word stream (rapidfuzz) segment (start, end, score)
Annotation validation SpeechAlignmentPipeline facade — transcribe, align, evaluate, re-segment metrics + accept/recut/drop decisions
Transcript metrics TranscriptEvaluator — raw + Whisper-normalized WER/CER, rapidfuzz similarity, xml-roberta semantic cosine TranscriptMetrics + is_acceptable
Contextual embeddings BERT (bert-base-uncased) (T, 768)
Contextual embeddings RoBERTa (roberta-large) (T, 1024)
Multilingual embeddings XML-RoBERTa (xlm-roberta-base) (T, 768)

Utilities

  • Device management — GPU/CPU selection via get_torch_device
  • Caching — @load_or_create decorator (safetensors, npy, pkl, fdet, vdet, track)
  • Normalization — global, per-feature, sliding-window
  • Padding — fixed-length sequence padding and masking
  • Loss functions — Bell, ecl1 losses
  • Concurrency — thread- and process-pool helpers

Installation

Requires uv.

uv pip install exordium          # base only
uv pip install exordium[all]     # all optional dependencies
uv pip install exordium[audio]   # audio extras only
uv pip install exordium[video]   # video extras only
uv pip install exordium[text]    # text extras only (includes whisperX)

Install uv if you don't have it yet:

curl -LsSf https://astral.sh/uv/install.sh | sh

Extras

Extra Dependencies
audio OpenSMILE, torchaudio — audio feature extraction
text transformers, torchaudio, rapidfuzz, whisperX — text, speech & alignment
video MediaPipe, Ultralytics, timm — face & video models
all all previously described extras

Development

git clone https://github.com/fodorad/exordium
cd exordium
uv pip install -e ".[all,dev]"
make check   # lint + type-check + test + docs

Documentation

Related Projects

EmotionLinMulT (202X)

Efficient, transformer-based, multi-task emotion detection system.

BlinkLinMulT (2023)

Transformer-based eye blink detection and eye state recognition across 7 public benchmark databases.

PersonalityLinMulT (2022)

LinMulT trained for Big Five personality trait estimation and sentiment analysis.

LinMulT

General-purpose multimodal transformer with linear-complexity attention mechanisms.

Contact

Ádám Fodor — adamfodor.com · fodorad201@gmail.com

Project links

Data verified by PyPI on Sep 8, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for fodorad from gravatar.com fodorad

Credits

Author: fodorad

GitHub Statistics

Data verified by PyPI on Sep 8, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.12

Provides Extra

all audio dev docs text video

Tags

audio detection multimodal preprocess text tracking utils video

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

exordium-2.11.4.tar.gz (39.4 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

exordium-2.11.4-py3-none-any.whl (202.0 kB view details)

Uploaded Python 3

File details

Details for the file exordium-2.11.4.tar.gz.

File metadata

  • Download URL: exordium-2.11.4.tar.gz
  • Upload date:
  • Size: 39.4 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for exordium-2.11.4.tar.gz
Algorithm Hash digest
SHA256 f171259c705f09a256968ee8808e3c200fc5bdb0d9d83bccc3276d7da1edc6d9
MD5 37d8b4891062ec34b5727c3e60e22816
BLAKE2b-256 3fafd1e9fb29366824635e09c19d8aabb3c814d7c8bdacfc4c8aeb5376e16607

See more details on using hashes here.

Provenance

The following attestation bundles were made for exordium-2.11.4.tar.gz:

Publisher: cd.yml on fodorad/exordium

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file exordium-2.11.4-py3-none-any.whl.

File metadata

  • Download URL: exordium-2.11.4-py3-none-any.whl
  • Upload date:
  • Size: 202.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for exordium-2.11.4-py3-none-any.whl
Algorithm Hash digest
SHA256 99057ae0c9cc9a4fa2f9d3c8a02dad907ceeef299d1c6a73a25dd46ec6575250
MD5 426a682854dda807d91a378dcdf59b66
BLAKE2b-256 d94966e6111d59fc36dd5a3802564d81ab67dab7bae8c1708c2989e59659b382

See more details on using hashes here.

Provenance

The following attestation bundles were made for exordium-2.11.4-py3-none-any.whl:

Publisher: cd.yml on fodorad/exordium

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

2.11.4 This release

2 files

2.11.3

2 files

2.11.2

2 files

2.11.1

2 files

2.11.0

2 files

2.10.1

2 files

2.10.0

2 files

2.9.1

2 files

2.9.0

2 files

2.8.0

2 files

2.7.0

2 files

2.6.2

2 files

2.6.1

2 files

2.6.0

2 files

2.5.0

2 files

2.4.1

2 files

2.4.0

2 files

2.3.1

2 files

2.3.0

2 files

2.2.0

2 files

2.1.0

2 files

2.0.0

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

1.3.0

2 files

1.2.5

2 files

1.2.4

2 files

1.2.0

2 files

1.1.1

2 files

1.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page