Exordium

Collection of Preprocessing Functions and Deep Learning Methods for Multimodal Feature Extraction

GitHub Release PyPI CI codecov Docs
Python PyTorch Code style: ruff License

Exordium is a comprehensive toolkit for multimodal feature extraction across audio, video, and text modalities. It provides preprocessing functions, utility tools, and deep learning wrappers for processing and analyzing multimodal data.

Features

Audio

Functionality Model / Method Output
I/O load, save, resample waveform
Spectral features MFCC, Mel-spectrogram (with pre-emphasis) spectrogram
Low-level descriptors OpenSMILE — eGeMAPSv02 88-d vector
Audio–language embeddings CLAP (laion/larger_clap_music_and_speech) 512-d vector
Speech representations Wav2Vec2 (base-960h / emotion-iemocap) (T, 768)
Speech representations WavLM (microsoft/wavlm-base/base+/large) (T, 768/1024) per layer
Speech emotion features emotion2vec+ (emotion2vec_plus_seed) (T, 768)

Video

Face Detection & Tracking

Functionality Model / Method Output
Face detection YOLOv8-Face (arnabdhar/YOLOv8-Face-Detection) bounding boxes
Face detection + keypoints YOLO11-pose (yolo11n/s-pose_widerface) bounding boxes + 5-pt keypoints
Multi-face tracking IoU-based tracker track IDs across frames
Face-ID tracking AdaFace (IResNet-18/50/101, CVPR 2022) + IoU gating track IDs with identity recovery
Protagonist extraction IoU tracklets + AdaFace global clustering (agglomerative, FaceClusterTracker) single protagonist track

Face Analysis

Functionality Model / Method Output
Dense facial landmarks MediaPipe FaceMesh (face_landmarker.task) 478 × (x, y)
Iris landmarks MediaPipe Iris 71 eye pts + 5 iris pts, EAR, diameters
Head pose 6DRepNet (300W-LP + AFLW2000) yaw, pitch, roll (degrees)
Gaze estimation L2CS-Net (ResNet-50, MPIIFaceGaze) pitch, yaw (radians)
Gaze estimation UniGaze (ViT-based) pitch, yaw (radians)
Facial action units OpenGraphAU (Swin-T backbone) 41-dim AU intensity vector

Deep Visual Features

Functionality Model / Method Output
Video features Swin Transformer (tiny/small/base) 768-d / 768-d / 1024-d
Face identity embeddings AdaFace (IResNet-18/50/101, CVPR 2022) 512-d L2-normalised
Face appearance features FAb-Net 256-d
Vision–language embeddings CLIP (ViT-H/14, laion2B) 1024-d
Self-supervised visual features DINOv2 (small/base/large/giant) 384 / 768 / 1024 / 1536-d
Facial expression features EmotiEffNet (EfficientNet-B0/B2, AffectNet) 1280-d / 1408-d
Universal face features FaRL (CLIP ViT-B/16, LAION-Face 20M, CVPR 2022) 512-d
Facial video features MARLIN (ViT, 16-frame clips, CVPR 2023) 384 / 768 / 1024-d

Text

Functionality Model / Method Output
Speech-to-text Whisper (OpenAI) transcript
Word-level timestamps Whisper + torchaudio MMS_FA or whisperX forced alignment word (start, end, score)
Transcript re-alignment fuzzy search of known transcript in word stream (rapidfuzz) segment (start, end, score)
Annotation validation SpeechAlignmentPipeline facade — transcribe, align, evaluate, re-segment metrics + accept/recut/drop decisions
Transcript metrics TranscriptEvaluator — raw + Whisper-normalized WER/CER, rapidfuzz similarity, xml-roberta semantic cosine TranscriptMetrics + is_acceptable
Contextual embeddings BERT (bert-base-uncased) (T, 768)
Contextual embeddings RoBERTa (roberta-large) (T, 1024)
Multilingual embeddings XML-RoBERTa (xlm-roberta-base) (T, 768)

Utilities

  • Device management — GPU/CPU selection via get_torch_device
  • Caching — @load_or_create decorator (safetensors, npy, pkl, fdet, vdet, track)
  • Normalization — global, per-feature, sliding-window
  • Padding — fixed-length sequence padding and masking
  • Loss functions — Bell, ecl1 losses
  • Concurrency — thread- and process-pool helpers

Installation

Requires uv. The video extras include unigaze, which pins timm==0.3.2 (broken with modern PyTorch). uv's override-dependencies in pyproject.toml silently upgrades it to timm>=1.0. Plain pip has no equivalent override mechanism and will fail to resolve this conflict.

uv pip install exordium          # base only
uv pip install exordium[all]     # all optional dependencies
uv pip install exordium[audio]   # audio extras only
uv pip install exordium[video]   # video extras only
uv pip install exordium[text]    # text extras only (includes whisperX)

Install uv if you don't have it yet:

curl -LsSf https://astral.sh/uv/install.sh | sh

Extras

Extra Dependencies
audio OpenSMILE, torchaudio — audio feature extraction
text transformers, torchaudio, rapidfuzz, whisperX — text, speech & alignment
video MediaPipe, Ultralytics, unigaze, timm — face & video models
all all previously described extras

Development

git clone https://github.com/fodorad/exordium
cd exordium
uv pip install -e ".[all,dev]"
make check   # lint + type-check + test + docs

Documentation

Related Projects

EmotionLinMulT (202X)

Efficient, transformer-based, multi-task emotion detection system.

BlinkLinMulT (2023)

Transformer-based eye blink detection and eye state recognition across 7 public benchmark databases.

PersonalityLinMulT (2022)

LinMulT trained for Big Five personality trait estimation and sentiment analysis.

LinMulT

General-purpose multimodal transformer with linear-complexity attention mechanisms.

Contact

Ádám Fodor — adamfodor.com · fodorad201@gmail.com

Project links

Data verified by PyPI on Aug 9, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2.11.4)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for fodorad from gravatar.com fodorad

Credits

Author: fodorad

GitHub Statistics

Data verified by PyPI on Aug 9, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.12

Provides Extra

all audio dev docs text video

Tags

audio detection multimodal preprocess text tracking utils video

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

exordium-2.11.2.tar.gz (39.3 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

exordium-2.11.2-py3-none-any.whl (203.7 kB view details)

Uploaded Python 3

File details

Details for the file exordium-2.11.2.tar.gz.

File metadata

  • Download URL: exordium-2.11.2.tar.gz
  • Upload date:
  • Size: 39.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for exordium-2.11.2.tar.gz
Algorithm Hash digest
SHA256 883b62a649b234ca747aabef2bfe0d423c844e7b3abd8c08dcc24487d979324f
MD5 638e1ee7cde2297290af800806e1e0b4
BLAKE2b-256 9f70d4c565b2ac46f6b07711b0dd4f5a125a2728ab81e47fc27dc82db98dfd41

See more details on using hashes here.

Provenance

The following attestation bundles were made for exordium-2.11.2.tar.gz:

Publisher: cd.yml on fodorad/exordium

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file exordium-2.11.2-py3-none-any.whl.

File metadata

  • Download URL: exordium-2.11.2-py3-none-any.whl
  • Upload date:
  • Size: 203.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for exordium-2.11.2-py3-none-any.whl
Algorithm Hash digest
SHA256 91bb504e1fbb7c80d1926a89dcdac8215a657317252bd98be601950c3b216126
MD5 6a71078d62d3ad123244ef76ff6c1d07
BLAKE2b-256 5b3b7b080289b8e6113062986c9abade63133154715ebe97f5d91fd0655b38c4

See more details on using hashes here.

Provenance

The following attestation bundles were made for exordium-2.11.2-py3-none-any.whl:

Publisher: cd.yml on fodorad/exordium

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

2.11.4

2 files

2.11.3

2 files

This release

2.11.2 This release

2 files

2.11.1

2 files

2.11.0

2 files

2.10.1

2 files

2.10.0

2 files

2.9.1

2 files

2.9.0

2 files

2.8.0

2 files

2.7.0

2 files

2.6.2

2 files

2.6.1

2 files

2.6.0

2 files

2.5.0

2 files

2.4.1

2 files

2.4.0

2 files

2.3.1

2 files

2.3.0

2 files

2.2.0

2 files

2.1.0

2 files

2.0.0

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

1.3.0

2 files

1.2.5

2 files

1.2.4

2 files

1.2.0

2 files

1.1.1

2 files

1.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page