Exordium

Collection of Preprocessing Functions and Deep Learning Methods for Multimodal Feature Extraction

GitHub Release PyPI CI codecov Docs
Python PyTorch Code style: ruff License

Exordium is a comprehensive toolkit for multimodal feature extraction across audio, video, and text modalities. It provides preprocessing functions, utility tools, and deep learning wrappers for processing and analyzing multimodal data.

Features

Audio

Functionality Model / Method Output
I/O load, save, resample waveform
Spectral features MFCC, Mel-spectrogram (with pre-emphasis) spectrogram
Low-level descriptors OpenSMILE — eGeMAPSv02 88-d vector
Audio–language embeddings CLAP (laion/larger_clap_music_and_speech) 512-d vector
Speech representations Wav2Vec2 (base-960h / emotion-iemocap) (T, 768)
Speech representations WavLM (microsoft/wavlm-base/base+/large) (T, 768/1024) per layer
Speech emotion features emotion2vec+ (emotion2vec_plus_seed) (T, 768)

Video

Face Detection & Tracking

Functionality Model / Method Output
Face detection YOLOv8-Face (arnabdhar/YOLOv8-Face-Detection) bounding boxes
Face detection + keypoints YOLO11-pose (yolo11n/s-pose_widerface) bounding boxes + 5-pt keypoints
Multi-face tracking IoU-based tracker track IDs across frames
Face-ID tracking AdaFace (IResNet-18/50/101, CVPR 2022) + IoU gating track IDs with identity recovery
Protagonist extraction IoU tracklets + AdaFace global clustering (agglomerative, FaceClusterTracker) single protagonist track

Face Analysis

Functionality Model / Method Output
Dense facial landmarks MediaPipe FaceMesh (face_landmarker.task) 478 × (x, y)
Iris landmarks MediaPipe Iris 71 eye pts + 5 iris pts, EAR, diameters
Head pose 6DRepNet (300W-LP + AFLW2000) yaw, pitch, roll (degrees)
Gaze estimation L2CS-Net (ResNet-50, MPIIFaceGaze) pitch, yaw (radians)
Gaze estimation UniGaze (ViT-based) pitch, yaw (radians)
Facial action units OpenGraphAU (Swin-T backbone) 41-dim AU intensity vector

Deep Visual Features

Functionality Model / Method Output
Video features Swin Transformer (tiny/small/base) 768-d / 768-d / 1024-d
Face identity embeddings AdaFace (IResNet-18/50/101, CVPR 2022) 512-d L2-normalised
Face appearance features FAb-Net 256-d
Vision–language embeddings CLIP (ViT-H/14, laion2B) 1024-d
Self-supervised visual features DINOv2 (small/base/large/giant) 384 / 768 / 1024 / 1536-d
Facial expression features EmotiEffNet (EfficientNet-B0/B2, AffectNet) 1280-d / 1408-d
Universal face features FaRL (CLIP ViT-B/16, LAION-Face 20M, CVPR 2022) 512-d
Facial video features MARLIN (ViT, 16-frame clips, CVPR 2023) 384 / 768 / 1024-d

Text

Functionality Model / Method Output
Speech-to-text Whisper (OpenAI) transcript
Word-level timestamps Whisper + torchaudio MMS_FA or whisperX forced alignment word (start, end, score)
Transcript re-alignment fuzzy search of known transcript in word stream (rapidfuzz) segment (start, end, score)
Annotation validation SpeechAlignmentPipeline facade — transcribe, align, evaluate, re-segment metrics + accept/recut/drop decisions
Transcript metrics TranscriptEvaluator — raw + Whisper-normalized WER/CER, rapidfuzz similarity, xml-roberta semantic cosine TranscriptMetrics + is_acceptable
Contextual embeddings BERT (bert-base-uncased) (T, 768)
Contextual embeddings RoBERTa (roberta-large) (T, 1024)
Multilingual embeddings XML-RoBERTa (xlm-roberta-base) (T, 768)

Utilities

  • Device management — GPU/CPU selection via get_torch_device
  • Caching — @load_or_create decorator (safetensors, npy, pkl, fdet, vdet, track)
  • Normalization — global, per-feature, sliding-window
  • Padding — fixed-length sequence padding and masking
  • Loss functions — Bell, ecl1 losses
  • Concurrency — thread- and process-pool helpers

Installation

Requires uv. The video extras include unigaze, which pins timm==0.3.2 (broken with modern PyTorch). uv's override-dependencies in pyproject.toml silently upgrades it to timm>=1.0. Plain pip has no equivalent override mechanism and will fail to resolve this conflict.

uv pip install exordium          # base only
uv pip install exordium[all]     # all optional dependencies
uv pip install exordium[audio]   # audio extras only
uv pip install exordium[video]   # video extras only
uv pip install exordium[text]    # text extras only (includes whisperX)

Install uv if you don't have it yet:

curl -LsSf https://astral.sh/uv/install.sh | sh

Extras

Extra Dependencies
audio OpenSMILE, torchaudio — audio feature extraction
text transformers, torchaudio, rapidfuzz, whisperX — text, speech & alignment
video MediaPipe, Ultralytics, unigaze, timm — face & video models
all all previously described extras

Development

git clone https://github.com/fodorad/exordium
cd exordium
uv pip install -e ".[all,dev]"
make check   # lint + type-check + test + docs

Documentation

Related Projects

EmotionLinMulT (202X)

Efficient, transformer-based, multi-task emotion detection system.

BlinkLinMulT (2023)

Transformer-based eye blink detection and eye state recognition across 7 public benchmark databases.

PersonalityLinMulT (2022)

LinMulT trained for Big Five personality trait estimation and sentiment analysis.

LinMulT

General-purpose multimodal transformer with linear-complexity attention mechanisms.

Contact

Ádám Fodor — adamfodor.com · fodorad201@gmail.com

Project links

Data verified by PyPI on Jul 26, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2.11.4)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for fodorad from gravatar.com fodorad

Credits

Author: fodorad

GitHub Statistics

Data verified by PyPI on Jul 26, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.13

Provides Extra

all audio dev docs text video

Tags

audio detection multimodal preprocess text tracking utils video

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

exordium-2.11.1.tar.gz (39.3 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

exordium-2.11.1-py3-none-any.whl (203.7 kB view details)

Uploaded Python 3

File details

Details for the file exordium-2.11.1.tar.gz.

File metadata

  • Download URL: exordium-2.11.1.tar.gz
  • Upload date:
  • Size: 39.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for exordium-2.11.1.tar.gz
Algorithm Hash digest
SHA256 be5c1f533a3003abde7559bf9d95ae8e280082a8950555dcab9769dbc727259f
MD5 b8a3d75102d8cd340e255c76f9bff976
BLAKE2b-256 e07820c373c4df7f3a558b9161a995c0319dfefa727e55fba9adaecc6009a3a9

See more details on using hashes here.

Provenance

The following attestation bundles were made for exordium-2.11.1.tar.gz:

Publisher: cd.yml on fodorad/exordium

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file exordium-2.11.1-py3-none-any.whl.

File metadata

  • Download URL: exordium-2.11.1-py3-none-any.whl
  • Upload date:
  • Size: 203.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for exordium-2.11.1-py3-none-any.whl
Algorithm Hash digest
SHA256 bca6700dc0cc4241965b567c9e48fb7554a70b52fcf7c42e0cfe4309cf8ce01f
MD5 6c1b5c39e938a084b2f9f373dbc46e81
BLAKE2b-256 b677680b96444f001911d451d3c9acd77f0b963e6c4111f1da3d496e5d1d0f1c

See more details on using hashes here.

Provenance

The following attestation bundles were made for exordium-2.11.1-py3-none-any.whl:

Publisher: cd.yml on fodorad/exordium

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

2.11.4

2 files

2.11.3

2 files

2.11.2

2 files

This release

2.11.1 This release

2 files

2.11.0

2 files

2.10.1

2 files

2.10.0

2 files

2.9.1

2 files

2.9.0

2 files

2.8.0

2 files

2.7.0

2 files

2.6.2

2 files

2.6.1

2 files

2.6.0

2 files

2.5.0

2 files

2.4.1

2 files

2.4.0

2 files

2.3.1

2 files

2.3.0

2 files

2.2.0

2 files

2.1.0

2 files

2.0.0

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

1.3.0

2 files

1.2.5

2 files

1.2.4

2 files

1.2.0

2 files

1.1.1

2 files

1.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page