languagebind 0.1.0

LanguageBind: multimodal (video/audio/image/depth/thermal) embedding model aligned to text via language-based semantic alignment. Packaged for pip-installability with compatibility patches for modern transformers/torchvision/torchaudio.

languagebind

LanguageBind (ICLR 2024) packaged as a pip-installable library with compatibility patches for modern transformers, torchvision, and torchaudio.

The original LanguageBind repo has no pyproject.toml, so it cannot be installed via pip. This package provides that, plus inline patches for five breaking changes introduced in newer dependency versions.

Installation

pip install languagebind

For video support:

pip install "languagebind[video]"

For audio support:

pip install "languagebind[audio]"

Usage

from languagebind import (
    LanguageBindVideo, LanguageBindVideoProcessor, LanguageBindVideoTokenizer,
    LanguageBindAudio, LanguageBindAudioProcessor, LanguageBindAudioTokenizer,
    LanguageBindImage, LanguageBindImageProcessor, LanguageBindImageTokenizer,
)

Compatibility patches

• _expand_mask / clip_loss: removed from transformers 4.40+, re-implemented in languagebind._compat
• torchaudio.set_audio_backend(): deprecated, guarded with try/except
• torchvision.transforms._transforms_video: private API fallback to public torchvision.transforms equivalents
• CLIPTokenizer.__init__ positional args: changed to keyword args for transformers 4.40+ compatibility

License

MIT — same as the original LanguageBind.