TRIBE v2 is a deep multimodal brain encoding model from Meta AI that predicts fMRI brain responses to naturalistic stimuli. It maps text, audio, and video through a Fusion Transformer onto the fsaverage5 cortical surface (~20k vertices).

Quick Start

pip install cognitive-scoring

Two-line API

from tribev2 import BrainAPI

api = BrainAPI.load()
result = api.analyze("She opened the letter and tears of joy streamed down her face.")

print(result.valence)         # +0.031  (positive = happy)
print(result.learning)        # 0.577   (deeper cognitive processing)
print(result.attention)       # 0.600   (focused attention)
print(result.scores)          # {"prefrontal": 0.62, "temporal": 0.63, ...}
print(result.classification)  # [("happy", 0.85), ("calm", 0.32), ...]
print(result.summary())       # Full formatted output with bar charts

HTTP API

pip install cognitive-scoring[server]
python -c "from tribev2.server import main; main()"

curl -X POST http://localhost:8000/analyze \
  -H "Content-Type: application/json" \
  -d '{"text": "She opened the letter and tears of joy streamed down her face."}'

Interactive docs at http://localhost:8000/docs.

Lower-level API

from tribev2 import TribeModel

model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

df = model.get_events_dataframe(text_path="story.txt")
preds, segments = model.predict(events=df)
print(preds.shape)  # (n_timesteps, 20484)

Installation

# Core (text + audio inference)
pip install cognitive-scoring

# With video support (adds torchvision + moviepy)
pip install cognitive-scoring[video]

# With HTTP API server
pip install cognitive-scoring[server]

# With brain visualization (3D surface plots)
pip install cognitive-scoring[plotting]

# Everything except training
pip install cognitive-scoring[all]

# For development
pip install -e ".[all,test]"

Optional extras

What You Get

Region Scores (0–1)

Each text is scored on 10 functional brain region groups. A score of 0.5 = baseline; above means more activation than average.

Composite Scores

Performance

Feature extraction (V-JEPA, Wav2Vec, LLaMA 3.2) is the bottleneck. Results are cached by content hash; repeated analysis of the same text is instant.

Training

export DATAPATH="/path/to/studies"
export SAVEPATH="/path/to/output"

# Local test run
python -m tribev2.grids.test_run

# Grid search on Slurm
python -m tribev2.grids.run_cortical
python -m tribev2.grids.run_subcortical

Project Structure

tribev2/
├── api.py               # BrainAPI: simple two-line interface
├── server.py            # FastAPI HTTP server
├── demo_utils.py        # TribeModel: model loading + inference
├── brain_states.py      # BrainAtlas, scoring, classification
├── model.py             # FmriEncoder: Fusion Transformer architecture
├── main.py              # Data + TribeExperiment pipeline
├── _mps_compat.py       # Apple Silicon MPS patches
├── eventstransforms.py  # Text/audio/video → events
├── plotting/            # Brain visualization backends
└── studies/             # Dataset definitions

Caveats

• Predictions are cortical surface only — subcortical structures (amygdala, hippocampus, basal ganglia) are NOT represented.
• All cognitive/emotional labels are approximations based on cortical correlates.
• NOT suitable for clinical diagnosis or treatment decisions.

Citation

@article{dAscoli2026TribeV2,
  title={A foundation model of vision, audition, and language for in-silico neuroscience},
  author={d'Ascoli, St{\'e}phane and Rapin, J{\'e}r{\'e}my and Benchetrit, Yohann and Brookes, Teon and Begany, Katelyn and Raugel, Jos{\'e}phine and Banville, Hubert and King, Jean-R{\'e}mi},
  year={2026}
}

License

Copyright © Meta Platforms, Inc. and affiliates. All rights reserved.

This work is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).

You may use, share, and adapt this material for non-commercial purposes only, provided you give appropriate credit, indicate if changes were made, and do not impose additional restrictions. See LICENSE for the full legal text.

Disclaimer of Warranties (§5): This software is provided "AS-IS" and "AS-AVAILABLE" without any warranties of any kind, express or implied, including but not limited to warranties of merchantability, fitness for a particular purpose, or non-infringement. In no event shall the licensor be liable for any damages arising from use of this software.

Modifications: This repository contains modifications to the original TRIBE v2 codebase by Meta Platforms, Inc., including (among other things) a high-level Python API, an HTTP server, ROI-based brain-state scoring, and Apple Silicon compatibility patches. For a complete record of all changes, see the git history. These modifications are also licensed under CC BY-NC 4.0.

Contributing

See CONTRIBUTING.md for how to get involved.