01What Is TRIBE V2 Trimodal Brain Encoder?

TRIBE V2 Trimodal Brain Encoder is an AI built by Meta that predicts exactly how a human brain will react to anything you see, hear, or read. You feed it a video, an audio clip, or a piece of text, and it tells you how a brain would respond across 70,000 individual data points. All without putting anyone in an fMRI scanner.

Think of it as a digital twin of neural activity. It is like a flight simulator for the brain. You can test how the brain responds to any content in software before running a real experiment.

That saves months of time and millions of dollars in scanner costs.

02Main Features of TRIBE V2

  • Trimodal input – It handles video, audio, and text separately. Three AI models each read one sense: VJEPA 2 for video, WAVE to VEC to BERT for audio, and LLAMA 3.2 (3 billion parameters) for text.
  • Prediction over 70,000 voxels – The model maps brain activity across a huge number of tiny brain units, not just big regions. This is 70 times more than the original TRIBE.
  • Zero-shot generalization – It can predict brain responses from a person it has never seen before. New people, new languages, new content types work without retraining.
  • Scaling law – Performance improves log-linearly with more fMRI data. There is no ceiling in sight. More data will keep making it better.
  • Noise reduction – The model learns the clean signal from 700 people and filters out scanner noise, heartbeat artifacts, and head movement. Its predictions are sometimes more accurate than a real scan.
  • Open release – Meta published the model weights, code on GitHub, a research paper accepted at ICLR 2026, and an interactive demo.

03How to Set Up TRIBE V2

  1. Go to Meta's GitHub page and download the model weights and full codebase for TRIBE V2.
  2. Ensure you have the required hardware. The model runs best on a machine with a strong GPU and at least 16 GB of RAM.
  3. Install the dependencies listed in the repository, including Python, PyTorch, and the multimodal encoders (VJEPA 2, WAVE to VEC to BERT, and LLAMA 3.2).
  4. Load the pretrained model using the provided scripts. Make sure you are using Python version 3.10 or newer.
  5. Prepare your input: a video file, an audio file, or a text transcript. The model can accept any combination of the three modalities.
  6. Run the inference script that feeds your input through the three-stage pipeline. The model will output a predicted fMRI scan as a 3D brain map with 70,000 voxel values.
  7. Visualize the prediction using the demo tool or save the data for further analysis.

04How to Use TRIBE V2 for Brain Predictions

Julian Goldie explains the three-stage process TRIBE V2 uses to predict brain activity. Stage 1 is encoding. Three separate AI models each read one sense: VJEPA 2 watches the video, WAVE to VEC to BERT listens to the audio, and LLAMA 3.2 reads the text transcript. Each one turns its input into a dense numerical description of what is happening moment by moment.

Stage 2 is fusion. An 8-layer transformer merges the three streams. This is needed because the brain does not process sight, sound, and language in isolation.

It weaves them together. The transformer learns that same weaving.

Stage 3 is mapping. The merged output is mapped onto brain activity. Not just brain regions, but 70,000 individual voxels across the whole brain.

The result is a predicted fMRI scan. You can then examine that scan to see how a brain would respond to any content.

For example, Meta tested the model with a Charlie Chaplin silent film. That test had no audio. The model handled it because it learned patterns from all three modalities and still predicted brain responses accurately.

05Results, Tests, and Comparisons

TRIBE V2 was tested in the Algonauts 2025 competition, run by MIT, Universal Berlin, and Goethe University Frankfurt. There were 263 teams from labs worldwide. The challenge was to predict brain responses to nearly 80 hours of movies, including a silent black-and-white Charlie Chaplin film.

TRIBE V1 won first place. The new V2 scales that same architecture with 70 times more brain resolution and 175 times more training subjects.

In direct comparisons, TRIBE V2 improved 2-3 times over previous methods for both movies and audiobooks. It explains about 54% of the explainable brain variance. That means nearly half of what the brain does is still unpredicted, but at 70,000 voxel resolution and zero-shot, it is a genuine leap.

The model is so clean that its predictions are sometimes more accurate than a real fMRI scan. Real scans have noise from heartbeat, head movement, and the scanner itself. TRIBE V2 averages over 700 people and outputs the clean signal.

06Tips, Limits, and Mistakes to Avoid

TRIBE V2 does not read your mind. It predicts brain reactions to external stimuli. It cannot decode private thoughts, intentions, emotions, or internal monologue. That gap is enormous and far from crossed.

Avoid the hype that says otherwise.

It explains only 54% of explainable brain variance. 46% remains unpredicted. Do not treat it as a complete brain map. Use it as a screening tool, not a final answer.

Real world is messier. The model was trained on people watching movies in a controlled environment. In the real world, attention drifts and environments change. Be careful about reverse inference: just because a brain region activates does not mean you know the psychological state behind it.

Privacy laws are changing. Multiple US states are passing neural data privacy laws. Meta released TRIBE V2 under a non-commercial license, but the code is public. Be aware of potential misuse.

The UN has raised concerns about brain prediction technologies.

07Who TRIBE V2 Is For and Why It Matters

TRIBE V2 is for neuroscience researchers, healthcare professionals, and drug developers. Researchers can screen hypotheses computationally in hours instead of months. They can test 500 audio clips instead of five. This speeds up brain science massively.

Healthcare can use it to compare predictions from a healthy brain against real scans from patients with aphasia, PTSD, or sensory processing disorders. The gaps show where neural signaling has broken down, which is useful for diagnosis.

Drug development can screen compounds that target the brain before human trials. This could reduce the brutal failure rate of brain drugs in clinical trials.

The feedback loop is also important. TRIBE V2 helps understand how the brain runs on 20 watts while processing sight, sound, and language simultaneously. Insights from that can make AI more efficient.

Both loops are running at the same time, and TRIBE V2 is a key tool in that process.