Brain-Score is a platform to evaluate computational models of brain function
on their match to brain measurements in primate vision.
The intent of Brain-Score is to adopt many (ideally all) the experimental benchmarks in the field
for the purpose of model testing, falsification, and comparison.
To that end, Brain-Score operationalizes experimental data into quantitative benchmarks
that any model candidate following the BrainModel interface can be scored on.
Note that you can only access a limited set of public benchmarks when running locally. To score a model on all benchmarks, submit it via the brain-score.org website.
See the documentation for more details, e.g. for submitting a model or benchmark to Brain-Score. For a step-by-step walkthrough on submitting models to the Brain-Score website, see these web tutorials.
See these code examples on scoring models, retrieving data, using and defining benchmarks and metrics. These previous examples might be helpful, but their usage has been deprecated after the 2.0 update.
Brain-Score is made by and for the community. To contribute, please send in a pull request.
Local installation
You will need Python = 3.11 and pip >= 18.1.
pip install brainscore-vision
Test if the installation is successful by scoring a model on a public benchmark:
import brainscore_vision
benchmark = brainscore_vision.load_benchmark('MajajHong2015public.IT-pls')
model = my_model()
score = benchmark(model)
# > <xarray.Score ()>
# > array(0.07637264)
# > Attributes:
# > Attributes:
# > error: <xarray.Score ()>\narray(0.00548197)
# > raw: <xarray.Score ()>\narray(0.22545106)\nAttributes:\...
# > ceiling: <xarray.DataArray ()>\narray(0.81579938)\nAttribut...
# > model_identifier: my-model
# > benchmark_identifier: MajajHong2015public.IT-pls
Some steps may take minutes because data has to be downloaded during first-time use.
Environment Variables
| Variable | Description |
|---|---|
| RESULTCACHING_HOME | directory to cache results (benchmark ceilings) in, ~/.result_caching by default (see https://github.com/brain-score/result_caching) |
License
MIT license
Troubleshooting
`ValueError: did not find HDF5 headers` during netcdf4 installation
pip seems to fail properly setting up the HDF5_DIR required by netcdf4. Use conda: `conda install netcdf4`repeated runs of a benchmark / model do not change the outcome even though code was changed
results (scores, activations) are cached on disk using https://github.com/mschrimpf/result_caching. Delete the corresponding file or directory to clear the cache.CI environment
Add CI related build commands to test_setup.sh. The script is executed in CI environment for unittests.
References
If you use Brain-Score in your work, please cite "Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?" (technical) and "Integrative Benchmarking to Advance Neurally Mechanistic Models of Human Intelligence" (perspective) as well as the respective benchmark sources.
@article{SchrimpfKubilius2018BrainScore,
title={Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?},
author={Martin Schrimpf and Jonas Kubilius and Ha Hong and Najib J. Majaj and Rishi Rajalingham and Elias B. Issa and Kohitij Kar and Pouya Bashivan and Jonathan Prescott-Roy and Franziska Geiger and Kailyn Schmidt and Daniel L. K. Yamins and James J. DiCarlo},
journal={bioRxiv preprint},
year={2018},
url={https://www.biorxiv.org/content/10.1101/407007v2}
}
@article{Schrimpf2020integrative,
title={Integrative Benchmarking to Advance Neurally Mechanistic Models of Human Intelligence},
author={Schrimpf, Martin and Kubilius, Jonas and Lee, Michael J and Murty, N Apurva Ratan and Ajemian, Robert and DiCarlo, James J},
journal={Neuron},
year={2020},
url={https://www.cell.com/neuron/fulltext/S0896-6273(20)30605-X}
}
3 maintainers
Metadata
3 maintainers
Release files for brainscore-vision 2.3.24
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| brainscore_vision-2.3.24.tar.gz | 37.3 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| brainscore_vision-2.3.24-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 76.9 MB
Release files / brainscore_vision-2.3.24.tar.gz
| Download URL | brainscore_vision-2.3.24.tar.gz |
|---|---|
| Size | 37.3 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
03daedf9b43ccb9f53e0a6adb505e7c61c9540f547cf2b0cb1df3c0112a9ab78
|
|
BLAKE2b-256 checksum How to use checksums |
996bdc4ddb61641555151593c9157a12aa3c5fbe7ed3ddae9eed2eb9ffddbc15
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 9, 2026.
Transparency logRelease files / brainscore_vision-2.3.24-py3-none-any.whl
| Download URL | brainscore_vision-2.3.24-py3-none-any.whl |
|---|---|
| Size | 39.6 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
209448b39ce444c2036e4c8d0b18dedc0daf53498ae1a6017ad632a6039a6bb4
|
|
BLAKE2b-256 checksum How to use checksums |
3529d4083f9739faec1d46938b9240b314c71d738e9db915a79d5195680817c5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 9, 2026.
Transparency log
