bioacoustics-model-zoo

Pre-trained models for bioacoustic classification tasks

Suggested Citation

Lapp, S., and Kitzes, J., 2025. "Bioacoustics Model Zoo version 0.13.4". https://github.com/kitzeslab/bioacoustics-model-zoo

Set up / Installation

Option 1: Install from PyPI (Recommended)

1. Create a python environment (3.10-3.13 supported) using conda or your preferred package manager:

conda create -n bmz python=3.11
conda activate bmz

2. Install the package with your desired model dependencies:

# Basic installation (core functionality only)
pip install bioacoustics-model-zoo

# Install with specific model dependencies
pip install bioacoustics-model-zoo[hawkears]  # For HawkEars models
pip install bioacoustics-model-zoo[birdnet]   # For BirdNET model
pip install bioacoustics-model-zoo[tensorflow] # For TensorFlow models (Perch, YAMNet)
pip install bioacoustics-model-zoo[birdset]   # For BirdSet models
pip install bioacoustics-model-zoo[perch]   # For Perch models

# Install all optional dependencies
pip install bioacoustics-model-zoo[all]

Option 2: Install from GitHub (Development)

1. Create a python environment:

conda create -n bmz python=3.11
conda activate bmz

2. Install from GitHub:

pip install git+https://github.com/kitzeslab/bioacoustics-model-zoo

For a specific version:

pip install git+https://github.com/kitzeslab/bioacoustics-model-zoo@0.12.2

3. Install additional dependencies as needed:

# For HawkEars models
pip install timm torch torchvision torchaudio

# For BirdSet models  
pip install torch torchvision torchaudio transformers

# For TensorFlow models (Perch, YAMNet)
pip install tensorflow kagglehub

# For BirdNET (Note: LiteRT not available on Windows yet)
pip install ai-edge-litert

Note that tensorflow installation sometimes requires careful attention to version numbers, see this section below

You can now use the package directly in python:

import bioacoustics_model_zoo as bmz
model = bmz.HawkEars()
model.predict(audio_files) 

See a description of each model and basic usage example below. Also see the transfer learning tutorials on OpenSoundscape.org for detailed advice on fine-tuning models from the Bioacoustics Model Zoo.

If you encounter an issue or a bug, or would like to request a new feature, make a new "Issue" on the Github Issues page. You can also reach out to Sam (sam.lapp@pitt.edu) for more specific inquiries.

Basic usage

List:

List available models in the GitHub repo bioacoustics-model-zoo

import bioacoustics_model_zoo as bmz
bmz.list_models() 

# or, for short textual descriptions: 
bmz.describe_models()

Load:

Get a ready-to-use model object: choose from the models listed in the previous command

model = bmz.BirdNET()

Inference:

Species classification models will have methods for predict, embed, and train.

For instance, use a classification model to generate class presence predictions on an audio file:

audio_file_path = bmz.birds_path # path to a 10-second audio clip
model = bmz.BirdNET()
scores = model.predict([audio_file_path],activation_layer='softmax')
scores

Model list

HawkEars V2

Install the HawkEars package, with version matching the model version you wish to use

pip install hawkears==2.2.0

Usage:

import bioacoustics_model_zoo as bmz
m = bmz.HawkEars2()

# get dataframe with arbitrary-length labels of detected species, one label per row
# this function applies HawkEars post-processing, such as location/date filtering and score heuristics
m.label(['test.wav'])

# additional customization options include:
m.label(
    'test.wav',
    threshold=0.2, #minimum score retained in returned labels
    include_lowband_classifier=False, # True by default, separate classifier for RUGR/SPGR
    class_names="scientific", # use alpha, scientific, common, or ebird for class naming convention
    region="US-PA", # location based filtering, for limited set of supported ebird regions
    date="0601", #MMDD or YYMMDD of date, for filtering
)

# create table of all 3s clip scores for all classes
# no post-processing is applied: returns averaged class outputs across ensembled models
# output matches other opensoundscape/bmz classifiers 
m.predict(['test.wav'])

# get scores on 0.25 second frames for all classes
# no post-processing is applied: returns averaged class outputs across ensembled models
m.predict_frames(['test.wav'])

# note that in general, you can also initialize the file with a config file with customization of inference parameters (see documentation on HawkEars repo)

Note: the model version always matches the version of the installed HawkEars version! For instance, if you need to use HawkEars v2.2.0 models, run pip install hawkears==2.2.0.

Note: training and embedding are not supported by this implementation of HawkEars v2.x

HawkEars v1.x models are implemented separately in the HawkEars class.

Perch V2

Classification and embedding model trained on a large set of annotated bird vocalizations

Preprint:

van Merriënboer, B., Dumoulin, V., Hamer, J., Harrell, L., Burns, A., & Denton, T. (2025). Perch 2.0: The Bittern Lesson for Bioacoustics. arXiv preprint arXiv:2508.04665.

Perch2 requires tensorflow >=2.20.0. For instance, update to the latest versions using:

pip install --upgrade opensoundscape bioacoustics-model-zoo tensorflow kagglehub

We also provide TFLite (bmz.Perch2LiteRT) and ONNX (bmz.Perch2ONNX) alternatives to the TensorFlow version of the Perch V2 model. These may be well suited for scenarios where installing TensorFlow is undesirable. The ONNX version can be loaded without the classification head using the headless=True argument for a much smaller and more efficient embedding-only model.

Example:

import bioacoustics_model_zoo as bmz
m = bmz.Perch2()
m.predict(['test.wav']) # returns dataframe of per-class scores
m.embed(['test.wav']) # returns dataframe of embeddings

# .forward() returns all outputs: a dictionary with 1D embeddings, 
# spatial embeddings, class scores, and spectrograms
m.forward(['test.wav'],batch_size=32,num_workers=4) 

Training:

The .train() method trains a shallow fully-connected neural network as a classification head while keeping the feature extractor frozen, since the Perch2 feature extractor is not trainable.

Please see opensoundscape.org documentation and tutorials for detailed walk through. Once you have multi-hot training and validation label dataframes with (file, start_time, end_time) multi-index and a column for each class, training looks like this:

# load the pre-trained Perch2 tensorflow model
m=bmz.Perch2()
# add a 2-layer PyTorch classification head
m.initialize_custom_classifier(classes=train_df.columns, hidden_layer_sizes=(100,))
# embed the training/validation samples with 5 augmented variations each,
# then fit the classification head
m.train(
  train_df,
  val_df,
  n_augmentation_variants=5,
  embedding_batch_size=64,
  embedding_num_workers=4
)
# save the custom Perch2 model to a file
m.save(save_path)
# later, to reload your fine-tuned Perch2 from the saved object:
# m = bmz.Perch2.load(save_path)

BirdNET

Classification and embedding model trained on a large set of annotated bird vocalizations

Additional required packages:

pip install ai-edge-litert

Example:

import bioacoustics_model_zoo as bmz
m = bmz.BirdNET()
m.predict(['test.wav']) # returns dataframe of per-class scores
m.embed(['test.wav']) # returns dataframe of embeddings

Training:

The .train() method trains a shallow fully-connected neural network as a classification head while keeping the feature extractor frozen, since the BirdNET feature extractor is not open-source.

Please see opensoundscape.org documentation and tutorials for detailed walk through. Once you have multi-hot training and validation label dataframes with (file, start_time, end_time) multi-index and a column for each class, training looks like this:

# load the pre-trained BirdNET tensorflow model
m=bmz.BirdNET()
# add a 2-layer PyTorch classification head
m.initialize_custom_classifier(classes=train_df.columns, hidden_layer_sizes=(100,))
# embed the training/validation samples with 5 augmented variations each,
# then fit the classification head
m.train(
  train_df,
  val_df,
  n_augmentation_variants=5,
  embedding_batch_size=64,
  embedding_num_workers=4
)
# save the custom BirdNET model to a file
m.save(save_path)
# later, to reload your fine-tuned BirdNET from the saved object:
# m = bmz.BirdNET.load(save_path)

BirdNET Occurrence Model

Predict the probability of observing bird species at a location and week of the year based on eBird occurrence data

Example:

# download and initialize the model
from bioacoustics_model_zoo import BirdNETOccurrenceModel
occurrence_model=BirdNETOccurrenceModel()

# get a dataframe listing species likely to occur at a location and week of the year,
# keeping species only with a relative occurrence probability of 10%
occurrence_model.get_species_list(lat=37.419871, lon=-119.153168,week=34,threshold=.1)

BirdCODE

Earth Species Project's BirdCODE sound event detection model

Associated preprint on BioRxiv: https://www.biorxiv.org/content/10.64898/2026.07.31.742086v1

Warning: no pinned versions yet, gets the latest version from the github repo

Warning: Embeddings are very large (188,440 dimensions at frame or clip level)

Warning: The model is large, and you will likely need to use much smaller batch sizes than with most other bioacoustics models. EG Apple M1 Chip: max batch_size=4.

Installation of required dependency:

pip install git+https://github.com/earthspecies/sound-event-detection

Usage:

import bioacoustics_model_zoo as bmz

# initialize the model, downloading from HF or using cached checkpoints
m = bmz.BirdCODE() 
# optionally use common names/alpha/ebird codes, e.g., `BirdCODE(class_names="common")`

# generate 5s window class prediction scores:
m.predict(['test.wav'],batch_size=4)

# generate frame-level predictions  (7.6 frames/sec, 38 frames per 5s clip):
m.predict_frames(['test.wav'])

# extract embeddings for each 5s window:
m.embed(['test.wav'])

# generate frame-level embeddings
m.embed(['test.wav'], avgpool=False, return_dfs=False)

MultiSpeciesWhale

Model developed by Google, trained to detect 11 sound types related to whales

See model card and attributions at: https://www.kaggle.com/models/google/multispecies-whale/TensorFlow2/default/1

• tensorflow pb model
• input is 5s audio windows at 24 kHz
• performs multi-target classification outputs on 11 classes
• embedding shape is 1280 (EfficientNet B0)

Terms of Use: This model has been developed as part of the AI for Nature and Society program at Google. The developers request that users adhere to Google’s AI principles, in particular #1 “Be socially beneficial." in only pursuing applications which have societal and/or environmental benefit, as well as wildlife conservation for not-for-profit decision-making, education, or research. (The official license remains Apache 2.0.) If you have any questions about appropriate use cases for this model, please contact bioacoustics-project@google.com.

Class Common Name	Class Code
Humpback	Mn
Orca	Oo
Bryde's	Be
Minke	Ba
Blue	Bm
Fin	Bp
Right (Atlantic)	Eg
Right (Pacific, upcall)	Upcall
Right (Pacific, gunshot)	Gunshot
Orca echolocation	Echolocation
Orca whistle	Whistle
Orca call	Call

Example Usage:

import bioacoustics_model_zoo as bmz
model=bmz.MultiSpeciesWhale()

# generate logit scores for 3.91s audio windows with 1s step size
predictions = model.predict(['test.wav'], clip_step=1.0) 

# generate 0-1 output scores and increase batch size (use large batch size for GPUs)
model.predict(file, clip_step=1.0, batch_size=32, activation_layer='sigmoid') 

#generate 2048-dimensional embeddings on audio windows
embeddings = model.embed(['test.wav']) 

#get all model outputs: spectrograms, logits, embeddings
all_outputs = model.forward(['test.wav']) 
all_outputs['feature'].shape, all_outputs['logit'].shape, all_outputs['spectrogram'].shape

Environment setup: MultiSpeciesWhale requires tensorflow and kagglehub packages, which can be installed with

pip install --upgrade opensoundscape bioacoustics-model-zoo tensorflow kagglehub

HumpbackWhale

Similar API to MultiSpeciesWhale (see above), but has only one output class for Humpback whale detection. Model details and attributions: https://www.kaggle.com/models/google/humpback-whale

Perch:

Embedding and bird classification model trained on Xeno Canto

Example:

import bioacoustics_model_zoo as bmz
m = bmz.Perch()
predictions = m.predict(['test.wav']) # predict on the model's classes
embeddings = m.embed(['test.wav']) # generate embeddings on each 5 sec of audio

Training: see BirdNET example above, training is equivalent (only trains shallow classifier on frozen feature extractor).

HawkEars

Bird classification model for 314 North American species

Note that HawkEars internally uses an ensemble of 5 CNNs.

Additional required packages:

timm, torchaudio

Example:

import bioacoustics_model_zoo as bmz
m = bmz.HawkEars()
m.predict(['test.wav']) # returns dataframe of per-class scores
m.embed(['test.wav']) # returns dataframe of embeddings

Training: Training this model is equivalent to training the Opensoundscape.CNN class. Please see documentation on opensoundscape.org for detailed examples and walk-throughs.

Because 5 models are ensembled, training is a bit heavy - you may need small batch sizes, and you might consider removing all but one model.

By default, training HawkEars uses a lower learning rate on the feature extractor than on the classifier - a "fine tuning" paradigm. These values can be modified in the .optimizer_params dictionary.

import bioacoustics_model_zoo as bmz
m = bmz.HawkEars()
m.train(train_df,val_df,epochs=10,batch_size=64,num_workers=4)

BirdSet ConvNeXT

Open-source PyTorch model trained on Xeno Canto (global bird species classification)

Rauch, Lukas, et al. "Birdset: A multi-task benchmark for classification in avian bioacoustics." arXiv e-prints (2024): arXiv-2403.

Environment set up:

conda create -n birdset python=3.10
conda activate birdset
pip install opensoundscape transformers torch torchvision torchaudio

Example: predict and embed:

import bioacoustics_model_zoo as bmz
m=bmz.BirdSetConvNeXT()
m.predict(['test.wav'],batch_size=64) # returns dataframe of per-class scores
m.embed(['test.wav']) # returns dataframe of embeddings

Example: train on different set of classes (see OpenSoundscape tutorials for details on training)

import bioacoustics_model_zoo as bmz
import pandas as pd

# load pre-trained network and change output classes
m=bmz.BirdSetConvNeXT()
m.change_classes(['crazy_zebra_grunt','screaming_penguin'])

# optionally, freeze feature extractor (only train final layer)
m.freeze_feature_extractor()

# load one-hot labels and train (index: (file,start_time,end_time))
train_df = pd.read_csv('train_labels.csv',index_col=[0,1,2])
val_df = pd.read_csv('val_labels.csv',index_col=[0,1,2])
m.train(train_df, val_df,batch_size=128, num_workers=8)

BirdSet EfficientNetB1

Open-source PyTorch model trained on Xeno Canto (global bird species classification)

Rauch, Lukas, et al. "Birdset: A multi-task benchmark for classification in avian bioacoustics." arXiv e-prints (2024): arXiv-2403.

Environment set up and examples: see BirdSet ConvNeXT, using m=bmz.BirdSetEfficientNetB1()

MixIT Bird SeparationModel

Separate audio into channels potentially representing separate sources.

This particular model was trained on bird vocalization data.

Additional required packages:

tensorflow, kagglehub

Example:

First, download the checkpoint and metagraph from the MixIt Github repo: install gsutil then run the following command in your terminal:

gsutil -m cp -r gs://gresearch/sound_separation/bird_mixit_model_checkpoints .

Then, use the model in python:

import bioacoustics_model_zoo as bmz
from opensoundscape import Audio

# provide the local path to the checkpoint when creating the object
# this example creates 4 channels; use output_sources8 to separate into 8 channels
model = bmz.SeparationModel(
  checkpoint='/path/to/bird_mixit_model_checkpoints/output_sources4/model.ckpt-3223090',
)

# separate opensoundscape Audio object into 4 channels:
# note that it seems to work best on 5 second segments
a = Audio.from_file('audio.mp3',sample_rate=22050).trim(0,5)
separated = model.separate_audio(a)

# save audio files for each separated channel:
# saves audio files with extensions like _stem0.wav, _stem1.wav, etc
model.load_separate_write('./temp.wav')

YAMNet:

Embedding model trained on AudioSet YouTube

Additional required packages:

tensorflow, kagglehub

Example:

import bioacoustics_model_zoo as bmz
m = bmz.YAMNet()
m.predict(['test.wav']) # returns dataframe of per-class scores
m.embed(['test.wav']) # returns dataframe of embeddings

RanaSierraeCNN:

Detect underwater vocalizations of Rana sierrae, the Sierra Nevada Yellow-legged Frog

example:

import bioacoustics_model_zoo as bmz
m = bmz.RanaSierraeCNN()
m.predict(['test.wav']) # returns dataframe of per-class scores

Other automated detection tools for bioacoustics

RIBBIT

Detect sounds with periodic pulsing patterns.

Implemented in OpenSoundscape as opensoundscape.ribbit.ribbit().

• Python notebooks demonstrating use
• Implementation for R
• Manuscript: Lapp et al 2021

Accelerating and decelerating sequences

Detect pulse trains that accelerate, such as the drumming of Ruffed Grouse (Bonasa umbellus)

Implemented in OpenSoundscape as

opensoundscape.signal_processing.detect_peak_sequence_cwt().

(note that in earlier versions of OpenSoundscape the module is named signal rather than signal_processing)

• Python notebooks demonstrating use
• Manuscript: Lapp et al 2022

Contributing

To contribute a model to the model zoo, email sam.lapp@pitt.edu or add a model yourself:

• fork this repository (help)
• add a .py module in the bioacoustics_model_zoo subfolder implementing a class that instantiates your model object
  • implement the predict() and embed() methods with an API matching the other models in the model zoo
  • optionally implement train() method
  • Note: if you have a pytorch model, you may be able to simply subclass opensoundscape.CNN without needing to override these methods
  • in the docstring, provide an example of use
  • in the docstring, also include a suggested citation for others using the model
  • decorate your class with @register_bmz_model
• add an import statement in __init__.py to import your model class into the top-level package API (from bioacoustics_model_zoo.new_model import NewModel)
• add your model to the Model List below in this document, with example usage
• submit a pull request (GitHub's help page)

Check out any of the existing models for examples of how to complete these steps. In particular, pick the current model class most similar to yours (pytorch vs tensorflow) as a starting point.

Troubleshooting

TensorFlow Installation in Python Environment

Some models in the model zoo require tensorflow to be installed in your python environment.

Installing TensorFlow can be tricky, and it may not be possible to have cuda-enabled tensorflow in the same environment as cuda-enabled pytorch. In this case, you can install a cpu-only version of tensorflow (pip install tensorflow-cpu). You may want to start with a fresh environment, or uninstall tensorflow and nvidia-cudnn-cu11 then reinstall pytorch with the appropriate nvidia-cudnn-cu11, to avoid having the wrong cudnn for PyTorch.

Alternatively, if you want to use the TensorFlow Hub models with GPU acceleration, create an environment where you uninstall pytorch and nvidia-cudnn-cu11 and install a cpu-only version (see this page for the correct installation command). Then, you can pip install tensorflow and let it choose the correct nvidia-cudnn so that it can use CUDA and leverage GPU acceleration.

Installing tensorflow: Carefully follow the directions for your system. Note that models provided in this repo might require the specific nvidia-cudnn-cu11 version 8.6.0, which could conflict with the version required for pytorch.

Error while Downloading TF Hub Models

Some of the models provided in this repo are hosted on the Tensorflow model hub.

If you encounter the following error (or similar) when downloading a TensorFlow Hub model:

ValueError: Trying to load a model of incompatible/unknown type. '/var/folders/d8/265wdp1n0bn_r85dh3pp95fh0000gq/T/tfhub_modules/9616fd04ec2360621642ef9455b84f4b668e219e' contains neither 'saved_model.pb' nor 'saved_model.pbtxt'.

You need to delete the folder listed in the error message (something like /var/folders/...tfhub_modules/....). After deleting that folder, downloading the model should work.

The issue occurs because TensorFlow Hub is looking for a cached model in a temporary folder where it was once stored but no longer exists. See relevant GitHub issue here: https://github.com/tensorflow/hub/issues/896