Skip to main content

Sound Event Detection

This repository implement a class which allow to build classifier for audio signal following the SED (Sound Event Detection) architecture. The model create as well mel spectrogram and use CNN backbone. This structure has been used during the birdcall competition which allow me to reach the 3th position in 2021.

This is implemented on pytorch and tensorflow, however the tensorflow version has not been tested for training purpose (only inference). I would advice to use only the pytorch version.

Installation

pip install audio-sed

if you use tensorflow:

pip install tflibrosa

Examples

Pytorch

import torch
import timm
from torch import nn
import numpy as np
from audio_sed.sed_config import ConfigSED
from audio_sed.pytorch.sed_models import AudioClassifierSequenceSED, AudioSED, shape_from_backbone

def load_model(model_name, num_classe, cfg_sed):
    backbone = timm.create_model( model_name, pretrained=False)
    if "efficientnet" in   model_name:
        backbone.global_pool =  nn.Identity()
        in_feat = backbone.classifier.in_features
        backbone.classifier = nn.Identity()
    elif "convnext" in cfg.model_name:
        in_feat = backbone.head.fc.in_features
        backbone.head = nn.Identity()

    in_features = shape_from_backbone(inputs=torch.as_tensor(np.random.uniform(0, 1, (1, int(5 * cfg_sed.sample_rate)))).float(), model=backbone, 
                                      use_logmel=True, config_sed = cfg_sed.__dict__)[2] # (batch size, channels, num_steps, y_axis) 
    print("Num timestamps features:",in_features)
    model = AudioSED(backbone, num_classes=[num_classe], in_features=in_feat, hidden_size=1024, activation= 'sigmoid', use_logmel=True, 
                spectrogram_augmentation = None, apply_attention="step", drop_rate = [0.5, 0.5], config_sed= cfg_sed.__dict__)

    model2 = AudioClassifierSequenceSED(model)
    
    return model, model2

cfg_sed =  ConfigSED(window='hann', center=True, pad_mode='reflect', windows_size=1024, hop_size=320,
                sample_rate=32000, mel_bins=128, fmin=50, fmax=16000, ref=1.0, amin=1e-10, top_db=None)

model_5, model = load_model(model_name="tf_efficientnet_b2_ns", num_classe=575, cfg_sed=cfg_sed)
inputs = torch.as_tensor(np.random.uniform(0,1, (20*32000)).reshape(1,4,-1)).float()
with torch.no_grad():
    o = model(inputs)
print(o[0]['clipwise'])

Tensorflow

import tensorflow as tf
from audio_sed.sed_config import ConfigSED
from audio_sed.tensorflow.sed_models import AudioClassifierSequenceSED as AudioClassifierSequenceSEDTF, AudioSED as AudioSEDTF, shape_from_backbone as shape_from_backboneTF



def load_modeltf(model_name, num_classe, cfg_sed):
    backbone =tf.keras.applications.efficientnet.EfficientNetB2(
    include_top=False)   
    if "efficientnet" in   model_name:
        in_feat = backbone.layers[-1].output.shape[-1] 
    elif "convnext" in cfg.model_name:
        in_feat = backbone.layers[-1].output.shape[-1]
    # batch size, num_steps, y_axis, channels
    in_features = shape_from_backboneTF(inputs=np.random.uniform(0, 1, (1, int(5 * cfg_sed.sample_rate))), model=backbone, use_logmel=True, config_sed = cfg_sed.__dict__)[1]
    print("Num timestamps features:",in_features)
    model = AudioSEDTF(backbone, num_classes=[num_classe], in_features=in_feat, hidden_size=1024, activation= 'sigmoid', use_logmel=True, 
                spectrogram_augmentation = None, apply_attention="step", drop_rate = [0.5, 0.5], config_sed= cfg_sed.__dict__)

    model = AudioClassifierSequenceSEDTF(model)
    
    return model

cfg_sed =  ConfigSED(window='hann', center=True, pad_mode='reflect', windows_size=1024, hop_size=320,
                sample_rate=32000, mel_bins=128, fmin=50, fmax=16000, ref=1.0, amin=1e-10, top_db=None)

inputs = np.random.uniform(0,1, (1, 4, 5*32000))
o_tf = model_tf.predict(inputs)
print(o_tf[0])

Examples 2:

You can find here an example of how this model is used with a GUI for inference with some checkpoint available.

Citation

Metadata

Release files for audio-sed 0.0.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for audio-sed 0.0.2
File Size Uploaded Provenance Actions
audio_sed-0.0.2.tar.gz 11.2 kB Without provenance Details for audio_sed-0.0.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for audio-sed 0.0.2
File Interpreter ABI Platform Actions
audio_sed-0.0.2-py3-none-any.whl Python 3 none any Details for audio_sed-0.0.2-py3-none-any.whl

Total release size: 23.7 kB

Release files / audio_sed-0.0.2.tar.gz

Download URL audio_sed-0.0.2.tar.gz
Size 11.2 kB
Tags Source
SHA-256 checksum
How to use checksums
98c43af628dcdf6add322ca41dc5eef22219c00b84adbf3988de78479d818874
BLAKE2b-256 checksum
How to use checksums
e8bb56ddcab0f9d1c3228146125c6ab7495fc411a24af65bf053e13fc24a4771
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.10.16

Release files / audio_sed-0.0.2-py3-none-any.whl

Download URL audio_sed-0.0.2-py3-none-any.whl
Size 12.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a77047d7661b0e037510475e130b7efef93279b96b74ca8621bad7da9aacf313
BLAKE2b-256 checksum
How to use checksums
2ecd57921bda604845cd3f4cf69d35a0c79d8236f6b20ccf7b29c3ffcd78be49
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.10.16

Security for audio-sed 0.0.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page