clustaudio

clustaudio is a Python library to detect natural groups or clusters of audio files. Multiple steps are pipelined where audio files are processed, acoustic features extracted, and the clusters evaluated across the feature space. The optimal number of clusters is determined using methods such as *silhouette, dbindex, and derivatives* in combination with clustering methods, such as *agglomerative, kmeans, dbscan and hdbscan*. clustaudio allows you to determine the most robust clustering by efficiently searching across the parameters and by evaluating the clusters. Besides clustering of audio, the ``clustaudio`` library can also find the most similar audio files for a new, unseen sample, denoise noisy recordings, separate foreground from background, and extract voice segments. ⭐️Star it if you like it⭐️

clustaudio overcomes the following challenges:

* 1. Robustly groups similar audio files using acoustic features.
* 2. Returns the unique representative audio samples per cluster.
* 3. Finds highly similar audio files for a given input audio.
* 4. Denoises audio with multiple methods (spectral, wiener, noisereduce).
* 5. Separates foreground (voice/signal) from background (noise).
* 6. Extracts voice segments using Voice Activity Detection (VAD).
* 7. Clusters audio by metadata (artist, album, genre) when available.

clustaudio is fun because:

* It does not require a learning process.
* It can group any set of audio files (music, speech, environmental sounds, ATC recordings).
* It can return only the unique() audio samples per cluster.
* It can find highly similar audio files given an input audio file.
* It provides advanced audio processing: denoising, separation, VAD.
* It handles noisy communications (ATC, radio) with specialized enhancement.
* It generates playlists and organizes files by cluster.
* It provides many plots (waveforms, spectrograms, scatter plots) to explore the feature-space.
* It is built on core audio processing (MFCC, spectral features, chroma) and robust clustering methods.
* It works out of the box.

Documentation pages

Installation

It is advisable to create a new environment

conda create -n env_clustaudio python=3.14
conda activate env_clustaudio

Install clustaudio from PyPI

pip install clustaudio            # new install
pip install -U clustaudio         # update to latest version

Directly install from GitHub source

pip install git+https://github.com/erdogant/clustaudio

Quick Start

from clustaudio import Clustaudio

# Initialize with MFCC features
cl = Clustaudio(method='mfcc', sr=22050)

paths = cl.import_example(data='various')
# Cluster audio files from a directory
results = cl.fit_transform(paths)

# Visualize clusters
cl.clusteval.scatter(s=250)

# Plot waveforms for each cluster
cl.plot(labels=[0, 1, 2], show_waveform=True)

# Find similar audio to a query
similar = cl.find(paths[0], k=10)

similar.keys()
# dict_keys(['feat', 'dog_barking_1.wav'])

# Organize files by cluster
cl.move_to_dir(create_playlists=True)

Core Features

The results obtained from the clustaudio library is a dictionary containing the following keys:

* audio         : Preprocessed audio arrays
* feat          : Acoustic features extracted (MFCC, spectral, chroma, etc.)
* xycoord       : X and Y coordinates from the embedding (t-SNE, UMAP)
* pathnames     : Absolute path location to the audio files
* filenames     : File names of the audio files
* labels        : Cluster labels
* sample_rates  : Sample rates of loaded audio
* durations     : Duration of each audio file

Feature Extraction Methods

clustaudio supports multiple acoustic feature extraction methods:

Advanced Features

Voice Activity Detection (VAD)

Extract voice segments from audio files:

# Extract voice segments from a podcast

from clustaudio import Clustaudio

# Initialize with MFCC features
cl = Clustaudio(method='mfcc', sr=22050)
paths = cl.import_example(data='atc')

segments = cl.extract_voice_segments(
    paths,
    method='webrtcvad',
    min_segment_duration=0.5
)

# Cluster the extracted voice segments
total_segments = [item for sublist in segments['pathnames_segments'] for item in sublist]
results = cl.fit_transform(total_segments)

cl.clusteval.scatter(results['xycoord'], s=200)

# Make D3Blocks chart
from d3blocks import D3Blocks
import pandas as pd
import numpy as np
from pathlib import Path

df = pd.DataFrame({
    "labels": results["labels"],
    "sample_rates": results["sample_rates"],
    "durations": results["durations"],
    "url": results["url"],
    "directory": results["directory"],
    "pathnames": results["pathnames"],
})


d3 = D3Blocks(chart="Scatter")
d3.set_node_properties(df)
d3.set_edge_properties(
    x=results['xycoord'][:, 0],
    y=results['xycoord'][:, 1],
    size=np.minimum(results['durations'], 25),
    tooltip=results['pathnames'],
    opacity=0.55,
    scale=True,
    title="Audio t-SNE",
    cmap="Set2",
)
d3.show()

Audio Denoising

Clean up noisy audio recordings:

from clustaudio import Clustaudio

# Initialize with MFCC features
cl = Clustaudio(method='mfcc', sr=22050)
paths = cl.import_example(data='atc')

# Denoise audio files
denoised = cl.denoise(
    pathnames=paths,
    method='spectral',
    save_denoised=True
)

# Cluster denoised audio
results = cl.fit_transform(denoised['pathnames_denoised'])
cl.clusteval.scatter(results['xycoord'], s=200)

Foreground/Background Separation

Separate signal from noise:

# Initialize with MFCC features
cl = Clustaudio(method='mfcc', sr=22050)
paths = cl.import_example(data='atc')

# Separate foreground (voice) from background (noise)
separated = cl.separate_foreground_background(paths,
    method='spectral_masking',
    denoise=True
)

# Cluster foreground audio
results = cl.fit_transform(separated['pathnames_foreground'])
cl.clusteval.scatter(results['xycoord'], s=200)

ATC Communication Enhancement

Specialized enhancement for Air Traffic Control recordings:

cl = Clustaudio(method='mfcc', sr=22050)
paths = cl.import_example(data='atc')

# Enhance noisy ATC communications
enhanced = cl.enhance_atc_audio(
    paths,
    method='aggressive'
)

# Process and cluster
results = cl.fit_transform(enhanced['pathnames_enhanced'])
cl.clusteval.scatter(results['xycoord'], s=200)

Example: Music Library Organization

from clustaudio import Clustaudio

# Initialize with combined features
cl = Clustaudio(method='mfcc-spectral', sr=22050)
paths = cl.import_example(data='various')

# Cluster your music library
results = cl.fit_transform(paths)

# Create playlists by cluster
playlists = cl.create_playlist(
    format='m3u8',
    savedir='/organized_music/'
)

# Visualize clusters
cl.clusteval.scatter(results['xycoord'], s=200)
cl.plot(labels=[0, 1, 2], show_waveform=True, show_spectrogram=True)

# Find similar songs
similar = cl.find(paths[0], k=20)
cl.plot_find()

Example: Podcast Speaker Identification

from clustaudio import Clustaudio

cl = Clustaudio()
paths = cl.import_example(data='atc')

# Extract voice segments
segments = cl.extract_voice_segments(
    paths,
    method='webrtcvad',
    min_segment_duration=1.0
)

# Cluster speakers
total_segments = [item for sublist in segments['pathnames_segments'] for item in sublist]
results = cl.fit_transform(total_segments)

# Each cluster represents a different speaker
cl.plot(labels=[0, 1], show_waveform=True)

Example: Environmental Sound Clustering

# Initialize for environmental sounds
cl = Clustaudio()
paths = cl.import_example(data='atc')

# Denoise and cluster
denoised = cl.denoise(
    paths,
    method='noisereduce',
    save_denoised=True
)

results = cl.fit_transform(denoised['pathnames_denoised'])

# Visualize sound clusters
cl.clusteval.scatter(results['xycoord'], s=200)
cl.plot_statistics()

Example: Duplicate Audio Detection

# Use fingerprinting for duplicate detection
cl = Clustaudio(method='fingerprint')
paths = cl.import_example(data='atc')


# Find duplicates
results = cl.fit_transform(paths)

# Clusters with multiple files are likely duplicates/covers
for label in np.unique(results['labels']):
    cluster_files = np.array(results['filenames'])[results['labels'] == label]
    if len(cluster_files) > 1:
        print(f"Potential duplicates: {cluster_files}")

Example: ATC Communication Analysis

Complete workflow for Air Traffic Control recordings:

from clustaudio import Clustaudio

# Initialize
cl = Clustaudio(method='mfcc', sr=16000)
paths = cl.import_example(data='atc')

# Step 1: Enhance noisy ATC audio
enhanced = cl.enhance_atc_audio(
    paths,
    method='aggressive'
)

# Step 2: Separate foreground (voice) from background (static)
separated = cl.separate_foreground_background(
    enhanced['pathnames_enhanced'],
    method='spectral_masking',
    denoise=True
)

# Step 3: Extract voice segments
segments = cl.extract_voice_segments(
    separated['pathnames_foreground'],
    method='energy',
    min_segment_duration=0.5
)

# Step 4: Cluster communications
total_segments = [item for sublist in segments['pathnames_segments'] for item in sublist]
results = cl.fit_transform(
    total_segments,
    cluster='agglomerative',
    evaluate='silhouette'
)

# Step 5: Visualize and analyze
cl.clusteval.scatter(results['xycoord'], s=200)
cl.plot(labels=[0, 1, 2], show_waveform=True)
cl.plot_statistics()

# Step 6: Organize by cluster
cl.move_to_dir(savedir='/organized_atc/', create_playlists=True)

Visualization Examples

Scatter Plot with Cluster Labels

# Create scatter plot
from scatterd import scatterd

cl.clusteval.plot()
cl.clusteval.plot_silhouette(results['xycoord'])
cl.clusteval.scatter(results['xycoord'], s=250)
cl.clusteval.dendrogram()
scatterd(results['xycoord'][:, 0], results['xycoord'][:, 1], density=False, labels=results['labels'])
#cl.pca.scatter3d(labels=results['labels'])

Waveform Visualization per Cluster

# Plot waveforms for specific clusters
cl.plot(labels=[0, 1, 2], show_waveform=True, max_per_cluster=4)

Spectrogram Visualization

# Plot spectrograms
cl.plot(labels=[0, 1], show_spectrogram=True)

Denoising Comparison

# Compare before/after denoising
cl.denoise(paths[0])
cl.plot_denoise(index=0)

Separation Results

# Visualize foreground/background separation
cl.separate_foreground_background(paths[0])
cl.plot_separation(index=0, figsize=(15, 20))

Cluster Statistics

# Comprehensive cluster statistics
cl.plot_statistics()

Supported Audio Formats

clustaudio supports all formats handled by librosa and soundfile:

• WAV - Uncompressed (fastest loading)
• MP3 - Compressed (most common)
• FLAC - Lossless compression
• OGG - Vorbis compressed
• M4A - AAC compressed
• AAC - Advanced Audio Coding

Documentation

Detailed documentation with many examples:

Example: Music Library Clustering

Example: Podcast Speaker Clustering

Example: ATC Communication Processing

Example: Environmental Sound Analysis

Example: Find Similar Audio

Example: Break Up the Steps

Feature Comparison

Citation

Please cite clustaudio if you use it in your research or project:

@software{clustaudio,
  author = {Taskesen, Erdogan},
  title = {clustaudio - Python library for clustering audio files},
  year = {2024},
  url = {https://github.com/erdogant/clustaudio},
}

Maintainer

• Erdogan Taskesen, GitHub: erdogant
• Contributions are welcome.
• Yes! This library is entirely free, but it runs on coffee! :) Feel free to support with a Coffee.