tsm-realtime 0.1.5

Real-time audio time-scale modification with look-up approximation and full computation methods

TSM Real-Time

A Python library for real-time audio time-scale modification using advanced DSP techniques including Phase Vocoder (PV) and Overlap-Add (OLA) methods.

Features

• Real-time audio processing with interactive controls
• Multiple time-stretching algorithms:
  • Phase Vocoder (PV) with baseline and lookup methods
  • Overlap-Add (OLA) processing
  • Hybrid approaches combining PV and OLA
• Harmonic/Percussive source separation using librosa
• Interactive keyboard controls for real-time tempo adjustment
• Modular design with reusable TSM functions
• Cross-platform support (Windows, macOS, Linux)

Installation

⚠️ Important: This package requires local machine installation and cannot be used in headless environments (e.g., Google Colab, Jupyter notebooks without display) due to keyboard control dependencies. Futhermore, if your computer prompts keyboard control, please allow it to in order to do the command-line module!

For full audio playback functionality, you must install the PortAudio library separately:

System-specific PortAudio installation:

• macOS: brew install portaudio
• Ubuntu/Debian: sudo apt-get install portaudio19-dev
• Windows: Usually included with PyAudio wheels

then,

# Install audio dependencies
pip install tsm-realtime

From Source

git clone https://github.com/HMC-MIR/TSMRealTime.git
cd TSMRealTime
# See system-specific PortAudio installations
pip install -e  

Quick Start

2. Install system audio libraries (see Installation section above)

Basic Usage

import tsm_realtime

# Create TSM processor instance
tsm = tsm_realtime.TSMRealTime()

# Play audio with real-time controls
tsm.play_hps_full("path/to/your/audio.wav")

Interactive Controls

When running the audio processing, use these keyboard controls:

• ↑ (Up Arrow): Increase time-stretch factor (alpha)
• ↓ (Down Arrow): Decrease time-stretch factor (alpha)
• Ctrl+C: Stop playback

Advanced Usage

# Use lookup-based method for better performance
tsm.play_hps_lookup("audio.wav", beta=0.25)

# The beta parameter controls the overlap factor for lookup analysis
# Lower values (0.1-0.3) provide better quality
# Higher values (0.4-0.8) provide faster performance

API Reference

TSMRealTime Class

Methods

• play_hps_full(filename): Play audio using hybrid baseline method
• play_hps_lookup(filename, beta=0.25): Play audio using hybrid lookup method
• generate_lookup(beta, xh): Generate lookup tables for efficient processing
• phase_vocoder_full(xh, Ha_PV, prev_phase): Complete phase vocoder analysis
• phase_vocoder_lookup(...): Phase vocoder using precomputed tables
• ola_process(xp, Ha_ola): Overlap-Add processing for percussive components

Parameters

• alpha: Time-stretch factor (1.0 = normal speed, >1.0 = faster, <1.0 = slower)
• beta: Overlap factor for lookup analysis (default: 0.25)
• sr: Sampling rate (default: 22050 Hz)

Requirements

System Requirements

• Python: 3.8 or higher
• Local machine: Cannot run in headless environments (Colab, remote servers without display)
• Audio system: Requires PortAudio for audio I/O (system-specific installation)

Python Dependencies

Core dependencies (included automatically):

• numpy>=1.20.0
• scipy>=1.7.0
• librosa>=0.9.0
• pydub>=0.25.0
• pynput>=1.8.1 (for keyboard controls)
• pyaudio>=0.2.11 (requires PortAudio system library)

Why Separate Audio Installation?

PortAudio cannot be bundled with the package due to:

• System-specific compilation: PortAudio must be compiled for each operating system
• Hardware dependencies: Audio drivers vary by machine
• Platform-specific libraries: Different audio backends (ALSA, CoreAudio, DirectSound)

Algorithm Details

Phase Vocoder (PV)

• Processes harmonic components of audio
• Maintains phase continuity for natural sound
• Supports both real-time and lookup-based processing

Overlap-Add (OLA)

• Handles percussive components efficiently
• Provides good quality for transient sounds
• Lower computational complexity

Hybrid Approach

• Combines PV for harmonics and OLA for percussives
• Achieves optimal balance of quality and performance
• Automatic source separation using median filtering

Performance Notes

• Lookup method: Faster processing, affects the audio quality
• Full method: Higher quality, more computational overhead
• Real-time performance: Optimized for interactive use
• Memory usage: Moderate, depends on audio length and beta parameter

Examples

Example 1: Basic Real-time Processing

import tsm_realtime

# Initialize processor
tsm = tsm_realtime.TSMRealTime()

# Play with real-time tempo control
tsm.play_hps_full("sample.wav")

Example 2: High-Performance Processing

# Use lookup method for better performance
tsm.play_hps_lookup("sample.wav", beta=0.2)

Example 3: Custom Processing Pipeline

import librosa
import tsm_realtime

# Load and preprocess audio
x, sr = librosa.load("sample.wav", mono=True, sr=22050)
tsm = tsm_realtime.TSMRealTime()

# Generate lookup tables for efficient processing
xh, xp = tsm._harmonic_percussive_separation(x, sr)
S_phase, S_mag, w_if, Ha_lookup = tsm.generate_lookup(0.25, xh)

# Process with custom parameters
# ... (advanced usage)

Troubleshooting

Common Issues

Import fails in headless environments (Colab, remote servers):

ImportError: this platform is not supported: failed to acquire X connection

Solution: This package requires a local machine with display capabilities. Cannot run in:

• Google Colab
• Remote servers without X11
• Docker containers without display forwarding

PyAudio installation fails:

# macOS
brew install portaudio
pip install tsm-realtime[audio]

# Ubuntu/Debian
sudo apt-get install portaudio19-dev
pip install tsm-realtime[audio]

# Windows
pip install tsm-realtime[audio]

Audio playback issues:

• Ensure audio drivers are properly installed
• Check that the audio file format is supported
• Verify PortAudio is correctly installed
• Try different audio backends if available

Performance issues:

• Use lookup method (play_hps_lookup) for better performance
• Reduce beta parameter for faster processing
• Ensure sufficient system resources

Contributing

We welcome contributions! Please see our GitHub repository for:

• Issue reporting
• Feature requests
• Pull requests
• Development guidelines

Citation

If you use this library in your research, please cite:

@software{tsm_realtime,
  title={TSM Real-Time: Real-time Audio Time-Scale Modification},
  author={Lubis, Sayema and Peng, Clark and Carreno, Jared},
  year={2025},
  url={https://github.com/HMC-MIR/TSMRealTime}
}

License

This project is licensed under the MIT License - see the LICENSE.md file for details.

Authors

• Sayema Lubis - Development and Initial work
• Clark Peng - Development
• Jared Carreno - Initial work
• TJ Tsai - Project Advisor

Acknowledgments

• Built on top of excellent libraries: NumPy, SciPy, librosa, and PyAudio
• Inspired by classic DSP research in time-scale modification
• Developed at Harvey Mudd College (HMC)

---

For more information, visit our GitHub repository or PyPI page.