keywordtensor 1.1.0

A Python library for training custom keyword spotting models and running real-time voice command detection.

⚡ About KeywordTensor

KeywordTensor is built for developers who want to integrate voice commands into their Python projects without requiring deep knowledge of audio processing.

• Download public datasets so you can choose your own words, and let KeywordTensor automatically distil them into a tiny personalized model for your device.
• Bring your own .wav files: Just put your audio files into folders (e.g., dataset/hello/, dataset/stop/).
• Trigger custom Python actions: Easily map recognized words directly to your own Python functions. No Speech-to-Text required—KeywordTensor detects predefined commands and directly triggers Python callbacks.
• Automated Export & Config: Training automatically generates your optimized model and its configuration file. This allows you to launch live inference with a single command later. No manual saving required!
• Built-in Audio Augmentation: We automatically mutate your .wav files during training (PitchShift, Gain & Polarity Inversion, Colored Noise) to improve robustness in noisy environments.
• SpecAugment Pipeline: Raw audio is converted to Mel-spectrograms with Time and Frequency Masking applied. The model learns to recognize commands even if the microphone crackles or the audio drops out.
• Continuous Listening: A rolling buffer averages predictions over time to prevent sudden false positive clicks.
• Full Control: We hide the complexity by default, but give you full access to all deep learning training and listening parameters (training configuration, validation settings, and inference thresholds).

📦 Pre-trained Models

Don't have time to record your own dataset? You can use our ready-to-go models.

• prawda_falsz

  A highly robust model trained specifically to handle high-pitched children's voices and extremely noisy environments. This model was successfully deployed in a live public demonstration during the "Noc Naukowców" (Researchers' Night) event.

• More models coming soon!

💻 Quick Start & API

1. Installation (Choose your variant)

The library is available in two variants on PyPI depending on your needs:

• pip install keywordtensor Installs the full training environment. Use this on your PC or Server to train your models.

• pip install keywordtensor-edge A lightweight runtime variant. It strips out heavy training dependencies (like fastai), providing only what is needed for real-time inference (listen()). Perfect for Raspberry Pi or IoT devices.

2. Creating your own dataset

If you don't want to use public datasets, you can easily record your own voice to build a custom dataset using the built-in .record() tool.

import keywordtensor as kt

model = kt.Engine()

# Record 50 samples of the word "hello" and "stop"
model.record(
    target="my_dataset", 
    classes=["hello", "stop"], 
    samples=50,
    duration=3.0
)

Record parameters: Available parameters in .record():

• target (required): Path where the audio folders will be saved.
• classes (required): List of strings. Words you want to record.
• samples (default: 100): Number of audio samples to record per class.
• actions (default: None): Optional dictionary mapping words to custom visual/audio cues during recording.
• source (default: "microphone"): Audio input source. You can use the default microphone or pass a custom generator.
• duration (default: 3.0): The exact duration of each audio clip in seconds.

3. Training your model

The .train() method takes your audio files and trains a neural network using PyTorch and FastAI under the hood.

import keywordtensor

model = keywordtensor.Engine()

# The engine automatically applies audio & spectrogram augmentations during training
model.train(
    dataset="google",
    classes=["up", "down", "mixed:other"],
    model_path="my_custom_model",
    epochs=10,
    batch_size=32
)

Training parameters: You have total control over the pipeline. Available parameters in .train():

• dataset (required): Path to your audio dataset. You can provide a local folder path, or use one of the built-in presets: "google", "mswc", or "hf:username/repo".
• classes (default: None): List of specific words (folders) you want to recognize. If None, trains on all available folders. Pro-tip: Add "mixed:other" to the list, and the engine will automatically aggregate random words from your dataset to create a robust background noise class!
• model_path (default: 'myownmodel'): Name of the final exported model.
• epochs (default: 30): Number of training cycles over your dataset.
• batch_size (default: 32): Number of audio samples processed simultaneously.
• learning_rate (Automatic): The engine dynamically searches for the optimal learning rate for your specific dataset and automatically applies the One-Cycle Policy.
• wd (default: 0.01): Weight decay (L2 penalty) to prevent overfitting.
• eps (default: 0.01): Label smoothing epsilon to improve generalization.
• valid_pct (default: 0.1): Percentage of data reserved for validation.
• duration (default: 3.0): The exact duration of your audio clips in seconds. If an audio clip is shorter, it will be automatically padded with zeros (silence). If it is longer, it will be accurately truncated to match this length.
• sr (default: 16000): Sample rate of your audio files.

4. Live Inference & Custom Actions

Once trained (or using a pre-trained model like prawda_falsz), you can run real-time inference using your microphone.

import keywordtensor as kt

model = kt.Engine()

# Define your custom actions
def on_hello():
    print("Action triggered: 'Hello' detected!")

def on_stop():
    print("Action triggered: Stopping the robot!")

# Map keywords to your Python functions with per-keyword cooldowns
model.listen(
    model_path="my_custom_model",
    actions={
        "hello": {"function": on_hello, "cooldown": 2.0},
        "stop": {"function": on_stop, "cooldown": 3.0}
    },
    min_confidence=0.6,
    n_averages=3,
    source="microphone"
)

Listen parameters: The .listen() method itself accepts the following runtime arguments:

• model_path (required): The name of the model to load. You can provide the path to your own trained model, or use the built-in "prawda_falsz" model which is highly robust to noise and pitched voices.
• actions (default: None): Optional dictionary mapping detected keywords to Python callbacks. You can pass just a function (defaults to 0.0s cooldown), or a dictionary for precise control: {"function": your_function, "cooldown": 2.0}. Cooldowns are tracked individually per keyword!
• min_confidence (default: 0.6): The probability threshold (0.0 to 1.0) required to trigger the action.
• n_averages (default: 3): Temporal smoothing. Averages the last N predictions to prevent false positive clicks.
• source (default: "microphone"): Audio input source.
  • "microphone" uses the default system microphone.
  • "microphone:1" uses a specific microphone ID.
  • my_variable You can also pass your own Python variable (a generator) that yields current microphone audio data (arrays of Float32 at 16000Hz) to easily plug KeywordTensor into your own custom apps (like FastAPI or Gradio)!
• listen_time (default: 0): How long to listen in seconds. 0 means listen forever.
• threads (default: None): Number of CPU threads to use for ONNX inference.

Config file parameters: The rest of the underlying parameters are loaded automatically from the <model_path>_config.json file! When you run .train(), this file is automatically generated for you. It looks like this:

{
    "labels": ["hello", "stop"],
    "mean": -40.15,
    "std": 17.35,
    "duration": 3.0,
    "sr": 16000
}

This file dictates the rules for the inference engine:

• labels: The list of keywords the model was trained on.
• duration: The size of the rolling audio buffer in seconds.
• sr: The microphone sample rate.
• mean / std: Normalization statistics for the Mel-spectrogram.

💡 Total Flexibility: Want to adjust the microphone sample rate or buffer duration without retraining? Just open the JSON file and edit it!

Bringing your own model? No problem! If you trained an ONNX model entirely outside of KeywordTensor, simply drop it into your folder, create a matching your_model_config.json file next to it with the parameters above, and the .listen() method will load and run your external model.