livekit-plugins-noise-cancellation 0.3.0

Livekit plugin for noise cancellation of inbound AudioStream

Enhanced Noise Cancellation Plugin for LiveKit

Add realtime enhanced noise cancellation to inbound AudioStream. Fully compatible with LiveKit Agents.

Requires LiveKit Cloud.

Read more in the documentation

Audio chain

This package runs server-side on inbound audio (e.g. in an agent or on an AudioStream). Remote participants’ tracks are cleaned before your code or the agent consumes the audio.

flowchart LR
  Room["LiveKit room"] --> Track["Remote track (inbound)"]
  Track --> NC["livekit.plugins.noise_cancellation"]
  NC --> Stream["AudioStream / Agent input"]
  Stream --> Consumer["Your code or agent"]

Usage

In LiveKit Agents

Include the filter in RoomInputOptions when starting your AgentSession:

from livekit.plugins import noise_cancellation

# ...
await session.start(
    # ...,
    room_input_options=room_io.RoomInputOptions(
        noise_cancellation=noise_cancellation.BVC(),
    ),
)
# ...

On AudioStream

Noise cancellation can also be applied to any individual inbound AudioStream:

from livekit.rtc import AudioStream
from livekit.plugins import noise_cancellation

stream = AudioStream.from_track(
    track=track,
    noise_cancellation=noise_cancellation.BVC(),
)

Available Models

There are three noise cancellation models available:

# Standard enhanced noise cancellation (NC)
noise_cancellation.NC()

# Background voice cancellation (NC + removes non-primary voices
# that would confuse transcription or turn detection)
noise_cancellation.BVC()

# Background voice cancellation optimized for telephony applications
noise_cancellation.BVCTelephony()

Notes

Noise cancellation only needs to be applied once, so if you choose to apply it here you should disable noise cancellation / Krisp filter in your frontend clients.

If you experience crashes when using noise_cancellation (especially on AMD CPUs), it may be due to a failure in OpenBLAS's CPU detection. Manually setting the OPENBLAS_CORETYPE environment variable to a more conservative value (e.g., Haswell) may resolve the issue.

License

See: https://livekit.io/legal/terms-of-service