Skip to main content

pipecat-behavioralsignals

Behavioral Signal Technologies

Discord readme.io CI Ruff uv PyPI Python Pipecat License Contributor Covenant

Lets a Pipecat voice agent hear how the user speaks, not just what they say. It streams the user's voice to the Behavioral Signals API and adds what it reads, such as emotion, energy and hesitation, to each user turn. The LLM reads it with the user's words.

It uses Pipecat's Real-Time Context Standard (RCS): it is a perception producer. Pipecat ships its own too: PipecatToneReader reads the words, and PipecatVoiceReader reads the voice with a Pipecat model. This one reads the voice with the Behavioral Signals API, and can run with them.

Maintained by Behavioral Signal Technologies, the company behind the Behavioral Signals API.

Installation

uv add pipecat-behavioralsignals

Or with pip:

pip install pipecat-behavioralsignals

Requires Python 3.12 or later and Pipecat 1.16.

Prerequisites

A Behavioral Signals account. Create one and an API key in the Behavioral Signals portal. Set your client ID (CID) and API key:

export BEHAVIORALSIGNALS_CID=...
export BEHAVIORALSIGNALS_API_KEY=...

You can also pass them to the service as cid and api_key.

The service streams all of the user's input audio for the whole call, silence included, and streaming is billed by the minute. See the portal for prices.

Usage

Put BehavioralSignalsPerceptionService right after your STT service, and turn on expressive conversation:

from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineParams, PipelineWorker

from pipecat_behavioralsignals import BehavioralSignalsPerceptionService

perception = BehavioralSignalsPerceptionService()

pipeline = Pipeline([
    transport.input(),
    stt,
    perception,
    user_aggregator,
    llm,
    tts,
    transport.output(),
    assistant_aggregator,
])

worker = PipelineWorker(pipeline, params=PipelineParams(enable_expressive_conversation=True))

Things to know:

  • It reads the audio that the STT service passes on. STT services do that by default; do not set audio_passthrough=False.
  • It expects 16-bit mono audio, which is what Pipecat transports give by default. On audio with more channels it reports an error and stops reading.
  • It adds its reading to the one a transcription already carries, such as from a PipecatToneReader placed before it, and declares that producer's signals again with its own.
  • Without enable_expressive_conversation=True, the LLM does not see the readings.

What the LLM sees

At start, the service declares its signals in a PerceptionMetadataFrame. Pipecat uses it to tell the LLM how to read them. Then each user message starts with the reading, like this:

([voice_emotion angry=0.71, neutral=0.20, sad=0.06, happy=0.03],[positivity negative=0.72, neutral=0.20, positive=0.08],[strength strong=0.64, neutral=0.30, weak=0.06],[speaking_rate fast=0.50, normal=0.40, slow=0.10],[hesitation no=0.80, yes=0.20],[engagement engaged=0.55, neutral=0.35, withdrawn=0.10],[intensity value=0.61]) I already told you, I was charged twice.

The signals have the names of the Behavioral Signals tasks, except voice_emotion (the emotion task), which is kept apart from Pipecat Tone's emotion, read from the words:

Signal What it reads Labels
voice_emotion The emotion in the voice happy, angry, sad, neutral
strength Arousal: the energy in the voice strong, weak, neutral
positivity Valence: how positive the tone of voice is positive, negative, neutral
speaking_rate How fast the user speaks, compared to speakers in general fast, slow, normal
hesitation Whether the user sounds hesitant yes, no
engagement Whether the user sounds involved or detached engaged, withdrawn, neutral
intensity How intense the emotion is, whichever emotion it is a value from 0 to 1

Each label has its probability, likeliest first. The API's speaker signals (age and gender) are not sent.

How it works

Behavioral Signals reads the voice in short windows of about 2 seconds, and each reading arrives about 1 to 2.5 seconds after its audio. On each final transcription, the service takes the latest reading of each signal that overlaps the transcription's audio: the audio since the previous final transcription. So a reading usually covers the start or middle of a turn rather than its last words, and a very short turn may have no reading. Then the message has no reading at the start.

Your users' audio

The service sends the user's audio to the Behavioral Signals API. The SDK README says a stream's results are saved there and can be fetched later. Tell your users, and check that this fits your privacy terms.

Running the example

examples/bot.py is a voice agent you talk to in the browser. It uses Deepgram for speech to text and text to speech, and OpenAI for the LLM.

git clone https://github.com/BehavioralSignalTechnologies/pipecat-behavioralsignals.git
cd pipecat-behavioralsignals
uv sync
cp examples/env.example .env   # then fill in the keys
uv run examples/bot.py

Open http://localhost:7860/client and talk. The bot logs each turn's reading, as Voice: ....

Compatibility

Tested with Pipecat v1.16.0 and behavioralsignals 1.0.0. RCS is an alpha in Pipecat, so its types may change between releases.

Development

uv sync
uv run pytest
uv run ruff check
uv run ruff format --check

The tests need no credentials: the Behavioral Signals client is faked. See CONTRIBUTING.md for more, and SECURITY.md to report a security problem.

License

BSD 2-Clause. See LICENSE.

Metadata

Release files for pipecat-behavioralsignals 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pipecat-behavioralsignals 0.2.0
File Size Uploaded Provenance Actions
pipecat_behavioralsignals-0.2.0.tar.gz 11.1 kB With provenance Details for pipecat_behavioralsignals-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for pipecat-behavioralsignals 0.2.0
File Interpreter ABI Platform Actions
pipecat_behavioralsignals-0.2.0-py3-none-any.whl Python 3 none any Details for pipecat_behavioralsignals-0.2.0-py3-none-any.whl

Total release size: 24.1 kB

Release files / pipecat_behavioralsignals-0.2.0.tar.gz

Download URL pipecat_behavioralsignals-0.2.0.tar.gz
Size 11.1 kB
Tags Source
SHA-256 checksum
How to use checksums
c2a5d9c750d32e171e29bc88757a6bc9f775cc4861c9e94feb81d5db5a6205fb
BLAKE2b-256 checksum
How to use checksums
f8690b591c078a1efba2cce5063786c9b38a9f24990e26de426278f24680a896
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.

Transparency log

Release files / pipecat_behavioralsignals-0.2.0-py3-none-any.whl

Download URL pipecat_behavioralsignals-0.2.0-py3-none-any.whl
Size 12.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
02343b645e1513cc2fbc8b7d42ec47a30f9ca9986fd546640369dded06857b30
BLAKE2b-256 checksum
How to use checksums
0ef94d6198b3f5357cec59208bbd88303de951d732e470960ce9708c85235790
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.

Transparency log

Security for pipecat-behavioralsignals 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for pipecat_behavioralsignals-0.2.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 12, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page