PocketStation for Python

PocketStation captures one desktop application and an optional microphone as separate live audio stems. A single native Session can send those stems to Python model code, a remote browser, and a multistem recording without mixing their source identities.

The Python package uses the PocketStation Rust engine for capture, routing, timing, recording, and Relay transport. Your Python code owns the model and application logic.

Capture a desktop application

Install PocketStation:

python -m pip install pocketstation

Capture one application without opening a microphone or writing files:

import pocketstation

with pocketstation.capture(application="Spotify") as live:
    for frame in live.audio:
        print(frame.source_id, frame.stem_id)

Add microphone=True when you need the default microphone as a second independent stem. Add record_to="recordings" when you want each selected stem recorded. Both behaviors are off by default.

Handle permissions and source changes

PocketStation does not prompt during import or discovery. Check microphone permission without prompting through pocketstation.sources.microphone_permission_observation(), then let Source opening report the authoritative result. Application capture and microphone capture use separate permissions.

When an application or device disappears, PocketStation reports the change and does not switch to another Source. Stop the current Session, discover again, confirm any changed selection, and create a new Session. Store a discovered identity only for its reported persistence scope. Keep fallback and provider retry policy explicit and finite.

See Prepare and qualify each Python platform for the permission states, persistence scopes, and recovery sequence.

Debug a voice interruption

examples/debug_voice_ai.py sends a physical microphone to OpenAI Realtime without another voice framework. PocketStation keeps microphone input, generated assistant audio, and the selected browser's output as independent recorded stems. Provider events and media events share one monotonic timeline, so you can see whether delay occurred before the model, inside the provider, in local output, or after Relay delivery.

See the voice-agent debugger instructions. Run repository examples from a source checkout or source archive. The installed pocketstation-demo command is the packaged application-and-microphone demo.

python -m pip install 'pocketstation[transcription]'
pocketstation-demo

Transcribe both sides of a voice application

Install the transcription extra, then run the example:

python -m pip install 'pocketstation[transcription]'
python examples/transcribe_voice_app.py

The program asks which desktop voice application to inspect. It sends the application and microphone through one faster-whisper model, then labels each transcript with the source that produced it. It does not start Relay or write a recording.

The Session preserves each source while the transcriber processes both:

voice application ── faster-whisper ── transcript labeled "application"
physical microphone ─ faster-whisper ─ transcript labeled "microphone"

The complete composition is visible in examples/transcribe_voice_app.py. The example adapter imports faster_whisper.WhisperModel when transcription starts; the provider is not part of the pocketstation namespace.

This example does not debug turn handling, interruption, agent latency, or browser playout. PocketStation does not receive those events in this program.

Stream any application audio to a browser

Run the Relay example when you want another person to listen in a browser:

python examples/stream_any_app_audio.py

Choose any running application that is producing audio. The example publishes that application as one named AudioBus, waits for Relay readiness, and prints a single-use word code and browser URL. It does not open the microphone or record audio.

The example uses PocketStation's small, rate-limited demo service unless you set POCKETSTATION_CONTROL_URL and POCKETSTATION_RELAY_URL to services you operate. The shared URLs live in pocketstation_demo; application code does not contain service credentials.

Read application and microphone audio

Set the optional microphone and recording parameters when the workflow needs both sides:

import pocketstation

with pocketstation.capture(
    application="Zoom",
    microphone=True,
    record_to="recordings",
) as live:
    for frame in live.audio:
        print(frame.source_id, frame.stem_id)

The iterator receives audio through a native queue that holds 32 frames by default. If Python stops reading and the queue fills, PocketStation drops new frames and reports the queue depth, dropped-frame count, and discontinuity.

Send application-owned audio into a Session

Use audio_input() when your application already owns PCM, such as generated speech or audio received from a call provider:

session = pocketstation.Session(recording_root="recordings")
agent = session.audio_input("agent-output")
agent.output.record("agent")

with session.start():
    agent.write(samples)

The input uses finite preallocated Core buffers. Writes report full, closed, cancelled, and invalid-buffer outcomes explicitly.

Create an integration

Create a Connector when Session audio needs to reach an API, socket, file, or provider. Most Python integrations need only a send function or a small class; PocketStation supplies the worker, queue, delivery observations, and shutdown.

Pass one function when the destination is already open:

import pocketstation as pks
import pocketstation.aio as pks_aio


async def send_audio(frame: pks.AudioFrame) -> None:
    await socket.send(frame.samples)


destination = pks_aio.Connector(send=send_audio)
application.send_to(destination)

Subclass the synchronous or asyncio Connector when the provider opens and closes resources. The provider class owns its connection; the Session owns route delivery, lineage, observations, drain, abort, and joined shutdown:

class WebSocketConnector(pks_aio.Connector):
    def __init__(self, url, token):
        self.url, self.token = url, token

    async def start(self):
        self.socket = await connect(self.url, token=self.token)

    async def send(self, frame: pks.AudioFrame):
        await self.socket.send(frame.samples)

    async def stop(self):
        await self.socket.close()

Attach one configured object to one or more stems:

destination = WebSocketConnector(url, token)
application.send_to(destination)
microphone.send_to(destination)

PocketStation calls start() once, interleaves both source-aware stems through send(), and calls stop() once. A second Connector object creates a separate destination. See Create an integration for deadlines, failures, and the advanced SPI.

Python provider callbacks execute on off-realtime workers. They cannot be used as native capture callbacks. Use a compiled native extension for native provider code, or a managed process when crash isolation is required.

Use a Source when media enters the Session, an Operator when work transforms media or emits typed signals, and an Endpoint when an integration needs direct control of outbound delivery. The integration guide starts with the normal Connector API and introduces those advanced APIs only when the task requires them.

Use Relay from Python

Python creates and deletes RelaySessions through the typed HTTP control client. The shared Rust pocketstation-relay connector publishes media. The Go Relay service forwards WebRTC audio. Python does not encode Opus, write RTP, or own a second media plane.

The control client limits request duration and response size, redacts secrets, and provides matching synchronous and asyncio APIs.

Sync and asyncio

pocketstation and pocketstation.aio operate the same native Session. The asyncio namespace provides awaitable lifecycle, stream, Relay, provider, and audio-input operations without creating another audio queue.

Python callbacks still enter the interpreter. Capture, routing, recording, and Relay transport remain native-speed; arbitrary Python model code does not have the same execution cost as Rust.

Platform support

The native binding uses PocketStation Core 1.1.7 and the shared Relay Connector 0.1.5.

The Rust-to-Python audio read currently copies native samples into Python-owned bytes before exposing a memoryview. The view avoids another Python-side copy; the call into Python still copies samples and is not zero-copy.

Develop the SDK

uv sync --extra transcription
uv run pytest -q
uv run ruff check python tests examples
uv run ruff format --check python tests examples
uv run mypy python tests/qualification/typing_contract.py examples

Reference

• RELEASE_NOTES.md — user-visible changes and upgrade guidance.
• docs/README.md — task guides, concepts, operations, and API ownership.
• Write application-owned audio — PCM input with explicit queue capacity and selective output cancellation.
• Process audio and typed signals — Operators, named ports, generated audio, and finite model work.
• Record and observe a Session — multistem outcomes, route metrics, and lifecycle events.
• Keep each source identifiable — selection, persistence, timestamps, generations, and discontinuities.
• Read events, metrics, outcomes, and errors — setup failure, live observations, and terminal results.
• Prepare each platform — permissions, source persistence, explicit rediscovery, and fallback policy.
• examples/README.md — runnable examples and prerequisites.
• pocketstation.capture — concise application and microphone capture.
• pocketstation.session — Session declarations and lifecycle.
• pocketstation.graph — stems, ports, routes, and signal specifications.
• pocketstation.connector — outbound provider authoring.
• pocketstation.operator_authoring — computation authoring.
• pocketstation.source_authoring — inbound provider authoring.
• pocketstation.aio — asyncio APIs for the same engine.