PocketStation for Python

Capture one desktop application and an optional microphone as separate live audio stems. Use the same native Session for Python model code, browser delivery, and one recording file per stem.

desktop application ─┐
system audio ────────┼─ native Session ─┬─ Python model code
microphone ──────────┤                  ├─ Relay and a browser
generated PCM ───────┘                  └─ separate recording stems

PocketStation runs capture, frame timing, routing, recording, and Relay publication in Rust. Python owns the application and provider code. A slow Python integration cannot run on the operating-system capture callback.

Capture a desktop application

You need Python 3.11 or newer, a supported desktop operating system, and one running application that is producing audio.

python -m pip install pocketstation

import pocketstation

with pocketstation.capture(application="Spotify") as live:
    for frame in live.audio:
        print(frame.source_id, frame.stem_id)

Replace Spotify with the display name or application identifier shown by the operating system. You may also pass a positive process ID. Selection must match one running application; PocketStation does not guess when several processes match.

The context manager starts one native Session and joins it when the block exits. No microphone opens and no file is written unless you request them.

Capture the complete output mix

Use a Session directly when the workflow intentionally needs every sound playing through the computer:

import pocketstation as pks

session = pks.Session()
system_audio = session.capture(pks.Source.system_audio())
system_audio.send(session.polled_audio())

with session.start() as running:
    for frame in running.audio:
        print(frame.source_id, frame.stem_id)

Use Source.application(...) instead when unrelated desktop audio must remain outside the Session.

Add a microphone or recording

import pocketstation

with pocketstation.capture(
    application="Zoom",
    microphone=True,
    record_to="recordings",
) as live:
    for frame in live.audio:
        print(frame.source_id, frame.stem_id)

The application and microphone keep different source and stem identities. Recording writes a separate stem for each source and reports its final result when the Session stops.

Use a microphone device ID instead of True when the application must select a specific input. See capture one application for source discovery, permissions, and asyncio.

Use each source independently

One Session can send a stem to more than one destination:

• iterate over frames in Python;
• transcribe or process audio with an Operator;
• publish audio with a Connector;
• send a named AudioBus through Relay;
• record the original source;
• add generated PCM without mixing it into captured audio.

Every destination receives source, stream, stem, sequence, timestamp, clock, and discontinuity information. Application and microphone audio do not have to be mixed before model processing or remote delivery.

The default Python audio queue holds 32 frames. If Python stops reading and the queue fills, new frames are dropped and the Session reports the queue depth, dropped-frame count, and discontinuity. Other destinations continue according to their own delivery settings.

Build a voice workflow

pocketstation.voice defines the interfaces for streaming transcription, response generation, speech synthesis, speech detection, and duplex voice models. Provider packages implement those interfaces; PocketStation does not embed a model catalog or API key.

Use separate providers when the application chooses each stage:

conversation = session.conversation(
    input=microphone,
    output=assistant,
    stt=transcriber,
    llm=response_model,
    tts=synthesizer,
    vad=speech_detector,
)

Use one duplex provider when it accepts audio and returns audio through one stateful connection:

conversation = session.conversation(
    input=microphone,
    output=assistant,
    voice_model=voice_model,
)

The two forms cannot be combined. Provider capabilities are checked before the Session starts. Generated speech enters the Session through audio_input(), so it can be recorded, published, observed, or removed from pending local output without stopping microphone capture.

This API does not provide model intelligence or a hosted inference service. Optional portable processing and explicit device AEC have separate Session APIs in the development candidate; published 0.1.5 wheels do not include them. Provider-side cancellation and receiver playout remain separate observations; PocketStation does not report that a person stopped hearing audio unless the receiver can prove it.

Read compose a voice workflow for transcript revisions, provider capabilities, interruption, retained history, and current limits.

Debug a voice interruption

examples/debug_voice_ai.py shows the complete Session declaration. It connects a physical microphone to OpenAI Realtime, returns assistant PCM to PocketStation, publishes named buses through Relay, captures the browser application playing the assistant, and records the three sources separately.

python -m pip install 'pocketstation[voice-agent-debug]'
export OPENAI_API_KEY='...'
python examples/debug_voice_ai.py

Speak while the assistant is replying. The final report separates provider events from what PocketStation observed:

• microphone capture and continuity;
• transcript and response events reported by the provider;
• assistant PCM accepted by the Session;
• pending output removed after cancellation;
• Relay and browser WebRTC observations;
• the browser application's captured output;
• finalized recording stems.

The example does not prove the exact loudspeaker sample a person heard, AEC, or provider-history truncation. Those fields remain unavailable when the provider or receiver cannot report them. Read the example instructions before running it.

Transcribe both sides of a voice application

The transcription example sends application and microphone audio through one faster-whisper model while keeping the resulting text attributed to its source:

python -m pip install 'pocketstation[transcription]'
python examples/transcribe_voice_app.py

The program asks which running application to inspect. It opens the default microphone because that example explicitly requests both sides. It does not start Relay or record audio.

faster-whisper processes finite audio windows and emits final transcripts. This is batch transcription, not a claim of streaming interim text or voice-agent turn handling. The adapter is example code and imports faster_whisper.WhisperModel only when transcription starts.

Stream any application to a browser

python examples/stream_any_app_audio.py

Choose a running application. The example publishes it as one named AudioBus, waits for Relay readiness, and prints a single-use word code and browser URL. It does not open a microphone or record audio.

The example uses PocketStation's small, rate-limited demonstration services unless you set POCKETSTATION_CONTROL_URL to a control plane you operate. The control plane supplies the authoritative Relay endpoints, so application code does not configure them separately. The demonstration is not a hosted service or SLA and can return HTTP 429 when its configured capacity is in use.

Send audio to your own provider

A Connector sends Session audio to an API, socket, file, or provider. Pass one async function when the connection is already open:

import pocketstation as pks
import pocketstation.aio as pks_aio


async def send_audio(frame: pks.AudioFrame) -> None:
    await socket.send(frame.samples)


destination = pks_aio.Connector(send=send_audio)
application.send_to(destination)

Use a class when the Connector opens and closes provider resources:

class WebSocketConnector(pks_aio.Connector):
    def __init__(self, url: str, token: str) -> None:
        self.url = url
        self.token = token

    async def start(self) -> None:
        self.socket = await connect(self.url, token=self.token)

    async def send(self, frame: pks.AudioFrame) -> None:
        await self.socket.send(frame.samples)

    async def stop(self) -> None:
        await self.socket.close()

One Connector object represents one configured provider connection:

destination = WebSocketConnector(url, token)
application.send_to(destination)
microphone.send_to(destination)

PocketStation calls start() once, sends both source-aware stems, and calls stop() once. Each stem retains its own delivery queue and observations. Create another Connector object when credentials, failure handling, or shutdown must be independent.

The integration guide covers deadlines, failures, sync Connectors, Sources, Operators, Endpoints, native extensions, and managed processes. The manifest and driver APIs are for distributable integrations that need typed configuration or custom service status; they are not required for a normal Python Connector.

Write PCM into a Session

Use audio_input() for generated speech, call audio, or decoded network media:

session = pocketstation.Session(recording_root="recordings")
assistant = session.audio_input("assistant")
assistant.output.record("assistant")

with session.start():
    assistant.write(samples)

Core preallocates the input buffers. A write reports whether it was accepted, full, closed, cancelled, or invalid. Read application-owned audio before selecting a queue capacity or cancelling pending output.

Handle permissions and source changes

PocketStation does not prompt during import or discovery. Check microphone permission without prompting with pocketstation.sources.microphone_permission_observation(), then let source opening report the operating system's final result. Application and microphone capture use separate permissions.

PocketStation reports a missing or stalled source; it does not choose a replacement. For a microphone in a running Session, the host can select a device and explicitly replace it without changing the logical stem or routes:

from pocketstation import Source

device_id = input("Selected microphone device ID: ")
selected_microphone = Source.microphone_id(device_id)
replacement = running.replace_microphone_source(microphone, selected_microphone)
print(replacement.source_id, replacement.source_generation)

reopen_microphone_source(microphone, selected_microphone) instead closes the old capture before reacquiring; if reopening fails, that microphone remains detached while unrelated sources continue. Both calls also accept a deliberate Source.microphone_default() selector. A successful call means the new source attached, not that a frame has arrived: inspect running.metrics().source_activities and evaluate it with your own SourceActivityPolicy. After a timeout, inspect source_replacements because the runtime may complete the operation later. Application-source recovery still requires a new Session. Store a discovered source only for its reported persistence scope.

See platform operations for permission states, persistence, recovery, and native dependencies.

Sync and asyncio

pocketstation and pocketstation.aio control the same Rust Session. The asyncio API adds awaitable lifecycle, streams, Relay calls, provider callbacks, and audio writes; it does not create another capture or routing engine.

Python callbacks enter the interpreter and pay Python scheduling and conversion costs. Capture, routing, recording, and the shared Relay Connector remain native. Python-authored model and provider code does not have identical cost to Rust code.

Platform support

Version 0.1.5 uses PocketStation Core 1.1.11 and the shared Relay Connector 0.1.5. Installed runtime checks cover CPython 3.11–3.14 on the six native targets, except Windows ARM64, which is tested on 3.13–3.14. Device results in the table are separate retained measurements. Public availability and exact release receipts are recorded on PyPI and GitHub Releases.

Reading native audio into Python copies samples into Python-owned bytes before exposing a memoryview. The view avoids another Python-side copy; the Rust-to-Python call is not zero-copy.

Continue from the task you have

Develop the SDK

uv sync --extra transcription
uv run pytest -q
uv run ruff check python tests examples
uv run ruff format --check python tests examples
uv run mypy python tests/qualification/typing_contract.py examples

License

PocketStation SDK source is available under the MIT license. Native wheels include dependencies with their own terms; see third-party notices.

Read support, security, and contributing for reporting and development expectations.

Version 0.1.6 uses Core 1.1.13 for live audio history, recording clips and optional AEC APIs. See the recording guide and echo processing guide for the APIs and their support limits.