argus-mask

A talking face for the Argus suite. Audio goes in; a stream of facial animation frames comes out; a browser page renders a rigged head from those frames, in step with the audio, on any screen on the LAN.

Part of the Argus suite. Mask sits off the main line, like argus-darkroom and argus-atlas. The design and build plan are in spec.md.

audio source ──▶ argus-mask :8109 ──▶ viewer (browser, any LAN device)
(file, agent,     drive + sync         flat panel │ Looking Glass HLD
 TTS output)          │
                      ▼
               Audio2Face server (external, by URL)

Status

Built and tested: the engine, both drivers, faces, the server, the viewer's flat and hld tiers, the CLI, and the Audio2Face server in a2f/.

Not built yet: the Looking Glass Go tier (0.2), and faces built from argus-curator output (0.3).

Not yet seen by a person: the stock head speaking on a real screen with sound. Its mouth was tuned from still frames of real Audio2Face output.

Quick start

make install
make head        # downloads the stock head into assets/stock-01 (10 MB, CC0)
uv run argus-mask serve --assets-dir assets

Open http://127.0.0.1:8109/, click once to allow sound, then from another terminal:

uv run argus-mask push hello.wav --face stock-01

That uses the energy driver, which moves the mouth from loudness and nothing more. For real lip sync, run the Audio2Face server (an NVIDIA GPU with about 1 GB of VRAM to spare) and point mask at it:

docker build -t argus-mask-a2f a2f/      # once; see a2f/README.md for what it needs
docker run -d --name a2f --gpus all -p 127.0.0.1:8110:8110 -v a2f-data:/data argus-mask-a2f
ARGUS_MASK_DRIVER=a2f ARGUS_MASK_A2F_URL=http://127.0.0.1:8110 uv run argus-mask serve --assets-dir assets

Or both in containers: docker compose --profile a2f up --build, with DRIVER=a2f and A2F_URL=http://a2f:8110 in .env.

Commands

Faces

A face is a directory under the assets root holding a model and a manifest.json. The model is a .glb (a mesh) or a LAM Gaussian-splat head's .zip (see Splat heads).

• Model: glTF 2.0 binary, with morph targets named for the 52 ARKit blendshapes (jawOpen, eyeBlinkLeft, …). Missing shapes are reported and ignored. Textures must be PNG or JPEG; the viewer loads no KTX2 or meshopt decoder.
• Manifest: id, name, the model's file name, sha256 and format (gltf, the default, or lam-splat), the shapes found, camera framing, and provenance.
• Provenance is required. A face with no provenance is not served. kind is one of:
  • personal: your own use. Needs only a source saying what it is. This is inspect's default.
  • stock or synthetic: a published asset, or a generated face with no real person or owned character going in. Needs a source and a license.
  • consented: a real person's likeness, or an owned character, under a signed release. Also needs the subject as the release names them and the release's sha256 (never the document).
  • curated is refused until faces from curator are designed.
• A face is also refused when its manifest id differs from its directory name, or its model no longer matches the recorded sha256. inspect says why.

Personal and cleared

This is self-hosted software. What you put on your own screens at home is up to you: a cartoon character, a character from a film, a face from a character LoRA. Mask records what it is and serves it as personal. The floor is the law: anything illegal to make or hold is not a face here either, and nothing that argus-darkroom refuses (its illegal rules, in SFW and NSFW alike) becomes acceptable because it is a face.

Showing or sharing is different. A face is cleared (cleared: true in its provenance, and in /faces) when it is stock, synthetic or consented. Only cleared faces and voices go in this repo, its fixtures and examples, demos, recordings made to publish, or anything commercial. The editor marks a face that is not cleared as personal.

To make a synthetic source portrait (an SDXL image of a person who does not exist) for the planned photoreal face routes, see docs/synthetic-faces.md.

The stock head

The repo carries no model files. make head (or python scripts/fetch_stock_head.py) downloads stock-01, a generated adult head built from CC0 MakeHuman assets, and checks it against the size and sha256 in scripts/stock_heads/stock-01.lock.json before writing anything. python scripts/fetch_stock_head.py --id stock-02 fetches the second, a woman of about 25. Both wear the crew uniform and carry the wet line and baked skin maps (revision 4 of stock-01, the first of stock-02). To build it yourself with headless Blender and MPFB, or to make another head from a new preset, see scripts/README.md.

Your own head

mkdir -p assets/my-head && cp /path/to/head.glb assets/my-head/
uv run argus-mask inspect assets/my-head/head.glb --write-manifest \
    --kind stock --source "where it came from" --license "CC0-1.0" \
    --distance 0.9 --height -0.08

Splat heads

A splat head is a LAM (Large Avatar Model) avatar: a cloud of Gaussians made from one portrait, deformed by a FLAME head with the ARKit shapes. It looks like the photo it came from in a way no mesh head here does. Mask drives it like any face: the same frames, rest set, idle life and lip budget, on screens, in the editor's previews and in record.

• The file: LAM's avatar archive, a .zip holding one folder with skin.glb (the FLAME mesh and its morph targets: 51 of the 52 ARKit shapes, no tongueOut), offset.ply (the splats), animation.glb, vertex_order.json and optionally iris_occlusion.json. Nothing else: zip the folder alone (macOS adds __MACOSX/ and .DS_Store, which are refused).

• Checked, never unpacked to disk. The server reads the archive in memory and refuses it, saying why in inspect, for: more than one top-level folder, a member it does not know or one missing, absolute paths, .., backslashes, duplicate names, links, encryption, more than 8 members, a member over 64 MiB or all of them over 160 MiB unpacked, a member that unpacks to over 100 times its packed size, an offset.ply that is not a PLY, JSON that does not parse, and a skin.glb whose morph targets are not ARKit shapes.

• Making one: run LAM on a frontal portrait and export its "chatting avatar" (LAM's tools/AVATAR_EXPORT_GUIDE.md: the FBX SDK and Blender turn the result into skin.glb, animation.glb and vertex_order.json beside the offset.ply), and zip that one folder. Then:

  mkdir -p assets/c101-splat && cp c101.zip assets/c101-splat/
  uv run argus-mask inspect assets/c101-splat/c101.zip --write-manifest \
      --source "LAM head from a character-LoRA portrait; LAM weights CC BY-NC 4.0, FLAME non-commercial"
  

  The manifest gets format: lam-splat and the splat framing: LAM's heads stand at about 1.65 m, so the camera aims at --height 1.65 from --distance 0.7, a tenth of the distance above its aim, with a 50 degree --fov.

• Licences: personal, never cleared. LAM's code is Apache-2.0, but its weights are CC BY-NC 4.0 and FLAME's model is for non-commercial research. A head made with them is personal (whatever the portrait) until those licences change: it may be on your own screens, and in nothing published, shared or commercial. The renderer (Myned AI's gsplat-flame-avatar-renderer 1.4.0) and JSZip are MIT and vendored under viewer/vendor/ with their licences.

• On screens: display=flat only (hld says it is not supported yet). ?bg= works as for a mesh (not transparent); ?jaw= moves the jaw cap's knee (default 0.35: past it, jawOpen and the lower lip open at 0.35 of the rate, since the inside of a splat mouth is weak; ?jaw=1 turns it off). The head turns through the rig's neck and head bones; LAM's own animation clips never play, so mask is the only thing that moves the face and a screen shows what a recording does. It needs no cross-origin isolation: it renders on a plain-http LAN page (checked in headless Chromium, GPU and software; check a real screen before relying on it). window.__mask.renderer says splat or mesh.

• Recording: argus-mask record and the Broadcast page record a splat face as they do a mesh: each frame is posed, depth-sorted for the camera and drawn before its screenshot, so a frame rendered twice is byte-identical.

Face aliases

An alias is a stable name for a face: holly for a persona's face, used by the agent that speaks and by the kiosk that shows it, while which face stands behind it is chosen in the editor. Mithrandir names holly once; changing her face is a click on Saved faces, not a config change.

• Where: aliases.json in the assets directory, {"aliases": {"holly": "ginger-3"}}. The editor writes it (Saved faces → Aliases: add, repoint, remove, open on a screen); the server only reads it, so its assets mount may stay read-only, and it reads it again whenever it changes: no restart.
• Everywhere a face is named: /face/holly, /face/holly/model, /session/holly/view and /source, POST /push?face=holly, /speak. GET /aliases and /health's aliases list them; /faces lists faces only, as before.
• Sources and screens meet on the name they use. A source on holly reaches screens watching holly, and /health's live and screens list holly. A push to ginger-3 directly reaches screens watching ginger-3, not holly.
• Repointing: new connections get the new face at once. A screen already watching holly switches on the next thing said: each stream's header names the face itself, and a viewer that loaded another face reloads, with its URL, and joins the stream where it has got to.
• Rules: an alias is a face id that isn't a face's directory (it never hides a face), it points at a face the server serves, and never at another alias. An entry that breaks a rule is ignored (the editor shows why). Renaming a face in the editor carries its aliases along; deleting one an alias points at is refused until the alias points elsewhere.

A kiosk for a persona, with the sound on the house speakers (see House audio). <GPU_HOST> and <LAN_HOST> below are placeholders; the real host/IP is in the local .env (see .env.example), so check it there:

http://<GPU_HOST>:8109/?face=holly&display=hld&kiosk=1&audio=external&delay=750

Rest sets and idle life

A face at rest shows a rest set: a named baseline expression it holds between broadcasts and speaks from. With every weight at zero, most heads look stern (the mouth's corners sit low), so the default is warm.

• Built in: neutral, calm, warm (the default), bright, attentive. Each is a few ARKit weights and a life, how much the face moves at rest (0 still, 1 the default, up to 2).
• Choosing: Saved faces → open a face → the rest: picker beside the poses. An exported face keeps the choice; screens pick it up when they next load the face (the server serves it as GET /face/<id>/rest). A screen can override it with ?rest=calm, and ?life=0.5 calms the movement.
• Your own sets: expressions.json in the assets directory, written by the editor (PUT /expressions/sets/{name}) or by hand: {"sets": {"cheery": {"weights": {"mouthSmileLeft": 0.5, "mouthSmileRight": 0.5}, "life": 1.2}}, "faces": {"ginger-3": "cheery"}}. A set of the file's own may replace a built-in one. Bad entries are ignored, and GET /expressions on the editor says why. The server only reads it, like aliases.json.
• While speaking: the brows, eyes and cheeks of the set hold fully, its mouth weights at half, so a face speaks from its mood while the mouth stays the driver's.
• Idle life, while nothing plays: blinks 2 to 5 s apart (some doubled), glances with the head following part of the way, now and then a longer look away, breathing, and small expression shifts that come and go (a lifted brow, a passing smile, a thoughtful press). It is the page's own: never in a take, never in a recording.

Lips

Audio2Face's weights are tuned for its own character. On another head, the shapes that close the lips (mouthClose, mouthShrugLower, the rolls, pucker, press) can push the lower lip up through the upper one. The viewer measures, once per face as it loads, how far each mouth shape moves the gap between the lips (window.__mask.lips), and on every frame scales the closing shapes down so they never close the lips further than the opening ones (jawOpen, funnel, ...) leave room for, plus 1 mm for lips pressing together. mouthShrugLower and the rolls also go at 0.7. On ginger-3, over three takes it was not tuned on, lips overlapping by more than 1 mm fell from 24% of frames to 3%, by more than 3 mm from 12% to none; lips still meet for m, b and p. It needs no setting, works for any face with the ARKit mouth shapes, and applies to screens, record and the editor's previews alike (viewer/motion.js).

Viewer

GET / serves the viewer. URL parameters:

The look (viewer/look.js, shared with the face editor's previews) lights the head with a three-point rig (warm key, cool fill, cool rims that lift it off a dark background), image-based light from a neutral room, and PBR Neutral tone mapping. On the profiles:

It finds the parts by the names the head recipe gives them: head is skin (with the normal and roughness maps the recipe bakes from its colour map, and generated pores over them), eyes holds MakeHuman's eyeball and its cornea shell (drawn again as a reflection, for catch-lights that follow the gaze), wetline is the recipe's tear line along each lower lid, and hair is hair cards. A face without those names still gets the lighting. A transparent background turns the post-processing off, since it would lose the alpha. window.__mask.look says what is on.

Lip sync runs on the page's audio clock: audio plays buffer ms behind arrival and each frame is shown when the clock reaches its timestamp. Network jitter costs latency, not sync.

Browsers start audio only after a click or key press. A kiosk can waive that; for Chromium, start it with --autoplay-policy=no-user-gesture-required.

Until then the viewer says so: a full-screen Tap to start. One tap or key press turns the sound on and fills the screen, and a ● Ready pill shows briefly once the screen is connected and can be heard. kiosk=1 is for dedicated screens: no status line, no cursor, full screen on that first tap; started with the flag above, it needs no tap at all. The viewer tells the server its state on its socket (audio, display, kiosk, sound; never media), and /health lists it under screens, per face.

House audio (Snapcast)

For Mithrandir's face+house mode the house speakers play the speech and the panel only moves. Synthesise once, send the same audio to Snapcast and to mask (push), and open the panel's viewer with audio=external:

http://<mask>:8109/?face=<id>&display=hld&kiosk=1&audio=external&delay=750

• No double voice: the page makes no AudioContext at all, so nothing reaches its own speakers, and it needs no tap to be ready (kiosk=1 still goes full screen on a tap, if wanted).
• The clock: with no audio clock, the viewer keeps time with the page's own clock, anchored to the session's timeline exactly as the audio clock would be. Frames are shown at their timestamps, buffer + delay ms after arrival.
• Choosing the delay: Snapcast plays its buffer (1000 ms by default) behind what it is sent, and the viewer already plays buffer (250 ms) behind arrival, so start at the Snapcast buffer less 250 ms, then adjust by eye: raise it if the lips lead the sound, lower it if they trail. A driver's lag (Audio2Face holds the audio until its frames cover it) shifts both alike only if mask and Snapcast are fed at the same moment; if the lips trail by a constant amount, take it off the delay.
• Ready: the screen reports sound: "external", and /health counts it in screens with audio_ready: true once it is connected: ready to speak in sync, not "can be heard". The editor shows it as "1 screen ready (house audio)".

Serving the LAN

The server speaks plain HTTP and WebSocket, and binds localhost by default. To put the viewer on other machines:

1. Bind an address your TLS proxy can reach: argus-mask serve --host <address>.
2. Proxy to it with TLS. Caddy forwards WebSocket upgrades without extra configuration.
3. Name the proxy's origin in ARGUS_MASK_CORS_ORIGINS, for example https://mask.example.org.

Both sockets refuse a browser whose Origin is neither the server's own host nor in that list. Clients that send no Origin, such as the CLI, are allowed. There is no authentication: reaching the port is access.

Face editor

argus-mask editor is a page for making heads. Pick a skin, hair, clothes, a hat, brows, lashes, eyes, teeth and tongue from thumbnails (hover for a large view). Set sex, age, build and ancestry, and shape the head, forehead, brow ridge, eyes, ears, cheeks, nose, lips, chin and neck with 117 sliders in collapsible panels (the filter box finds one by name). Press Build and a preview appears about ten seconds later (the preview fades behind a spinner and a timer while Blender works); click it for a large view that can rest, open, smile, talk or turn. Under it, a Before card shows the build before this one and what changed between them; the large view shows the two side by side, posed together. Undo and Redo (Ctrl+Z, Ctrl+Shift+Z) step back and forth through your builds. Every build is kept in a cache (assets/.preview/.cache/, the last ARGUS_MASK_BUILD_CACHE builds, 40 by default), so going back to one, or putting a slider back where it was, copies it in a fraction of a second instead of running Blender. With Live update on, it builds by itself a second after you stop changing things; changes made during a build get one more build, and typing a name or id never builds.

A build is a preview, kept in assets/.preview/, which the server never lists. Export copies the preview on show into assets/<id>, and it is on the server's /faces at once. Download preset saves the form as a .json preset, the same format as scripts/stock_heads/, and Open preset… loads one.

Saved faces (/faces.html, linked from the editor) shows every face in the assets root and every preview as a card with a thumbnail. Review opens it in 3D with its provenance; Remix opens its preset in the editor under a new id, so the original is untouched; Rename changes its name or id (the export and the preview together); Delete moves it to assets/.trash/, which the server skips, so moving it back restores it. Stock and hand-added faces can be reviewed, not changed.

On screen… (beside Export, and on each exported face's card) takes a face to a screen. The editor checks that the server is up and lists the face, and says plainly if it does not (the usual cause: the server and the editor were started on different assets directories). It builds the viewer link at the screens' origin, with display, quality, kiosk, status line and background, to open, copy, or show as a QR code for a tablet or panel to scan. It shows which screens are watching and whether they can be heard ("1 screen ready", "1 screen, no sound yet: tap it"), plays a test chime through the server's own source socket (refused, like any source, by a read-only server), and gives the argus-mask say … --face <id> command to speak from a terminal. The editor reaches the server at ARGUS_MASK_SERVER_URL (--server-url, default http://127.0.0.1:8109) and hands out links at ARGUS_MASK_VIEWER_URL (--viewer-url: the TLS proxy on a LAN; default, the server's).

Broadcast (/broadcast.html, linked from both pages) is where a face speaks. It lists the faces the server serves and whether each is ready (a screen watching that can be heard), already speaking, or has no one to hear it. Pick one, and either Say text through the TTS service, with every option argus-mask say takes (voice, from the service's own list when it is a Wyoming server or a Miso-style HTTP service (its speakers and clones), or typed; pause between sentences; temperature, 0.1 to 1.5 or the service's own; seed; stream; the TTS URL; extra -p key=value fields), or Push a WAV file, as argus-mask push does. The editor runs the CLI's own code against the server, so the server's read-only switch and one-source-per-face rule apply as from a terminal; the page shows each step (the session, every sentence, the speech's length, done or why not) and can stop it. The matching command line is shown, to copy. The text goes to the TTS service and is never logged; the page shows its first 80 characters while it plays. Record to file records the broadcast as argus-mask record does: the recorder listens on the face first, then, once the broadcast ends, renders every frame in a headless browser (frame size 854×480, 1280×720, 1920×1080 or the HLD's 1080×1920; the viewer's high look by default) and muxes the MP4 with the audio, its WAV and take beside it, into ARGUS_MASK_RECORDINGS_DIR (--recordings-dir, default ./recordings). The page shows the clip to play or download, and lists recent ones. It needs the record extra and playwright install chromium-headless-shell where the editor runs; without them the checkbox says so.

Voices (/voices.html) is where a voice is made. Upload one to three reference clips with their transcripts, or generate a synthetic candidate (the TTS service's own speaker, or a described voice on a service that designs them, reading a script you choose), and each is checked as a reference: speech length, sample rate, bandwidth, clipping, level, background noise, edges, pauses, delivery, and whether the transcript fits the audio, each with what to change. Make the voice sends the chosen clips and transcript to the TTS service (/clone-voice, Miso's contract) and keeps a record in ARGUS_MASK_VOICES_DIR (--voices-dir, default ./voices): where the voice comes from (generated, your own, a consenting speaker's under a release, by its sha256, or personal: anyone else, for your own speakers, with a note of where the clips came from), its delivery, and the clips by hash, kept beside it. A clip with a bad check is not made into a voice. Try a voice speaks a line in the browser, no face involved. Voices made elsewhere show as having no record. See docs/voices.md: what a good reference sounds like, recording your own, synthetic voices, and one voice per delivery (warm, brisk, …).

It needs what scripts/README.md sets up, on the machine that runs it: Blender 4.2 or later, the MPFB extension, and the MakeHuman asset packs (makehuman_system_assets and faceunits01). None of it ships with argus-mask; the server never needs it.

Asset packs (/packs.html) browses the MakeHuman community's asset packs (hair, eyebrows, clothes and more) from a catalogue checked into the repo, asset_catalogue.json (refreshed by scripts/make_asset_catalogue.py, as a reviewed commit). Only CC0 and CC-BY packs are listed. A pack is installed by its catalogue id alone, one at a time: the zip comes from the catalogued URL, is refused unless its sha256 matches, is unpacked under MPFB's data directory and never overwrites an installed file. Its assets then show in the editor with their licences. Like every change, an install is refused when the editor is read-only.

argus-mask editor --assets-dir assets \
    --blender $WORK/blender/blender --blender-user $WORK/user    # BLENDER_USER_RESOURCES

• Only licensed assets. The editor offers the assets that an installed pack lists with a licence, and writes each part's licence and author into the face's provenance (kind: synthetic). An asset from a CC-BY pack shows up in the licence line.
• Adults only. Age defaults to about 25 (MakeHuman's 0.5); the slider stops at 18 (0.3542), and the preset model and the recipe refuse anything younger.
• It only replaces its own faces. Each face it builds keeps its preset.json beside the model; a directory without one (a stock head, a face added by hand) is never overwritten.
• Builds and exports go one at a time. A failed one leaves nothing behind, and the face that was there stays.
• Keep it local. A build starts Blender. The editor binds 127.0.0.1, and /build and /export are behind the same origin and read-only guards as the server's writes.

Configuration

With DRIVER=a2f and no server answering, mask starts, /health reports the driver down, and a source is refused with the reason. It never falls back to energy on its own.

Drivers

Neither driver blinks or moves the eyes, or moves the head. Mask reads the voice itself (loudness and pitch: stressed syllables, phrase ends) and its idle layer adds the rest: nods on stresses and brows on emphasis, a settle at each phrase end, blinks in the pauses, saccades and glances with the head following, small movements of brow and mouth, and a slow sway. All from the audio and a seed, so a take replays identically.

Speech from a TTS service

say asks a TTS service for the speech and plays it into the face at the speed of speech. The whole utterance is fetched first: Miso TTS 8B generates at about a quarter of real time on an RTX 3090, and streaming it live would stall the face mid-sentence. --stream forwards the audio as it is generated, for a service that is faster than real time. The face stays free for other sources while the speech is being generated, and Ctrl-C stops the wait at once. The text is sent to that service and nowhere else; mask's server only ever receives audio.

Two kinds of service, told apart by the URL:

• http(s)://…: an endpoint that takes {"text": …} and streams raw PCM, stating the format in X-Sample-Rate, X-Audio-Format (pcm_f32le or pcm_s16le) and X-Channels headers. Miso TTS's /synthesise/stream is one. Extra request fields go in with -p key=value.
• wyoming://host:port: a Wyoming TTS server, the protocol Home Assistant's voice pipeline uses; agent-composer's XTTS and Miso bridges speak it. -p language=… and -p speaker=… pass through. Audio must be 16-bit.

--voice (default ARGUS_MASK_TTS_VOICE) picks the voice; docs/voices.md covers making one. A name is a stored clone (voice_clone_id on the HTTP contract, the voice's name on Wyoming, where agent-composer's clone: prefix is also understood); a number is one of the service's own speakers, and so is a name written speaker:<name> (Qwen3-TTS's CustomVoice names its speakers: --voice speaker:Vivian; a bare name would be taken for a clone). Any -p speaker=…, voice_clone_id=… or voice=… you give wins over it.

export ARGUS_MASK_TTS_URL=http://127.0.0.1:8034/synthesise/stream
export ARGUS_MASK_TTS_VOICE=inara-nobility
argus-mask say "Hello from the other side of the room." --face stock-01

argus-mask say "Or through a Wyoming server." --face stock-01 --tts-url wyoming://<LAN_HOST>:10200 --voice "Claribel Dervla"

say sends the text one sentence at a time and leaves --pause ms of silence between sentences (default 300; twice that between paragraphs, which are blank lines in the text). TTS models run sentences into each other, and a short request wanders less than a long one. --pause 0 sends the text whole. By default say also tidies the speech (argus_mask.speech): numbers, times, dates and symbols go as words ("9:30am" as "nine thirty a m"), since a model guesses at digits; a sentence of one to three words goes with the next, since a lone "Greetings." comes back furthest from the rest in pitch and level; and each sentence is faded in and out (5 ms, 30 ms) and brought to the same speech level (-20 dBFS, within 9 dB), since a model often stops while the last sound is still at full level, and sentences generated apart differ in loudness. With --stream there are fades but no levelling. --as-is (the Broadcast page's Tidy speech checkbox) sends the text as written and plays the audio as it came. --context (experimental, off by default; the Broadcast page's Previous sentence as context) sends each sentence to an HTTP service with the one before it, its text and levelled audio, for a CSM-style model like Miso to carry on from; a service that refuses the field is asked again without it. With a cloned voice it can make the tone drift rather than hold. say also puts 150 ms of silence before the utterance and 300 ms after it: TTS models start speaking on their first sample, and an output device that has to wake up would otherwise eat the first syllable.

Recording a clip

Screen capture fights the viewer for the machine and leaves the audio to the operating system. A take is deterministic, so a clip need not be captured at all:

uv sync --extra record && uv run playwright install chromium-headless-shell   # once
argus-mask record --face stock-01 --out clip.mp4 &
argus-mask say "Hello from the bridge." --face stock-01

record listens on the face's viewer socket, keeps the next session's header, frames and audio, renders every frame in a headless browser with the viewer's high look (--quality low|medium|high) at --width×--height (default 1280×720; --display hld --width 1080 --height 1920 for the portrait panel), and muxes them with the audio using the ffmpeg on PATH or the one the extra brings. The session's take and WAV are written beside the clip. Rendering runs at a few frames a second in software; a 20 s clip takes a minute or two and never drops a frame.

Wire

schema/mask-wire.schema.json is the contract. A viewer socket carries:

• one header (JSON): session, fps, seed, the order of the blendshape weights, the audio format;
• frame messages (JSON): seq, t_ms on the audio timeline, weights, head rotation in degrees;
• audio chunks (binary): 8 bytes of little-endian float64 giving the chunk's start in ms, then mono f32le PCM. The server sends a chunk only once frames reach its end, so a driver that answers late delays the sound, never the mouth;
• one end (JSON).

A take file is the same header, frames and end, one JSON object per line.

A source socket takes one JSON hello (sample_rate, channels, format, optional seed), then binary PCM, then {"type": "end"}. Sources should send no faster than real time plus about a second; a viewer that falls 2048 messages behind is dropped.

Agent clients: HTTP push and speak

An agent that would rather not speak the socket protocol can drive a face over plain HTTP. Both routes are sources like any other: one source per face (409 if the face is taken), refused by a read-only server (403) and by another site's page (the same origin guard as every write), refused with the reason when the driver is down (503; never a fall-back to energy). Each request lasts as long as the speech, played at its pace (real time, about a second ahead, as the CLI sends), and answers with the stream's closing numbers once the face is free again:

{"session_id": "54a3d9b0…", "frames": 90, "duration_ms": 3000}

POST /push?face=<id> takes audio the agent already has. The server never sees any text. The body is a WAV file, or raw PCM described by the same headers a TTS stream carries (X-Sample-Rate, X-Audio-Format: pcm_f32le or pcm_s16le, X-Channels); at most 25 MB. Query: seed (the idle layer's, default 0), require_screen (true: 409 unless a screen watching the face can be heard, see /health screens).

curl -X POST "http://127.0.0.1:8109/push?face=stock-01&require_screen=true" \
     -H "Content-Type: audio/wav" --data-binary @briefing.wav

# raw PCM, as a TTS service streamed it (24 kHz mono float32 here)
curl -X POST "http://127.0.0.1:8109/push?face=stock-01" \
     -H "X-Sample-Rate: 24000" -H "X-Audio-Format: pcm_f32le" -H "X-Channels: 1" --data-binary @speech.f32

To play the same speech in the house (Snapcast) and on the face, synthesise once and send the same audio to both.

POST /speak takes text, as argus-mask say does, through the server's own TTS service (ARGUS_MASK_TTS_URL; 422 if none is set). The text goes to that service and is never logged. JSON body: face, text, and optionally voice (default ARGUS_MASK_TTS_VOICE; a clone, a speaker number or speaker:<name>), instruction (how to say it, for a service that takes one, as Qwen3-TTS's CustomVoice does: "calm, low and measured"; never logged), temperature, seed, pause (ms between sentences, default 300), tidy (default true: numbers as words, short sentences joined, even loudness, soft edges), require_screen. The speech is fetched whole before the face is claimed, so a slow TTS delays the start, never the mouth.

curl -X POST http://127.0.0.1:8109/speak -H "Content-Type: application/json" \
     -d '{"face": "stock-01", "text": "Good morning. Three things to look at today.", "temperature": 0.7}'

From Python, argus_mask.client.push and say drive the socket directly and report each step as it happens.

Develop

make install   # venv + editable install with every extra
make test
make lint
make schema    # after changing models.py

Tests use fake Audio2Face and TTS services and a one-triangle .glb built in code. The viewer is not covered by CI; check it by hand with argus-mask push. The real Audio2Face server has its own live check, a2f/check.py, run by hand against a running container.

CI / Release

• CI runs via the shared argus-ci reusable workflow, plus argus-mask schema --check.
• Release publishes to PyPI (OIDC trusted publishing) and GHCR on v* tags.
• Versioning is derived from git tags via hatch-vcs: tag vX.Y.Z to cut a release.

This repo was scaffolded from argus-pkg-template. Run copier update to pull template changes.

The viewer vendors three.js (MIT); its licence is in src/argus_mask/viewer/vendor/LICENSE-three.