Skip to main content

haru_mlx

Haru on Apple silicon: the MLX implementation of the Haru full-duplex spoken dialogue model. It loads the same checkpoint as the PyTorch package haru (higurashi-ai/Haru-4B) and has the same Python API.

Haru listens and speaks at the same time. Every 133 ms it takes one frame of your audio and produces one frame of its own speech together with its text.

Install

pip install -e .                 # from this directory

Requires a Mac with Apple silicon and Python 3.10 or later. The checkpoint is about 10 GB; it is downloaded from Hugging Face on first use.

Talk to it

Haru for macOS is the way to talk to the model on a Mac: a native app that handles the microphone and speakers with Core Audio and runs this package as its model server. Build it with swift/build_app.sh.

Haru ships without voice clips: in the app, choose or record a clip of the voice the model should speak with (one speaker, 3 seconds to 3 minutes). Only use voices you have the right to use.

To give the app a default voice, save a clip as haru_mlx/assets/voices/<language>.flac (en.flac, ja.flac, ...; the path is relative to this directory) and reopen the app: it then offers the clip as the default voice of that language. Without the file, the app does not show the option.

Offline

python -m haru_mlx.run_inference user.wav reply.wav --reference voice.wav --system-prompt "..."

A recording in, the model's side of the conversation out. --model DIR or --hf-repo REPO selects the checkpoint, and --quantize its precision. The app starts in bfloat16 and can switch to 8-bit in its settings:

--quantize Backbone weights
none (default) bfloat16, as published
8 8-bit: smaller, and a frame reads fewer bytes

Quantization covers the language model's weight matrices and the embedding table; the audio codec, the audio head and all normalization gains keep the checkpoint's precision.

Python API

import numpy as np
from haru_mlx import DuplexConfig, DuplexSession, HaruModel
from haru_mlx.audio import load_audio, prepare_reference

model = HaruModel.from_pretrained("higurashi-ai/Haru-4B")       # quantize=8 for an 8-bit backbone
voice = prepare_reference(load_audio("voice.wav"))          # mono float32, 24 kHz
session = DuplexSession(model, system_prompt="...", reference=voice, config=DuplexConfig(seed=0))

frame = session.start()                                     # model frame 0
while not frame.done:
    user = np.zeros(3200, dtype=np.float32)                 # 133 ms of your audio
    frame = session.step(user)                              # -> frame.audio (3200 samples), frame.text

DuplexConfig has the same fields as in haru. A live client can also hand over microphone audio as it arrives, which spreads the audio encoder's work over the frame:

session.feed(block)              # any length, as the microphone delivers it
if session.frames_ready:
    frame = session.step()

MLX keeps its GPU streams per thread: load the model and drive its sessions from one thread (haru_mlx.worker.MLXWorker is that thread for the app's model server).

Metal kernels

Two inner loops of a frame run as custom Metal kernels: the Gated DeltaNet recurrence of the language model and the depthwise convolution blocks of the audio codec. Both have plain MLX fallbacks with the same results, which are used on devices without Metal.

Layout

Module
models/haru.py HaruModel: the three input streams, the backbone, text logits
models/qwen3_5.py Qwen3.5 text backbone (Gated DeltaNet + attention), laid out for one position per frame
models/gated_delta.py the Gated DeltaNet recurrence (Metal kernel, with an array fallback)
models/meanflow_head.py one-step MeanFlow head: hidden state -> audio latent
models/vibevoice_codec.py VibeVoice acoustic codec, streaming (Metal kernel for its depthwise convolutions, with an array fallback)
models/loaders.py, weights.py checkpoint loading; the tensor -> parameter mapping
session.py DuplexSession: one conversation
native.py the model server of the macOS app (a Unix socket)
run_inference.py offline inference

Tests

pip install -e ".[dev]"
pytest                                   # the tests that need no checkpoint
HARU_MODEL=/path/to/Haru-4B pytest       # + checkpoint coverage and a short conversation

Most tests run without MLX. tests/README.md describes the numerical comparison with the PyTorch package and the checks to run on a Mac.

Metadata

Release files for haru-speech-mlx 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for haru-speech-mlx 0.1.0
File Size Uploaded Provenance Actions
haru_speech_mlx-0.1.0.tar.gz 67.0 kB With provenance Details for haru_speech_mlx-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for haru-speech-mlx 0.1.0
File Interpreter ABI Platform Actions
haru_speech_mlx-0.1.0-py3-none-any.whl Python 3 none any Details for haru_speech_mlx-0.1.0-py3-none-any.whl

Total release size: 132.4 kB

Release files / haru_speech_mlx-0.1.0.tar.gz

Download URL haru_speech_mlx-0.1.0.tar.gz
Size 67.0 kB
Tags Source
SHA-256 checksum
How to use checksums
c52ac5261d2ef205662e5920aa76b5da6c96a547cc400104686b9aae3cfbae71
BLAKE2b-256 checksum
How to use checksums
5526512ee0f6e655dfc7ddc9b8002a183b28c82e1998d06a5c6844046c469beb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.

Transparency log

Release files / haru_speech_mlx-0.1.0-py3-none-any.whl

Download URL haru_speech_mlx-0.1.0-py3-none-any.whl
Size 65.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
5e57120ff2649ebce3b498aad346502c1b517c78e700952cb2e5289b3c866561
BLAKE2b-256 checksum
How to use checksums
c82f8fb954abbe3b17093cd1f29ddad467633d0bebbccbefab82bc782a01b337
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.

Transparency log

Security for haru-speech-mlx 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for haru_speech_mlx-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page