bithuman

Run a bitHuman avatar on your own machine, in your own process.

pip install bithuman
DEMO=$(python -c 'import bithuman,os; print(os.path.join(os.path.dirname(bithuman.__file__), "assets", "demo_sample.wav"))')
python -m bithuman render sofia-ramirez "$DEMO"     # -> sofia-ramirez.mp4

Two arguments — an avatar and some audio — and a video you can play (with your API secret in BITHUMAN_API_SECRET; see the key). The avatar is a showcase name (python -m bithuman list), your agent's ten-character code (python -m bithuman list --mine; fetched once, which is free), or a file you already have. The audio above is 15 s of speech that ships with this package, so the first run needs nothing you do not already have. The commands are spelled as in the separate bithuman CLI: render <avatar> <audio> [-o out.mp4] [--limit N] [--json] and list [--mine]. If you signed in with that CLI (bithuman login, which this package does not install), the secret it saved is read here too.

In your own program it is the same two things. The command above fetched the avatar to ~/.cache/bithuman/showcase/:

import bithuman, os

speech = os.path.join(os.path.dirname(bithuman.__file__),
                      "assets", "demo_sample.wav")   # 15 s, ships in the wheel
avatar_file = os.path.expanduser("~/.cache/bithuman/showcase/sofia-ramirez.imx")

avatar = bithuman.open(avatar_file)
frames = 0
for image in avatar.render(speech):   # each image: (height, width, 3) uint8, RGB
    frames += 1
print(frames, "frames")

That is the whole thing: open an avatar, then render audio through it.

every family, the same two lines

Essence 1, Essence 2 (full body essence-2, or face only essence-2f) and Expression 2 avatars are opened and rendered by the code above, unchanged. Nothing you write says which one you have, and you do not have to know.

Two families need one extra package on the machine:

pip install "bithuman[essence-2]"      # the new Essence 2
pip install "bithuman[expression-2]"   # Expression 2

Open one of those avatars without its extra and the refusal says so, and says that line. Nothing else differs.

from the command line

python -m bithuman render <avatar> speech.wav              # -> <avatar>.mp4
python -m bithuman render <avatar> speech.wav --mode face  # Essence 2: face only
python -m bithuman pull <avatar>                            # fetch it now, render later with no network
python -m bithuman pack redeem                              # install a prepaid offline pack on this machine

An agent code renders in the agent's own model. An avatar made with Essence 2 renders in Essence 2; an agent made earlier keeps the model it has (Essence 2 legacy, Expression 2, Essence 1). --model picks another of the agent's models: essence-2, essence-2f (face), expression-2, or essence-2-legacy.

With an offline pack installed, a machine renders with no network and no API secret until the pack is spent; an avatar fetched before is used as it is.

The surface

There is nothing to configure: this package runs the avatar on this machine. For a live conversation, where audio arrives as it is spoken, use AsyncBithuman (streaming below).

audio in

audio is 16 kHz mono, and it is either a buffer or a stream — the same call:

avatar.render(speech)                      # an audio file path
avatar.render(samples)                     # int16 or float32 in [-1, 1]
avatar.render(raw_bytes)                   # 16 kHz mono, signed 16-bit
avatar.render(microphone())                # any iterable of the above

frames out

Each frame is a (height, width, 3) uint8 array in RGB order, in order, at the avatar's own frame rate, which is a property of the avatar, not something to choose: 25 per second for Essence 2, 20 per second for Expression 2.

import cv2
for image in avatar.render(speech):
    cv2.imshow("avatar", image[:, :, ::-1])   # OpenCV wants BGR
    cv2.waitKey(1)

stopping early

Someone interrupting the avatar is "stop consuming and close the iterator":

images = avatar.render(speech)
for n, image in enumerate(images):
    if n == 50:                       # e.g. the user started talking
        images.close()
        break

releasing it

with frees everything at the end of the block; without it, the avatar is freed when it is garbage collected.

with bithuman.open(avatar_file) as avatar:
    for image in avatar.render(speech):
        frames += 1

streaming a live conversation

AsyncBithuman takes audio as it arrives and yields frames with their audio, paced at the avatar's play rate:

from bithuman import AsyncBithuman          # inside an async function:

avatar = await AsyncBithuman.create(model_path=avatar_file)   # reads BITHUMAN_API_SECRET
await avatar.push_audio(pcm16_bytes, 16000, last_chunk=False)  # as the audio arrives
await avatar.flush()                                            # end of the reply
async for frame in avatar.run():
    if frame.has_image:
        show(frame.bgr_image)            # BGR numpy array
    if frame.audio_chunk:
        play(frame.audio_chunk.array)    # the audio that goes with this frame
await avatar.shutdown()

The full example is at https://docs.bithuman.ai/platforms/python/app.

The four refusals

Each one leads to a different fix, and none of them asks you to know anything about how we are built.

try:
    avatar = bithuman.open(avatar_file)
    for image in avatar.render(speech):
        pass
except bithuman.InvalidAvatar:
    ...   # fix the path or the code, or fetch the avatar again
except bithuman.NotSupported:
    ...   # use the cloud package, or another device
except bithuman.NotAuthorised:
    ...   # fix the credential
except bithuman.Failed:
    ...   # retry, then report it

Every one of them is an AvatarError, so except bithuman.AvatarError catches all four.

The key

Rendering is metered, and the key belongs in the environment rather than in your code:

export BITHUMAN_API_SECRET=...

Without one, render refuses with NotAuthorised before it hands you a frame. Get a key at https://www.bithuman.ai/developer/api-keys.

python -m bithuman also reads a .env file in the current directory; that is deprecated and will stop, so export the variable instead. What is already in the environment always wins.

Where it runs

ffmpeg is used to read an audio file when it is on your PATH; when it is not, the decoder this package already installs reads the same file in this process, so it is not something to install first.

Three environment variables:

This package never puts a command on your PATH

pip install bithuman installs a library and nothing else — and python -m bithuman is why that costs you nothing: a module needs no script, cannot collide with one, and is there the moment pip finishes. The full bithuman command-line tool (a live avatar, a conversation) is a different artifact and is not installed with pip:

curl -fsSL https://install.bithuman.ai | sh
brew tap bithuman/bithuman https://gitlab.com/bithuman/sdk/homebrew-bithuman \
  && brew install bithuman/bithuman/bithuman-cli         # macOS, equivalently (always tap with the URL)

That is on purpose: a pip-installed command named bithuman would overwrite the one the CLI installer or Homebrew put at the same path.

Using it from a LiveKit agent

The LiveKit integration is a separate package, livekit-plugins-bithuman, published by LiveKit out of github.com/livekit/agents:

pip install livekit-plugins-bithuman "bithuman[expression-2]"

Do not give that plugin your account API secret. Up to and including 1.8.4, the plugin copies the secret it is given into the avatar's participant attributes, which every participant in the LiveKit room can read. Keep the secret on your server under another name (BITHUMAN_MASTER_SECRET), mint a one-hour runtime token per session (POST /v1/runtime-tokens/mint) and pass that token to the plugin instead. The worker is at https://docs.bithuman.ai/platforms/livekit.

Licence

Commercial software. Terms of use: https://www.bithuman.ai/legal/terms