vmcp 1.0.6

Virtual Motion Capture protocol package

Virtual Motion Capture (VMC) protocol package for Python

The VMC protocol is used for realtime Motion Capture transmission and based on the Open Sound Control (OSC) protcol. This Python module is an highly modular implementation of it's most basic features.

Features

VMC protocol

• Root transform (Supported specs: v2.0.0, v2.1.0)
• Bone transform
• Device transform (Supported specs: v2.2.0, v2.3.0)
• Blendshapes
• State changes (Supported specs: v1.0.0, v2.5.0, v2.7.0)
• Relative time

and it's receiving events.

OSC protocol

• Multiple independent sender- and receiver-channels
• TimeTag support

Specials

• Independent usage of the VMC and OSC protocol layers possible.
• Backend agnostic (feel free to replace osc4py3 with something else).

Installation

• If you only want to use the VMC protocol layer, you can simply install it via pip install vmcp (without OSC).
• If you want it ready-to-use including OSC, install it with the dependencies of your choosen backend. For example for vmcp.osc.backend.osc4py3.as_comthreads you install it via pip install vmcp[osc4py3] (recommended)

Usage

You may find the following sections in the documentation helpful:

• vmcp.protocol
• vmcp.typing
• vmcp.events
• vmcp.facades
• vmcp.osc

Virtual Motion Capture (VMC) protocol

Basic VMCP example

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# SPDX-License-Identifier: AGPL-3.0-or-later

"""Basic VMC protocol example."""

from math import radians
# OSC
from vmcp.osc import OSC
from vmcp.osc.typing import Message
from vmcp.osc.backend.osc4py3 import as_comthreads as backend
# VMC protocol layer
from vmcp.events import (
    Event,
    RootTransformEvent,
    BoneTransformEvent,
    BlendShapeEvent,
    BlendShapeApplyEvent,
    DeviceTransformEvent,
    StateEvent,
    RelativeTimeEvent
)
from vmcp.typing import (
    CoordinateVector,
    Quaternion,
    Bone,
    DeviceType,
    BlendShapeKey as AbstractBlendShapeKey,
    ModelState,
    Timestamp
)
from vmcp.protocol import (
    root_transform,
    bone_transform,
    device_transform,
    blendshape,
    blendshape_apply,
    state,
    time
)
# Facades for easy usage (optional)
from vmcp.facades import on_receive


# Required, if blend shapes are used.
class BlendShapeKey(AbstractBlendShapeKey):
    """Example model blend shape keys.

    Depends on the used avatar model.

    """

    # Eye blink
    EYES_BLINK_R = "Blink_R"
    # Face expressions
    FACE_FUN = "Fun"


LISTENING = True


def received(event: Event):
    """Receive transmission."""
    print(event)
    if isinstance(event, DeviceTransformEvent):
        print(event.device_type)
        print(event.is_local)
    if isinstance(event, RelativeTimeEvent):
        global LISTENING  # pylint: disable=global-statement
        LISTENING = False


try:
    osc = OSC(backend)
    with osc.open():
        # Receiver
        in1 = osc.create_receiver("127.0.0.1", 39539, "receiver1").open()
        on_receive(in1, RootTransformEvent, received)
        on_receive(in1, BoneTransformEvent, received)
        on_receive(in1, DeviceTransformEvent, received)
        on_receive(in1, BlendShapeEvent, received)
        on_receive(in1, BlendShapeApplyEvent, received)
        on_receive(in1, StateEvent, received)
        on_receive(in1, RelativeTimeEvent, received)
        # Sender
        osc.create_sender("127.0.0.1", 39539, "sender1").open().send(
            (
                Message(*root_transform(
                    CoordinateVector(.5, .2, .5),
                    Quaternion.identity()
                )),
                Message(*bone_transform(
                    Bone.LEFT_UPPER_LEG,
                    CoordinateVector.identity(),
                    Quaternion.from_euler(0, 0, radians(-45))
                )),
                Message(*bone_transform(
                    Bone.RIGHT_LOWER_ARM,
                    CoordinateVector.identity(),
                    Quaternion(0, 0, 0.3826834323650898, 0.9238795325112867)
                )),
                Message(*device_transform(
                    DeviceType.HMD,
                    "Head",
                    CoordinateVector.identity(),
                    Quaternion.identity()
                )),
                Message(*blendshape(
                    BlendShapeKey.FACE_FUN,
                    1.0
                )),
                Message(*blendshape(
                    BlendShapeKey.EYES_BLINK_R,
                    1.0
                )),
                Message(*blendshape_apply()),
                Message(*state(ModelState.LOADED)),
                Message(*time(Timestamp()))
            )
        )
        # Processing
        while LISTENING:
            osc.run()
except KeyboardInterrupt:
    print("Canceled.")
finally:
    osc.close()

Only Open Sound Control (OSC) protocol

You are free use use the OSC protocol directly.

Basic OSC example

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# SPDX-License-Identifier: AGPL-3.0-or-later

"""Basic OSC protocol example."""

from typing import Any
from vmcp.osc import OSC
from vmcp.osc.typing import Message
from vmcp.osc.backend.osc4py3 import as_comthreads as backend

LISTENING: bool = True


def received(*args: Any):
    """Receive transmission."""
    global LISTENING  # pylint: disable=global-statement
    print(args)
    LISTENING = False


try:
    osc = OSC(backend)
    with osc.open():
        # Receiver channel
        in1 = osc.create_receiver("127.0.0.1", 39539, "receiver1")
        in1.register_handler("/test/one", received)
        in1.open()
        # Sender channel
        out1 = osc.create_sender("127.0.0.1", 39539, "sender1").open()
        out1.send(Message("/test/one", ",sif", ["first", 672, 8.871]))
        # Additional sender channel
        osc.create_sender("127.0.0.1", 39540, "sender2").open().send(
            (
                Message("/test/one", ",sif", ["second", 672, 8.871]),
                Message("/test/two", ",sif", ["third", 234, 2.513])
            )
        )
        # Processing
        while LISTENING:
            osc.run()
except KeyboardInterrupt:
    print("Cancheled.")
finally:
    osc.close()

VMC protocol layer only

If you are only want to use the VMC protocol layer without the underlying OSC stuff.

VMCP only example

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# SPDX-License-Identifier: AGPL-3.0-or-later

"""VMCP protocol layer only example."""

from vmcp.typing import (
    CoordinateVector,
    Quaternion
)
from vmcp.protocol import root_transform
from vmcp.events import (
    Event,
    RootTransformEvent
)


def received(event: Event):
    """Receive transmission."""
    print(repr(event))


position = CoordinateVector(1.0, 2.0, 3.0)
rotation = Quaternion.identity()

print(
    root_transform(
        position,
        rotation
    )
)
received(
    RootTransformEvent.from_message_data(
        "example",
        RootTransformEvent.ADDRESS_PATTERN,
        (
            "root",
            position.x, position.y, position.z,
            rotation.x, rotation.y, rotation.z, rotation.w
        )
    )
)

In this case it's NOT required to install any backend dependencies.

Additional examples

Basic OSC with logging

Basic OSC with custom receiving parameters (Backend dependant)

Basic OSC with TimeTag's (Backend dependant)

Currently available backends

vmcp.osc.backend.osc4py3.as_eventloop

You need to call OSC.run() to process message sending and receiving. Depending on the amount of traffic, multiple calls may be required.

Dependencies

• osc4py3

Install

pip install vmcp[osc4py3]

vmcp.osc.backend.osc4py3.as_comthreads

Only message receiving requires OSC.run() calls. Sending runs in separate threads. This backend might be your choice to avoid race conditions inside your application.

Dependencies

• osc4py3

Install

pip install vmcp[osc4py3]

vmcp.osc.backend.osc4py3.as_allthreads

You don't need to call OSC.run(), but it doesn't harm. Everything runs in separate threads.

Dependencies

• osc4py3

Install

pip install vmcp[osc4py3]

Known bugs

• If you want to use the vmcp.osc.backend.osc4py3.as_comthreads backend, then you need to downgrade osc4py3 to version 1.0.3 (pip install osc4py3==1.0.3).
• If you want to use logging with any vmcp.osc.backend.osc4py3.* backend, then you need to use osc4py3 version 1.0.4 or higher.

Contribution

1. Install Git LFS and (Mini)conda
2. Make sure, that conda is available for git (for example run echo ". /c/tools/miniconda3/etc/profile.d/conda.sh" >> ~/.profile in GitBash for Windows).
3. Prepare LFS by issuing the command git lfs install once.
4. Fork and clone this repository.
5. Install git hooks by issuing the command git config core.hooksPath .githooks.
6. Create conda environment and activate it:

   conda create -n vmcp python=3.10
   conda activate vmcp
   

7. Install the package in editable mode with all tools: pip install -e ".[osc4py3,dev]"
8. If you finished your work, uninstall it by: python setup.py develop -u
9. You can delete your conda environment by: conda remove -n vmcp --all

TODO

• Add more VMC protocol commands.
• Add abstract application patterns for Assistant, Performer and Marionette.
• Add VRM model parser.
• Add more OSC backends.

Project state meanings

• Unstable: Still an possibly broken draft or breaking issues are known.
• Refactoring: Structural and technical changes have currently priority.
• Stable: Everything seems working as expected.
• Fixes only: The project is maintained at the minimum level to apply at least fixes.
• Takeover: The project is currently being taked over by another team and will possibly move.
• Not maintained: The project is not maintained any more (ready to take over).

Feature requests are welcomed all the time, of course! ;-)

License

This project is free under the terms of the AGPL 3.0 (or later) license. For more details please see the LICENSE file or: GNU