voxelpic

The voxelpic C library provides functionality to transcode point clouds to and from RGB images. Starting with a sequence of colored point clouds like the one below:

https://github.com/user-attachments/assets/de768bbf-b180-472a-873f-da10aa52e05a

The library builds an OcTree (to a specified depth) for each point cloud in the sequence and then encodes a level of that OcTree to an RGB image, which looks like this:

The octree is rendered from four orthographic cameras, and the system stores the color of each voxel and its location in the octree along the axis of projection:

Resulting in a video like this one that encodes the above sequence:

https://github.com/user-attachments/assets/68bc1afb-3f37-49c5-b0ca-22594007c1e6

To decode this sequence back into a point cloud, the library decodes the image into an OcTree level and then returns a point cloud consisting of voxel centroids and voxel colors. In fact, the original animation above is the result of decoding the voxelpic sequence shown.

[!NOTE] Encoding Depth as Color

The technique used to encode the depth as RGB color values has its origins in a codec developed by Intel for the RealSense camera. There is more information in the Hue Codec Repository. We found that limiting the range of values encoded by this codec to the central 1273 values reduced ambiguity at decode time and resulted in much tighter error bars, especially after h.264 compression.

Getting Started

To build this library, its tests, and tools, you will need CMake and a C compiler. The rests additionally require Python to build the test files. The vpic tool further requires libpng if you want to be able to read and write PNG images. Once you have the requirements in place, you can build as normal, for example:

mkdir build
cd build
cmake .. --preset release
cmake --build .
ctest

vpic tool

In addition to the C library, we provide a command-line tool that acts both as a way to try out the library and as tutorial and example code on its usage. You can build octrees from point cloud data, encode them as images, and decode them back into octree levels. For example:

./vpic -m build -c sample_cloud.dat -v sample_voxels.dat -d 7
Building an OcTree to depth 7 from 20000 points...done.
Saving 17425 voxels to sample_voxels.dat...done.

./vpic -m encode -v sample_voxels.dat -i sample.png
Loading voxels from sample_voxels.dat...done
Encoding 17425 voxels at level 7 to sample.png...done.

./vpic -m decode -i sample.png -v sample_voxels_out.dat
Decoding voxels from sample.png...done
Saving 10666 voxels to sample_voxels_out.dat...done.

sample_cloud.dat is generated by cloud_gen.py and looks like a spiral galaxy. The resulting image sample.png looks like this:

Note that when decoded, we have lost some voxels. This is because any interior voxels which are occluded from all four views cannot be encoded by this format and are lost. sample_cloud.dat demonstrates the cloud data format, which is:

(size: int32 in network byte order (i.e. big-endian))
(x: float32, y: float32, z: float32, red: uint8, green: uint8, blue: uint8) x size

voxelpic Python Library

We also provide a Python interface via the voxelpic Python library. To install this library, run:

pip install voxelpic

to get the latest released version or:

pip install -e .

to build the library directly from a clone of this repository.

The library understands numpy arrays and can use them directly. To achieve the same functionality as the vpic library above, we could use the following Python script:

import cv2
import voxelpic as vp

# NB all variables below are numpy arrays
sample_cloud = vp.PointCloud.load("sample_cloud.dat")
image = vp.encode(sample_cloud, depth=7)
image_bgr = cv2.cvtColor(image, cv2.COLOR_RGBA2BGRA)
cv2.imwrite("sample.png", image_bgr)
sample_cloud_out = vp.decode(image)

Trademarks This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft’s Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party’s policies.