aestream 0.4.0

Streaming library for Address-Event Representation (AER) data

AEStream - Address Event streaming library

AEDAT parses event-based dynamic-vision system (DVS) data from an input source and streams it to a sink (see table below).

Installation

AEStream can be installed with the Python package manage pip. However, AEStream depends on PyTorch, LZ4, and libcaer, so please install these dependencies manually before you install AEStream. For PyTorch, recall whether you are using the CPU or CUDA version and use the corresponding command below to install AEStream:

PyTorch version	Command
With CPU	pip install aestream --extra-index-url https://download.pytorch.org/whl/cpu
With CUDA	pip install aestream --extra-index-url https://download.pytorch.org/whl/cu116

Note that this uses CUDA 11.6. Other versions can be found and used by copying links from the website of PyTorch.

We do not currently support other platforms than Linux, but contributions are most welcome.

Usage (Python)

The Python API exposes two classes for reading DVS data sources into PyTorch tensors: USBInput and UDPInput.

# Stream events from a DVS camera over USB
with USBInput((640, 480)) as stream:
    while True:
        frame = stream.read() # Provides a (640, 480) tensor
        ...

# Stream events from UDP port 3333 (default)
with UDPInput((640, 480), port=3333) as stream:
    while True:
        frame = stream.read() # Provides a (640, 480) tensor
        ...

More examples can be found in our example folder. Please note the examples may require additional dependencies (such as Norse for spiking networks or PySDL for rendering). To install all the requirements, simply stand in the aestream root directory and run pip install -r example/requirements.txt

Usage (CLI)

Installing AEStream also gives access to the command-line interface (CLI) aestream. To use aestraem, simply provide an input source and an optional output sink (defaulting to STDOUT):

aestream input <input source> [output <output sink>]

Supported Inputs and Outputs

Input	Description	Usage
DAVIS, DVXPlorer	Inivation DVS Camera over USB	input inivation
EVK Cameras	Prophesee DVS camera over USB	input prophesee
File	AEDAT file format as .aedat or .aedat4	input file x.aedat4

Output	Description	Usage
STDOUT	Standard output (default output)	output stdout
Ethernet over UDP	Outputs to a given IP and port using the SPIF protocol	output udp 10.0.0.1 1234
.aedat4 file	Output to .aedat4 format	output file my_file.aedat4
CSV file	Output to comma-separated-value (CSV) file format	output file my_file.txt

CLI examples

Example	Syntax
Read file to STDOUT	aestream input file example/davis.aedat4
Stream DVS Davis346 (USB 0:2) by iniVation AG to STDOUT (Note, requires Inivation libraries)	aestream input inivation output stdout
Stream Prophesee 640x480 (serial Prophesee:hal_plugin_gen31_fx3:00001464) to STDOUT (Note, requires Metavision SDK)	aestream input output stdout
Read file to remote IP X.X.X.X	aestream input file example/davis.aedat4 output udp X.X.X.X

Custom installation (C++)

AEStream requires libtorch. Metavision SDK and libcaer are optional dependencies, but are needed for connecting to Prophesee and Inivation cameras respectively.

AEStream is based on C++20. Since C++20 is not yet fully supported by all compilers, we recommend using GCC >= 10.2.

To build the binaries of this repository, run the following code:

export CMAKE_PREFIX_PATH=`absolute path to libtorch/`
# Optional: Ensure paths to libcaer, Metavision, or OpenCV is in place
mkdir build/
cd build/
cmake -GNinja ..
ninja

If your default C++ compiler doesn't support C++ 20, you will have to install an up-to-date compiler and provide the environmental variable CXX. For instance like this: CXX=/path/to/g++ cmake -GNinja ..

Inivation cameras

For Inivation cameras, the libcaer library needs to be available, either by a -DCMAKE_PREFIX_PATH flag to cmake or included in the PATH environmental variable. For examble: cmake -GNinja -DCMAKE_PREFIX_PATH=/path/to/libcaer. Inivation made the library available for most operating systems, but you may have to build it yourself.

Prophesee cameras

For Prophesee cameras, a version of the Metavision SDK needs to be present. The open-source version the SDK openeb is available with installation instructions at https://github.com/prophesee-ai/openeb. Using openeb, it should be sufficient to install it using cmake && make && make install to put it in your path. Otherwise, you can point to it using the -DCMAKE_PREFIX_PATH option in cmake.

Acknowledgments

AEStream is created by

• Jens E. Pedersen (@GitHub jegp), doctoral student at KTH Royal Institute of Technology, Sweden.
• Christian Pehle (@GitHub cpehle), PostDoc at University of Heidelberg, Germany.

The work has received funding from the EC Horizon 2020 Framework Programme under Grant Agreements 785907 and 945539 (HBP) and by the Deutsche Forschungsgemeinschaft (DFG, German Research Fundation) under Germany's Excellence Strategy EXC 2181/1 - 390900948 (the Heidelberg STRUCTURES Excellence Cluster).

Thanks to Philipp Mondorf for interfacing with Metavision SDK and preliminary network code.

Citation

Please cite aestream if you use it in your work:

@software{aestream2022,
  author       = {Pedersen, Jens Egholm and
                  Pehle, Christian-Gernot},
  title        = {AEStream - Address Event Streaming library},
  month        = {August},
  year         = 2022,
  publisher    = {Zenodo},
  version      = {0.4.0},
  doi          = {10.5281/zenodo.6322829},
  url          = {https://doi.org/10.5281/zenodo.6322829}
}