aestream 0.6.0

Streaming library for Address-Event Representation (AER) data

AEStream effiently sends event-based data from A to B. AEStream can be used from the command-line, via Python, or as a C++ library. We support multiple inputs and outputs, providing seamless integration with files, event cameras, network data, Python libraries via Numpy or PyTorch, and visualization tools.

Read more about the inner workings of the library in the AEStream publication.

Installation

Read more in our installation guide

The fastest way to install AEStream is by using pip: pip install aestream. See below for other sources.

Source	Installation	Description
pip	pip install aestream pip install aestream[torch] pip install aestream --no-binary	PyTorch support Requires camera drivers*
nix	nix run github:aestream/aestream nix develop github:aestream/aestream	Command-line interface Python environment
docker	See Installation documentation

* Event camera support requires available drivers. A step-by-step guide is available in our documentation.

Contributions to support AEStream on additional platforms are always welcome.

Usage (Python)

Read more in our Python usage guide

AEStream can process fixed input sources like files like so:

FileInput("file", (640, 480)).load()

Usage: stream real-time data in Python

AEStream also supports streaming data in real-time without strict guarantees on orders. This is particularly useful in real-time scenarios, for instance when operating with USBInput or UDPInput

# Stream events from a DVS camera over USB
with USBInput((640, 480)) as stream:
    while True:
        frame = stream.read() # Provides a (640, 480) tensor
        ...

# Stream events from UDP port 3333 (default)
with UDPInput((640, 480), port=3333) as stream:
    while True:
        frame = stream.read() # Provides a (640, 480) tensor
        ...

More examples can be found in our example folder. Please note the examples may require additional dependencies (such as Norse for spiking networks or PySDL for rendering). To install all the requirements, simply stand in the aestream root directory and run pip install -r example/requirements.txt

Example: real-time edge detection with spiking neural networks

We stream events from a camera connected via USB and process them on a GPU in real-time using the spiking neural network library, Norse using fewer than 50 lines of Python. The left panel in the video shows the raw signal, while the middle and right panels show horizontal and vertical edge detection respectively. The full example can be found in example/usb_edgedetection.py

Usage (CLI)

Read more in our CLI usage documentation page

Installing AEStream also gives access to the command-line interface (CLI) aestream. To use aestraem, simply provide an input source and an optional output sink (defaulting to STDOUT):

aestream input <input source> [output <output sink>]

Supported Inputs and Outputs

Input	Description	Example usage
DAVIS, DVXPlorer	Inivation DVS Camera over USB	input inivation
EVK Cameras	Prophesee DVS camera over USB	input prophesee
File	Reads .aedat, .aedat4, .csv, .dat, or .raw files	input file x.aedat4
SynSense Speck	Stream events via ZMQ	input speck
UDP network	Receives stream of events via the SPIF protocol	input udp

Output	Description	Example usage
STDOUT	Standard output (default output)	output stdout
Ethernet over UDP	Outputs to a given IP and port using the SPIF protocol	output udp 10.0.0.1 1234
File: .aedat4	Output to .aedat4 format	output file my_file.aedat4
File: .csv	Output to comma-separated-value (CSV) file format	output file my_file.csv
Viewer	View live event stream	output view

CLI examples

Example	Syntax
View live stream of Inivation camera (requires Inivation drivers)	aestream input inivation output view
Stream Prophesee camera over the network to 10.0.0.1 (requires Metavision SDK)	aestream input output udp 10.0.0.1
Convert .dat file to .aedat4	aestream input example/sample.dat output file converted.aedat4

Acknowledgments

AEStream is developed by (in alphabetical order):

• Cameron Barker (@GitHub cameron-git)
• Juan Pablo Romero Bermudez (@GitHub jpromerob)
• Alexander Hadjivanov (@Github cantordust)
• Emil Jansson (@GitHub emijan-kth)
• Jens E. Pedersen (@GitHub jegp)
• Christian Pehle (@GitHub cpehle)

The work has received funding from the EC Horizon 2020 Framework Programme under Grant Agreements 785907 and 945539 (HBP) and by the Deutsche Forschungsgemeinschaft (DFG, German Research Fundation) under Germany's Excellence Strategy EXC 2181/1 - 390900948 (the Heidelberg STRUCTURES Excellence Cluster).

Thanks to Philipp Mondorf for interfacing with Metavision SDK and preliminary network code.

Citation

Please cite aestream if you use it in your work:

@misc{aestream,
  doi = {10.48550/ARXIV.2212.10719},
  url = {https://arxiv.org/abs/2212.10719},
  author = {Pedersen, Jens Egholm and Conradt, Jörg},
  title = {AEStream: Accelerated event-based processing with coroutines},
  publisher = {arXiv},
  year = {2022},
}