evlib: Event Camera Data Processing Library

An event camera processing library with a Rust backend and Python bindings, designed for scalable data processing with real-world event camera datasets.

Architecture

evlib keeps a thin Rust core and does all DataFrame work in Polars from Python:

• Rust (evlib._evlib) handles only what cannot be expressed as DataFrame operations: binary format parsing (EVT2/EVT3/EVT2.1, AEDAT, AER, HDF5 with the ECF codec), construction of the Polars frame from decoded primitives, and the native dense scatter-add kernels that build RVT stacked-histogram representations (evlib.representations_rs.stacked_histogram_dense on the CPU, plus _cuda and _metal GPU kernels).
• Python Polars handles all processing: loading filters, filtering (evlib.filtering), and representations (evlib.representations, evlib.rvt). Every query is a lazy Polars LazyFrame collected with a selectable engine, so the same code runs on the CPU streaming engine today and on the GPU via cudf-polars (collect(engine="gpu")) where CUDA is available.

evlib.load_events returns a LazyFrame and applies any time, spatial, or polarity filters as Polars expressions, so loading and filtering fuse into one GPU-collectable query.

EVT2 decode is byte-identical to the OpenEB reference decoder, checked by a committed-digest conformance gate in tests/test_openeb_conformance.py.

What else is in the box

Beyond loading, filtering, and representations, evlib ships full training and evaluation support for event-based detection models:

• evlib.models: E2VID and RVT (Recurrent Vision Transformer), with pretrained weight loading and GPU inference.
• evlib.data: PyTorch datasets and a DataModule for RVT-style training (SequenceRandomDataset, SequenceStreamDataset, SequenceAugmentor, label preprocessing, collate functions).
• evlib.eval: Prophesee-compatible detection mAP scoring.
• evlib.simulation: ESIM video-to-events simulation (requires PyTorch).
• evlib-rvt-preprocess: a console script (installed with the package) that runs the RVT preprocessing pipeline end to end from the command line.

• Quick Start
  • What are Event Cameras?
  • Basic Usage
  • RVT preprocessing backends
  • Performance
• Installation
• Documentation
• Examples
• Development
• Community & Support
• License

Quick Start

What are Event Cameras?

Event cameras (also called neuromorphic or dynamic vision sensors) operate asynchronously: each pixel independently reports brightness changes as they occur, rather than sampling frames at a fixed rate.

Each event is represented as a 4-tuple:

$$e = (x, y, t, p)$$

Where:

• $x, y \in \mathbb{N}$: Pixel coordinates
• $t \in \mathbb{R}^+$: Timestamp (microsecond precision)
• $p \in {-1, +1}$ or ${0, 1}$: Polarity (brightness change direction)

An event fires when the logarithmic brightness change exceeds a threshold:

$$\log(L(x,y,t)) - \log(L(x,y,t_{\text{last}})) > \pm C$$

where $C$ is the contrast threshold. This yields microsecond temporal resolution, 120 dB+ dynamic range, and data sparsity proportional to scene motion.

For a deeper introduction, see the user guide.

Basic Usage

import evlib

# Automatic format detection: returns a Polars LazyFrame
events = evlib.load_events("data/prophesee/samples/evt2/80_balls.raw")

df = events.collect(engine="streaming")
print(f"Loaded {len(df):,} events")
print(f"Resolution: {df['x'].max()} x {df['y'].max()}")
print(f"Duration:   {df['t'].max() - df['t'].min()}")

Chain Polars expressions for efficient filtering and representation extraction:

import evlib
import evlib.representations as evr
import polars as pl

events = evlib.load_events("data/prophesee/samples/hdf5/pedestrians.hdf5")

# Temporal + spatial + polarity filtering, lazily
filtered = events.filter(
    (pl.col("t").dt.total_microseconds() / 1_000_000).is_between(0.1, 0.5)
    & pl.col("x").is_between(100, 500)
    & (pl.col("polarity") == 1)
)

# Produce a stacked histogram ready for an RVT-style model
hist = evr.create_stacked_histogram(
    filtered.collect(),
    height=180, width=240,
    bins=5, window_duration_ms=50.0,
)

The transformation turns a raw asynchronous event stream into a dense, model-ready tensor. Below, the pedestrians sequence: on the left, 250ms of raw events (red +1, blue -1); on the right, the same window as a stacked histogram of five 50ms temporal bins, where the walking figures advance bin to bin:

Both this and a fully reproducible 80_balls version (from the tracked EVT2 sample) are generated by python scripts/generate_representation_figures.py.

See the representations guide for voxel grids, time surfaces, and mixed density stacks.

RVT preprocessing backends

evlib.rvt.process_sequence(...) reproduces the RVT stacked-histogram preprocessing pipeline and offers four interchangeable backends via backend=:

• "polars": Polars on the CPU, or on the cudf GPU engine when you pass an engine= of "gpu" or a pl.GPUEngine(...).
• "rust": Rust dense scatter-add on the CPU.
• "cuda": a custom CUDA scatter-add kernel on an NVIDIA GPU. It loads the nvcc-built librvt_scatter.so via the EVLIB_CUDA_LIB environment variable.
• "metal": a Metal scatter-add kernel on Apple Silicon. Build it with CC=clang maturin develop --features metal.

The underlying native kernels are exposed directly as evlib.representations_rs.stacked_histogram_dense (CPU), stacked_histogram_dense_cuda, and stacked_histogram_dense_metal.

Performance

evlib is bit-validated against the reference implementations it competes with: RVT (PyTorch), tonic, OpenEB, and dv_processing. On the gen4_1mpx validation set (18 sequences, RTX 4090), the RVT preprocessing output matches RVT torch exactly bar a single roughly 1e-10 boundary quirk, and the timings are:

• evlib CUDA: 283.6s, slightly ahead of RVT torch-GPU at 286.3s (parity-plus, about 1.01x).
• evlib Rust-CPU: 406.2s, 1.32x faster than RVT torch-CPU at 534.2s.
• evlib CUDA is 1.88x faster than RVT torch-CPU.

For the standalone representations (20M events, versus tonic NumPy): voxel_grid 1.35x, event_frame 2.9x, time_surface 2.1x.

The Polars GPU engine is not a free win for single operations, and the CUDA-versus-RVT-GPU margin is parity-plus rather than a large speedup. The biggest margins are evlib's CPU backends and the standalone representations.

[!Note]

State of the GPU and Metal work: the CUDA backend is the production GPU path and edges out RVT's torch-GPU pipeline. The Metal backend matches the CPU kernel exactly on an M2 Pro, but about 3x slower there: the workload is memory-bound and the M2 Pro's CPU cores win. Metal is a portability path (an on-device kernel where torch-CUDA cannot run), not a speed win on M2-class hardware; use backend="rust" for the fastest Apple-CPU path.

More plots: the full five-backend chart rvt_final_time.png (and rvt_final_memory.png for peak memory), plus tonic_bench_time.png for the representations-versus-tonic comparison.

Full documentation: https://tallamjr.github.io/evlib/

Installation

# Basic install: macOS and Linux wheels statically link HDF5, so HDF5 files
# (including Prophesee ECF-compressed data) read through the Rust path with
# no system HDF5 install and no hdf5plugin needed.
pip install evlib

# With PyTorch integration
pip install evlib[pytorch]

From source (requires Rust nightly and maturin):

git clone https://github.com/tallamjr/evlib.git
cd evlib
uv venv --python 3.12 && source .venv/bin/activate
uv pip install -e ".[dev]"
maturin develop                           # default minimal build, no HDF5
maturin develop --features hdf5-static    # HDF5 built from source, no system HDF5 needed
maturin develop --features hdf5           # HDF5 dynamically linked against a system install

[!Warning]

Known issue: --features hdf5 fails against Homebrew HDF5 2.x. The Rust binding (hdf5-metno-sys 0.10.1) only supports HDF5 1.8/1.10/1.12/1.14 and panics on a 2.x header with Invalid H5_VERSION: "2.1.1". Homebrew now ships 2.x, and even its hdf5@1.14 formula currently resolves to 2.1.1, so there is no Homebrew-based fix.

Prefer --features hdf5-static (shown above) instead: it builds HDF5 from source, so no system HDF5 install is needed at all, and it is what the release wheels use. On macOS it relies on a vendored patch to hdf5-metno-sys shipped in this repository (see vendor/hdf5-metno-sys-0.10.1-static-fix/PROVENANCE.md), so it works for builds from this repository but not for plain crates.io consumers of the evlib crate.

If you do need the dynamic --features hdf5 build, point HDF5_DIR at a genuine 1.8-1.14 install from another source, for example conda-forge:

conda install -c conda-forge "hdf5=1.14"
HDF5_DIR="$CONDA_PREFIX" maturin develop --features hdf5

Without either HDF5 feature, HDF5 files can still be read via h5py, or via the EVT2/EVT3 readers (which need no HDF5 feature). On Windows, neither HDF5 feature is available; HDF5 is always read through h5py there.

Distributable wheels are built with the opt-in extension-module feature, e.g. maturin build --release --features python,polars,extension-module,hdf5-static on macOS/Linux (Windows wheels omit hdf5-static, since HDF5 support is not available on Windows). The extension-module feature is deliberately off by default so cargo test and maturin develop build and run without linking errors.

GPU scatter-add kernels are opt-in features. For the CUDA backend, build the nvcc kernel and point EVLIB_CUDA_LIB at the resulting librvt_scatter.so. For the Metal backend on Apple Silicon, build with CC=clang maturin develop --features metal.

Since 0.13.1, the published macOS and Linux wheels include statically linked HDF5 support out of the box; source builds opt in with --features hdf5 or --features hdf5-static (see above). HDF5 support is unavailable on Windows in all cases; use h5py directly for HDF5 I/O there. Full details and platform-specific notes live in the installation guide.

Documentation

Complete documentation is published at https://tallamjr.github.io/evlib/:

• Quick Start
• Loading Data: formats, polarity encoding, streaming
• Event Representations
• Polars Preprocessing
• Performance Guide: benchmarks, memory monitoring, troubleshooting
• API Reference
• Platform Support

Examples

Runnable examples live in examples/:

python examples/simple_example.py
python examples/filtering_demo.py
python examples/stacked_histogram_demo.py

# Jupyter notebooks
pytest --nbmake examples/

Benchmarks live in benchmarks/: the Python suite (bench_rvt_dataset.py, bench_tonic.py) at the top level, and the Rust criterion benches under benchmarks/rust/.

Development

# Tests (both run directly, no special flags needed)
pytest                        # Python (test suite only)
cargo test                    # Rust
pytest --markdown-docs docs/  # doc examples (explicit)
pytest --nbmake examples/     # example notebooks (explicit)

# Formatting / linting
black python/ tests/ examples/
cargo fmt
ruff check python/ tests/
cargo clippy -- -D warnings

See CONTRIBUTING and the architecture overview for design details.

Community & Support

• Issues: Report bugs and request features

License

MIT License. See LICENSE.md for details.