Skip to main content

Bucketed Scene Flow Evaluation

Project description

Bucketed Scene Flow Evaluation

This repo provides:

  • A speed and class aware evaluation protocol called Bucketed Scene Flow Evaluation
  • A standardized interface for working with Scene Flow datasets.

Currently supported datasets:

  • Argoverse 2 (Human Labeled and NSFP Pseudolabeled)
  • Waymo Open

The Bucketed Scene Flow Evaluation protocol

The Bucketed Scene Flow Evaluation protocol is designed to quantitatively measure the failure of state-of-the-art scene flow methods to properly capture motion on smaller objects. In the Autononous Vehicle domain, SotA methods almost universially fail on important objects like Pedestrians and Bicyclists. As part of organizing the community around addressing these issues, the Argoverse 2 2024 Scene Flow Challenge uses this protocol to evaluate submissions. More details about the protocol can be found in the challenge blogpost.

Installation

pip install bucketed-scene-flow-eval

Setup

Follow our Getting Started for setup instructions.

Demo

We provide a demo script which shows off the various features of the API.

To run with Argoverse 2:

python scripts/demo.py --dataset Argoverse2SceneFlow --root_dir /efs/argoverse2/val/

To run with Waymo Open:

python scripts/demo.py --dataset WaymoOpenSceneFlow --root_dir /efs/waymo_open_processed_flow/validation/

Evaluating AV2 flow submissions

To evaluate an AV2 Scene Flow challenge entry named ./submission_val.zip against validation dataset masks /efs/argoverse2/val_official_masks.zip, run

python scripts/av2_eval.py /efs/argoverse2/val /efs/argoverse2/val_official_masks.zip ./submission_val.zip

Documentation

See docs/ for more documentation .

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bucketed_scene_flow_eval-1.0.2.tar.gz (36.4 kB view hashes)

Uploaded Source

Built Distribution

bucketed_scene_flow_eval-1.0.2-py3-none-any.whl (47.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page