Skip to main content

Bird-eye's view for CARLA simulator

Project description

Bird eye's view for CARLA

Freeway-oriented shape Learning By Cheating-like shape
cruising square-cruising

(GIFs above present feature maps after applying as_rgb() function)

Motivation

During our research we found a very inspiring paper called Learning By Cheating. Bird-eye's view is made specifically to learn faster thanks to much simpler, 2D world representation (cheating oracle) which we think fits well in Reinforcement Learning setup.

This repository is an almost complete reimplementation that gives better performance and compatibility with most recent versions of CARLA. You can use it out-of-the-box as input for your model, and if necessary convert and visualize into RGB.

Features

  • one-hot 3D feature map (8x2D layers, each representing other entities, e.g. road layer, pedestrians layer) - made specifically to feed your CNN
  • feature map can be converted to an RGB image
  • layers can be easily removed
  • caching mechanism for static layers like: roads and lanes
  • using OpenCV rendering (efficient, multi-threading friendly) instead of slow Pygame method
  • huge FPS speedup thanks to restricted rendering (only agent's surroundings, not whole map)
  • all CARLA maps are supported out-of-the-box, custom maps with valid OpenDrive file made in RoadRunner are also supported
  • current implementation is specifically adjusted for highway scenarios (prolonged shape), but other shapes and crops are easy to implement

Installation

pip install carla-birdeye-view

How to run

Make sure that PYTHONPATH env variable contains CARLA distribution egg, so that carla package can be imported.

# Launch server instance
./CarlaUE4.sh

# (optional) For CARLA 0.9.8+ you may get additional performance improvement with this
python PythonAPI/util/config.py --no-rendering

# Preview while cruising on autopilot (birdview/__main__.py)
birdview-demo

Development

# From repo root
python -m birdview

Contribution and feedback

We'd :heart: to collct any feedback, issues and pull requests!

Credits

Project born at deepsense.ai, made by:

Michał Martyniak (@micmarty)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for carla-birdeye-view, version 1.0.1
Filename, size File type Python version Upload date Hashes
Filename, size carla_birdeye_view-1.0.1.tar.gz (3.3 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page