Skip to main content
Help the Python Software Foundation raise $60,000 USD by December 31st!  Building the PSF Q4 Fundraiser

Tools for working with optical flow

Project description

flowpy 💾 - A python package for working with optical flows

Optical flow is the displacement map of pixels between two frames. It is a low-level analysis used in many computer vision programs.

Working with optical flow may be cumbersome:

  • It is quite hard to represent it in a comprehensible manner.
  • Multiple formats exist for storing it.

Flowpy provides tools to work with optical flow more easily in python.

Installing

We recommend using pip:

pip install flowpy

Features

The main features of flowpy are:

  • Reading and writing optical flows in two formats:
    • .flo (as defined here)
    • .png (as defined here)
  • Visualizing optical flows with matplotlib
  • Backward and forward warp

Examples

A simple RGB plot

This is the simplest example of how to use flowpy, it:

  • Reads a file using flowpy.flow_read.
  • Transforms the flow as an rgb image with flowpy.flow_to_rgb and shows it with matplotlib

Code:

import flowpy
import matplotlib.pyplot as plt

flow = flowpy.flow_read("tests/data/kitti_occ_000010_10.flo")

fig, ax = plt.subplots()
ax.imshow(flowpy.flow_to_rgb(flow))
plt.show()

Result:

simple_example

Sample image from the KITTI dataset

Plotting arrows, showing flow values and a calibration pattern

Flowpy comes with more than just RGB plots, the main features here are: - Arrows to quickly visualize the flow - The flow values below cursor showing in the tooltips - A calibration pattern side by side as a legend for your graph

Code:

flow = flowpy.flow_read("tests/data/Dimetrodon.flo")
height, width, _ = flow.shape

image_ratio = height / width
max_radius = flowpy.get_flow_max_radius(flow)

fig, (ax_1, ax_2) = plt.subplots(
    1, 2, gridspec_kw={"width_ratios": [1, image_ratio]}
)

ax_1.imshow(flowpy.flow_to_rgb(flow))
flowpy.attach_arrows(ax_1, flow)
flowpy.attach_coord(ax_1, flow)

flowpy.attach_calibration_pattern(ax_2, flow_max_radius=max_radius)

plt.show()

Result:

complex_example

Sample image from the Middlebury dataset

Warping images (backward):

If you know the flow (first_image -> second_image), you can backward warp the second_image back to first_image.

flow = flowpy.flow_read("static/kitti_occ_000010_10.png")
first_image = np.asarray(Image.open("static/kitti_000010_10.png"))
second_image = np.asarray(Image.open("static/kitti_000010_11.png"))

flow[np.isnan(flow)] = 0
warped_first_image = flowpy.backward_warp(second_image, flow)

fig, axes = plt.subplots(3, 1)
for ax, image, title in zip(axes, (first_image, second_image, warped_first_image),
                            ("First Image", "Second Image", "Second image warped to first image")):
    ax.imshow(image)
    ax.set_title(title)
    ax.set_axis_off()

plt.show()

Result:

backward_warp_example

Note that the artifacts in the warp are normal, they are caused by unknown flows and occlusions.

Warping images (forward):

Forward warp is often less used as it is quite more complex. It relies on a k-nearest neighbor search instead of direct bi-linear interpolation.

forward_warp is about 10x slower than backward_warp but you still may find it useful.

flow = flowpy.flow_read("static/kitti_occ_000010_10.png")
first_image = np.asarray(Image.open("static/kitti_000010_10.png"))
second_image = np.asarray(Image.open("static/kitti_000010_11.png"))

flow[np.isnan(flow)] = 0
warped_second_image = flowpy.forward_warp(first_image, flow)

fig, ax = plt.subplots()

ax.imshow(warped_second_image)
ax.set_title( "First image warped to the second")
ax.set_axis_off()

plt.show()

Result:

forward_warp_example

More

You can find the above examples in the examples folder. You can also look in tests. If you encounter a bug or have an idea for a new feature, feel free to open an issue.

Most of the visualization and io handling has been translated from matlab and c code from the Middlebury flow code. Credits to thank Simon Baker, Daniel Scharste, J. P. Lewis, Stefan Roth, Michael J. Black and Richard Szeliski.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for flowpy, version 0.4.2
Filename, size File type Python version Upload date Hashes
Filename, size flowpy-0.4.2-py3-none-any.whl (10.9 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size flowpy-0.4.2.tar.gz (9.8 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page