pyequilib 0.2.0

equirectangular image processing with python

equilib

Processing Equirectangular Images with Python

• A library for processing equirectangular image that runs on Python.
• Developed using numpy, pytorch, and c++.
• Able to use GPU for faster processing.
• No need for other dependencies except for numpy and pytorch.
• Added functionality like creating rotation matrices, batched processing, and automatic type detection.
• Highly modular

Installation:

Prerequisites:

• Python (>=3.6)
• Pytorch

pip install pyequilib

For developing, use:

git clone --recursive https://github.com/haruishi43/equilib.git
cd equilib

pip install -r requirements.txt

python setup.py develop

Basic Usage:

equilib has different transforms of equirectangular (or cubemap) images (note each transform has class and func APIs):

• Cube2Equi/cube2equi: cubemap to equirectangular transform
• Equi2Cube/equi2cube: equirectangular to cubemap transform
• Equi2Equi/equi2equi: equirectangular transform
• Equi2Pers/equi2pers: equirectangular to perspective transform

There are no real differences in class or func APIs:

• class APIs will allow instantiating a class which you can call many times without having to specify configurations (class APIs call the func API)
• func APIs are useful when there are no repetitive calls
• both class and func APIs are extensible, so you can extend them to your use-cases or create a method that's more optimized (pull requests are welcome btw)

Each API automatically detects the input type (numpy.ndarray or torch.Tensor), and outputs are the same type.

An example for Equi2Pers/equi2pers:

`Equi2Pers`

`equi2pers`

import numpy as np from PIL import Image from equilib import Equi2Pers # Input equirectangular image equi_img = Image.open("./some_image.jpg") equi_img = np.asarray(equi_img) equi_img = np.transpose(equi_img, (2, 0, 1)) # rotations rot = { 'roll': 0., 'pitch': np.pi/4, # rotate vertical 'yaw': np.pi/4, # rotate horizontal } # Intialize equi2pers equi2pers = Equi2Pers( w_pers=640, h_pers=480, fov_x=90, skew=0.0, sampling_method="default", mode="bilinear", ) # obtain perspective image pers_img = equi2pers( equi=equi_img, rot=rot, )

import numpy as np from PIL import Image from equilib import equi2pers # Input equirectangular image equi_img = Image.open("./some_image.jpg") equi_img = np.asarray(equi_img) equi_img = np.transpose(equi_img, (2, 0, 1)) # rotations rot = { 'roll': 0., 'pitch': np.pi/4, # rotate vertical 'yaw': np.pi/4, # rotate horizontal } # Intialize equi2pers pers_img = equi2pers( equi=equi_img, rot=rot, w_pers=640, h_pers=480, fov_x=90, skew=0.0, sampling_method="default", mode="bilinear", )

For more information about how each APIs work, take a look in .readme or go through example codes in the tests or demo.

Coordinate System:

Right-handed rule XYZ global coordinate system. x-axis faces forward and z-axis faces up.

• roll: counter-clockwise rotation about the x-axis
• pitch: counter-clockwise rotation about the y-axis
• yaw: counter-clockwise rotation about the z-axis

See demo scripts under scripts.

Grid Sampling

To process equirectangular images fast, whether to crop perspective images from the equirectangular image, the library takes advantage of grid sampling techniques. Some sampling techniques are already implemented, such as scipy.ndimage.map_coordiantes and cv2.remap. This project's goal was to reduce these dependencies and use cuda and batch processing with pytorch and c++ for a faster processing of equirectangular images. There were not many projects online for these purposes. In this library, we implement varieties of methods using c++, numpy, and pytorch. This part of the code needs cuda acceleration because grid sampling is parallelizable. For c++ and pytorch, I tried to take advantage of cuda. For numpy, I implemented naive and faster approaches for learning purposes. Developing faster c++ and pytorch approaches are WIP. Currently, sampling_method defaults to the fastest methods which are named "default". See here for more info on implementations.

Develop:

Test files for equilib are included under tests.

Running tests:

pytest tests

Check CONTRIBUTING.md for more information

TODO:

• Documentations for each transform
• Add graphs and statistics for speed improvements

Acknowledgements:

• py360convert
• Perspective-and-Equirectangular