panorai 3.0.0

Panoramic image projection and blending using Gnomonic and other spherical projections.

PanorAi: Spherical Image Processing & Projection

PanorAi lets you work with spherical (equirectangular) images and efficiently transform them into Gnomonic projections and back to equirectangular format. The framework offers flexible samplers and blenders that optimize projection and reconstruction processes.

---

Data Types

PanorAi organizes data into three main containers:

• EquirectangularImage – holds a full panorama and exposes methods such as to_gnomonic and to_gnomonic_face_set.
• GnomonicFace – represents a single rectilinear face with methods like to_equirectangular.
• GnomonicFaceSet – a collection of gnomonic faces that can be blended back into an equirectangular image.

Each container includes a convenient show() method that uses PIL to quickly preview the underlying image data.

DataFactory can create these objects from arrays, dictionaries or files, allowing the data type to drive the processing pipeline.

Transformation Flow

The main data containers can transform into each other using the built‑in projection helpers. The diagram below illustrates the typical direction of each conversion:

EquirectangularImage
    |     \-- to_gnomonic_face_set --> GnomonicFaceSet -- to_equirectangular -->
    |                                                ^
    |                                                |
    \-- to_gnomonic ----------> GnomonicFace -- to_equirectangular --/

Both GnomonicFace and GnomonicFaceSet can be retro‑projected back to an equirectangular panorama. This step often happens after image processing on the faces has been performed.

Attachable Components

Each container can attach three types of helpers that shape the projection workflow:

• Projector – performs the geometric transformation between the equirectangular panorama and a rectilinear face. The same projector is used when creating the face and when mapping it back.
• Sampler – chooses the tangent points on the sphere from which faces are extracted. Built‑in samplers like cube or fibonacci provide different coverage strategies.
• Blender – combines multiple retro‑projected faces into a single panorama, controlling how overlaps are weighted.

This design lets you project faces, perform image‑level processing on them (for instance with a neural network), and then retro‑project the results back onto the panorama using the attached projector and blender.

---

🚀 Quick Start

Installation

pip install panorai[depth]

1️⃣ Load an Equirectangular Image

Convert an image to an EquirectangularImage object.

from panorai.data import DataFactory

eq_image = DataFactory.from_file("path/to/image.png", data_type="equirectangular")

Other helpers load data from different sources:

eq_image = DataFactory.from_array(ndarray, data_type="equirectangular")
eq_image = DataFactory.from_dict(my_dict, data_type="equirectangular")
eq_image = DataFactory.from_pil(pil_image, data_type="equirectangular")
face_set = DataFactory.from_list(list_of_faces)  # attaches default blender

---

📌 Core Functions

2️⃣ Convert to Gnomonic Projection

Extract a rectilinear (Gnomonic) face from the equirectangular image.

face = eq_image.to_gnomonic(lat=45, lon=90, fov=60)
face.show()

3️⃣ Convert Back to Equirectangular

Reproject a gnomonic face back to equirectangular.

eq_reprojected = face.to_equirectangular(eq_shape=(512, 1024))
eq_reprojected.show()

4️⃣ Preprocess the Image

You can apply the same preprocessing operations directly on the container.

eq_image.preprocess(delta_lat=5.0, delta_lon=15.0, resize_factor=0.5)

---

🛠️ Advanced Usage

5️⃣ Convert to Multiple Gnomonic Faces

Use sampling strategies (e.g., "cube", "fibonacci") to extract multiple faces.

face_set = eq_image.to_gnomonic_face_set(fov=60, sampling_method="cube")
face_set[0].show()  # View first face

6️⃣ Reconstruct Using a Blender

Back-project multiple faces using different blending methods ("closest", "average").

eq_reconstructed = face_set.to_equirectangular(eq_shape=(512, 1024), blend_method="closest")
eq_reconstructed.show()

MultiChannelHandler

MultiChannelHandler helps when your data is stored in multiple channels (for example an RGB image plus a depth or mask channel). It can stack a dictionary of arrays into a single (H, W, C) array, apply a projection to all channels at once and then unstack the result back to the original layout.

from panorai.data.multi_handler import MultiChannelHandler
from panorai.projections.gnomonic_projection import GnomonicProjection
import numpy as np

data = {
    "rgb": rgb_array,      # shape (H, W, 3)
    "mask": mask_array     # shape (H, W, 1)
}

handler = MultiChannelHandler(data)
projector = GnomonicProjection(fov_deg=90)

# Project every channel together
handler.apply_projection(projector.project)

Customizing With Attachables

Each data type can attach processing components at runtime:

# Attach a sampler to control how multiple faces are sampled
eq_image.attach_sampler("fibonacci", n_points=8)

# Override the projection used by a gnomonic face
face.attach_projection("gnomonic", lat=30, lon=45, fov=75)

# Attach a blender to merge a set of faces
face_set.attach_blender("feathering")

Preprocessing Without Containers

Alternatively, if you want to operate on raw NumPy arrays, the Preprocessor.preprocess_eq performs NumPy-based preprocessing on a panorama. It can extend the vertical field of view, rotate by latitude and longitude offsets and optionally resize the image. Parameters may be supplied directly or via a PreprocessorConfig which stores defaults.

from panorai.preprocessing.preprocessor import Preprocessor
from panorai.preprocessing.config import PreprocessorConfig

# define preprocessing defaults
cfg = PreprocessorConfig(
    shadow_angle=10.0,
    delta_lat=5.0,
    delta_lon=15.0,
    resize_factor=0.5,
)

processed = Preprocessor.preprocess_eq(
    eq_image.data,
    shadow_angle=cfg.shadow_angle,
    delta_lat=cfg.delta_lat,
    delta_lon=cfg.delta_lon,
    resize_factor=cfg.resize_factor,
    config=cfg,
)

The shadow_angle parameter represents the portion of the panorama a 3D scanner misses near the bottom of the sphere. It is measured from the South Pole upward and padding this region ensures that subsequent projections cover any blind spots.

The returned array can be assigned back to the EquirectangularImage for further steps.

🔧 Configuring Samplers & Blenders

You can fine-tune sampling & blending strategies or modify the default projection configuration with ConfigManager.

Set Custom Sampler

from panorai.samplers.config import SamplerConfig

# Create a sampler configuration and attach it
custom_cfg = SamplerConfig(n_points=12, rotations=[(0, 45)])
eq_image.attach_sampler("fibonacci", config=custom_cfg)

Override the Default Projection

from panorai.config.config_manager import ConfigManager

# Update the global gnomonic config before attaching
cfg = ConfigManager.create("gnomonic_config", fov_deg=120, x_points=512, y_points=512)
eq_image.attach_projection("gnomonic", fov=cfg.fov_deg)

Select Blender

from panorai.blenders.registry import BlenderRegistry

blend = BlenderRegistry.create("gaussian", sig=1.2)
face_set.attach_blender("gaussian", sig=1.2)

Component Attachment & Configuration Flow

Data containers such as EquirectangularImage and GnomonicFace expose attach_sampler, attach_projection, and attach_blender helpers. These simply call PanoraiFactory which in turn pulls the requested object from the appropriate registry. The keyword arguments or configuration object you pass are forwarded directly to the constructor:

def attach_projection(self, name: str, lat: float = 0.0, lon: float = 0.0,
                      fov: float = 90.0, **kwargs):
    from panorai.factory.panorai_factory import PanoraiFactory
    self.projection = PanoraiFactory.get_projection(
        name, lat=lat, lon=lon, fov=fov, **kwargs
    )

PanoraiFactory performs minimal processing before delegating to the registry:

@classmethod
def get_projection(cls, name: str, lat: float, lon: float, fov: float, **kwargs):
    available = ProjectionRegistry.available_projections()
    kwargs["phi1_deg"] = lat
    kwargs["lam0_deg"] = lon
    kwargs["fov_deg"] = fov
    if name not in available:
        raise ProjectionNotFoundError(name, available)
    return ProjectionRegistry.create(name, **kwargs)

Every sampler, blender or projection can be built from a config object or direct keyword parameters. When both are supplied the config takes precedence, as seen in the sampler base class:

class Sampler(ABC):
    def __init__(self, config: Optional[SamplerConfig] = None, **kwargs: Any):
        if config is not None:
            self.config = config
        else:
            self.config = SamplerConfig(**kwargs)

This design lets you quickly attach components with simple parameters or manage shared settings via ConfigManager. All attachments ultimately flow through the factory, ensuring a consistent creation mechanism.

---

Factory Helpers (Advanced)

Use PanoraiFactory to load files or arrays and directly access registered components.

from panorai.factory.panorai_factory import PanoraiFactory
import numpy as np

# Load an equirectangular image
eq_img = PanoraiFactory.load_image("pano.jpg")

# Create a gnomonic face from a NumPy array
arr = np.zeros((256, 256, 3), dtype=np.uint8)
face = PanoraiFactory.create_data_from_array(arr, data_type="gnomonic_face",
                                             lat=0, lon=0, fov=90)

# Obtain a sampler or blender directly
sampler = PanoraiFactory.get_sampler("fibonacci", n_points=6)
blender = PanoraiFactory.get_blender("feathering")

📌 Summary

Feature	Function
Load Image	DataFactory.from_file()
Convert to Gnomonic	to_gnomonic(lat, lon, fov)
Convert to Face Set	to_gnomonic_face_set(fov, sampling_method)
Convert Back to EQ	to_equirectangular(eq_shape, blend_method)
Use Samplers & Blenders	ConfigManager, BlenderRegistry

---

Samplers

Samplers define how tangent points are chosen when generating face sets. The strategy affects coverage and the number of faces:

• cube – six orthogonal faces.
• icosahedron – vertices of an icosahedron; can be subdivided for density.
• fibonacci – nearly uniform distribution using the Fibonacci spiral.
• spiral – a simple spiral path around the sphere.
• blue_noise – random placement while keeping points apart.

eq_image.attach_sampler("cube")             # basic 6 faces
eq_image.attach_sampler("fibonacci", n_points=20)
faces = eq_image.to_gnomonic_face_set(fov=60)

Blenders

Blenders merge multiple faces back into a panorama. They control how overlaps are resolved:

• average – uniform averaging of pixels.
• feathering – smooth, distance-based weighting.
• gaussian – Gaussian weights projected onto the sphere.
• closest – choose the closest face for every pixel.
• huber – robust averaging that reduces outlier impact.

face_set.attach_blender("gaussian", sig=1.0)
result = face_set.to_equirectangular(eq_shape=(512, 1024))

Point Cloud Export

GnomonicFace and GnomonicFaceSet objects can be transformed into a PCD point cloud via their respective to_pcd() methods. The conversion is implemented in PCDHandler, which also provides convenience helpers such as create_axis_arrows() for quick Open3D visualisation or gradient masking functions used during conversion.

📚 Next Steps

• Experiment with different samplers ("cube", "fibonacci").
• Try blenders ("closest", "average") for optimal reconstructions.
• Use Torch tensors for deep learning integration.

🔗 PanorAi Documentation (Link to full API reference)

---

Running Tests

To run the tests execute:

pytest

The library uses a paths.yaml file to store paths to datasets and checkpoints. By default this file is expected in the project root, but you can override the location by setting the PANORAI_PATHS environment variable.

from panorai.path_config import get_path

ckpt_path = get_path("metric3d", "ckpt_file")

Building Documentation

To generate the HTML documentation run:

cd docs
make html

The output will be written to docs/_build/html/index.html.

Pre-commit Hook for Documentation

To automatically check for documentation issues before each commit, install pre-commit:

pip install pre-commit
pre-commit install

The hook runs sphinx-build -n -W to fail the commit if any warnings or broken references are found in the RST files.