geozl 0.7.5

What is OpenZL and GeoZL?

OpenZL is a new compression framework that treats compression as a graph of codecs. Each frame carries the recipe needed to decode it, which lets a universal OpenZL decoder follow the graph without knowing how the data was originally encoded.

That model works well for one-dimensional streams, but it does not know that a raster has rows, columns, neighbours, or spatial structure. geozl adds that missing spatial layer.

A geozl codec is an OpenZL graph node that understands raster tiles. It transforms a typed numeric stream, stores the metadata needed to reverse that transform in the codec header, and lets the rest of the OpenZL graph continue as usual.

If you want to implement a new codec, see docs/adding-a-codec.md.

Status

geozl is experimental.

[!WARNING] geozl codecs are not part of OpenZL.

They are registered at runtime as OpenZL custom transforms and use CTids in the 0x72D700-0x72D7FF range. A frame that uses geozl codecs can only be decoded by a reader that has geozl registered. Frames that use only built-in OpenZL codecs remain portable OpenZL frames.

Install

pip install geozl

Example

geozl has two entry points: a high-level API that compresses a tile in one call, and a low-level API that places individual codecs in an OpenZL graph.

High-level API

geozl.compress takes a tile and returns a frame. With the default method it sweeps every predictor and keeps the smallest one.

import numpy as np
import geozl

tile = np.random.randint(0, 4096, (1024, 1024), dtype=np.uint16)

frame = geozl.compress(tile)                 # sweep, keep the smallest frame
frame = geozl.compress(tile, method="med")   # or name one predictor
frame = geozl.compress(tile, max_error=2)    # near-lossless, absolute bound

Low-level API

For anything else, place the codecs in an openzl.ext graph yourself, alongside regular OpenZL nodes.

import openzl.ext as zl
import geozl

c = zl.Compressor()
g = zl.graphs.Compress()

g = zl.nodes.Zigzag()(c, g)
g = geozl.lossless.Planar(width=512)(c, g)

c.select_starting_graph(g)

Decoding

Either way, a reader has to register the geozl decoders before it can follow the frame.

import openzl.ext as zl
import geozl

d = zl.DCtx()
geozl.register_decoders(d)
tile = d.decompress(frame)[0].content.as_nparray()

Codecs

geozl currently provides two codec families:

• near-lossless codecs, under geozl.lossy
• lossless codecs, under geozl.lossless

Both families are registered as OpenZL custom codecs and can be chained with other OpenZL graph nodes.

The call column shows the Python call used to place the codec in a graph.

Near-lossless codecs

Near-lossless codecs quantize the tile once, then store enough information in the frame to report and bound the reconstruction error. A near-lossless frame is no longer bit-exact, it declares the bound it holds to instead.

A frame carries at most one near-lossless codec, as the head transform, so the loss happens once and every stage after it is lossless.

Lossless codecs

Lossless codecs are bit-exact transforms over a raster tile. After decoding, the reconstructed tile is identical to the original input.

Predictors replace each sample with its residual against a prediction from neighbours the decoder already holds. They take the row width, and one successor.

Splits cut one stream into two lanes, so they take two successors, one per lane, and each lane can go on to its own graph.

License

BSD-3-Clause

Made with ♥ by