geozl 0.7.10

What is OpenZL and GeoZL?

OpenZL is a new compression framework that treats compression as a graph of codecs. Each frame carries the recipe needed to decode it, which lets a universal OpenZL decoder follow the graph without knowing how the data was originally encoded.

That model works well for one-dimensional streams, but it does not know that a raster has spatial structure. geozl adds that missing spatial layer.

A geozl codec is an OpenZL graph node that understands raster tiles. It transforms a typed numeric stream, stores the metadata needed to reverse that transform in the codec header, and lets the rest of the OpenZL graph continue as usual.

If you want to implement a new codec, see docs/adding-a-codec.md.

Status

geozl is experimental.

[!WARNING] geozl codecs are not part of OpenZL.

They are registered at runtime as OpenZL custom transforms and use CTids in the 0x72D700-0x72D7FF range. A frame that uses geozl codecs can only be decoded by a reader that has geozl registered. Frames that use only built-in OpenZL codecs remain portable OpenZL frames.

Install

pip install geozl

Example

geozl has two entry points: a high-level API that compresses a tile in one call, and a low-level API that places individual codecs in an OpenZL graph.

High-level API

Three calls. geozl.profile measures a set of candidate graph on your tile and ranks them; geozl.compress runs the one you name and returns the frame. It never searches, so the slow call happens once and the fast one happens on every tile after that. Finally, geozl.decompress reverses the frame back to a tile.

import numpy as np
import geozl

tile = np.random.randint(0, 4096, (1024, 1024), dtype=np.uint16)

rows = geozl.profile(tile)                        # the slow call, run once
best = rows[0]["graph"]                           # e.g. "planar>zigzag>transpose>entropy"

frame = geozl.compress(tile, method=best)         # the fast call, run always
frame = geozl.compress(tile, method=best, max_error=2)  # near-lossless, absolute bound

back = geozl.decompress(frame, dtype="uint16", width=1024)

Low-level API

For anything else, place the codecs in an openzl.ext graph yourself, alongside regular OpenZL nodes.

import openzl.ext as zl
import geozl

c = zl.Compressor()
g = zl.graphs.Compress()

g = zl.nodes.Zigzag()(c, g)
g = geozl.lossless.Planar(width=512)(c, g)

c.select_starting_graph(g)

Decoding

Either way, a reader has to register the geozl decoders before it can follow the frame.

import openzl.ext as zl
import geozl

d = zl.DCtx()
geozl.register_decoders(d)
tile = d.decompress(frame)[0].content.as_nparray()

Codecs

geozl currently provides two codec families:

• near-lossless codecs, under geozl.lossy
• lossless codecs, under geozl.lossless

Both families are registered as OpenZL custom codecs and can be chained with other OpenZL graph nodes.

The call column shows the Python call used to place the codec in a graph.

Near-lossless codecs

Near-lossless codecs quantize the tile once, then store enough information in the frame to report and bound the reconstruction error. A near-lossless frame is no longer bit-exact, it declares the bound it holds to instead.

A frame carries at most one near-lossless codec, as the head transform, so the loss happens once and every stage after it is lossless.

Lossless codecs

Lossless codecs are bit-exact transforms over a raster tile. After decoding, the reconstructed tile is identical to the original input.

Predictors replace each sample with its residual against a prediction from neighbours the decoder already holds. They take the row width, and one successor.

Splits cut one stream into two lanes, so they take two successors, one per lane, and each lane can go on to its own graph.

License

BSD-3-Clause

Made with ♥ by