geozl 0.8.0

What is OpenZL and GeoZL?

OpenZL is a new compression framework that treats compression as a graph of codecs. Each frame carries the recipe needed to decode it, which lets a universal OpenZL decoder follow the graph without knowing how the data was originally encoded.

That model works well for one-dimensional streams, but it does not know that a raster has spatial structure. GeoZL adds that missing spatial layer.

A GeoZL codec is an OpenZL graph node that understands raster tiles. It transforms a typed numeric stream, stores the metadata needed to reverse that transform in the codec header, and lets the rest of the OpenZL graph continue as usual.

If you want to implement a new codec, see docs/adding-a-codec.md.

Status

GeoZL is experimental.

[!WARNING] GeoZL codecs are not part of OpenZL.

They are registered at runtime as OpenZL custom transforms and use CTids in the 0x72D700-0x72D7FF range. A frame that uses GeoZL codecs can only be decoded by a reader that has GeoZL registered. Frames that use only built-in OpenZL codecs remain portable OpenZL frames.

Install

pip install geozl

Example

GeoZL has two entry points: a high-level API that compresses a tile in one call, and a low-level API that places individual codecs in an OpenZL graph.

High-level API

geozl.profile measures a set of candidate graphs on your tile and ranks them, geozl.compress runs the one you name and returns the frame. It never searches, so the slow call happens once and the fast one happens on every tile after that. geozl.decompress reverses the frame back to a tile.

import numpy as np
import geozl

tile = np.random.randint(0, 4096, (1024, 1024), dtype=np.uint16)

rows = geozl.profile(tile)                        # the slow call, run once
best = rows[0]["graph"]                           # e.g. "planar>zigzag>transpose>entropy"

frame = geozl.compress(tile, method=best)         # the fast call, run always
frame = geozl.compress(tile, method=best, error="LINEAR:MAX_ERROR=2")  # near-lossless
frame = geozl.compress(tile, method=best, error="LOG:MAX_ERROR=1%")   # bound follows the value

back = geozl.decompress(frame, dtype="uint16", width=1024)

Low-level API

For anything else, place the codecs in an openzl.ext graph yourself, alongside regular OpenZL nodes.

import openzl.ext as zl
import geozl

c = zl.Compressor()
g = zl.graphs.Compress()

g = zl.nodes.Zigzag()(c, g)
g = geozl.lossless.Planar(width=512)(c, g)

c.select_starting_graph(g)

Decoding

Either way, a reader has to register the geozl decoders before it can follow the frame.

import openzl.ext as zl
import geozl

d = zl.DCtx()
geozl.register_decoders(d)
tile = d.decompress(frame)[0].content.as_nparray()

Codecs

SQRT counts sigmas of the sensor curve a + b*x. Left out of the recipe, that curve is fitted from whatever raster is being compressed, so neighbouring tiles land on different grids. Measure it once over the product instead.

noise = geozl.lossy.fit_noise(stack)   # (N, H, W), or any sequence of rasters
frame = geozl.compress(tile, method=best, error=noise.recipe(0.5))

License

BSD-3-Clause

Made with ♥ by