raquet-io 0.4.0

RaQuet - Raster data in Parquet format with QUADBIN spatial indexing

RaQuet is a specification for storing and querying raster data using Apache Parquet, a column-oriented data file format. Users of data warehouse platforms rely on the simple interoperability of Parquet files to move data and perform queries.

Documentation | Online Viewer | Specification

Overview

Each row in a RaQuet file represents a single rectangular block of data. Location and zoom are given by a Web Mercator tile z/x/y tile identifier stored in the block column as a single 64-bit cell Quadbin identifier. Empty tiles can be omitted to reduce file size.

Raster data pixels are stored in row-major order binary packed blobs in per-band columns named band_1, band_2, etc. Valid pixel values include integers or floating point values. These blobs can be optionally compressed with gzip to further reduce file size.

Pixel bands can be decoded via simple binary unpacking in any programming environment and converted to wire image formats like PNG or displayed directly in web visualization libraries like MapLibre.

Similar to GeoParquet, RaQuet metadata is stored as a JSON object with details on coverage area, raster resolution, pixel data format, and other needed information. For compatibility with data warehouses the metadata is stored within a Parquet row at a special “0” row (block=0x00).

Installation

# Basic installation (GeoTIFF conversion)
pip install raquet-io

# With rich output for CLI
pip install "raquet-io[rich]"

# With ImageServer support
pip install "raquet-io[imageserver]"

# All features
pip install "raquet-io[all]"

Note: GDAL must be installed separately. On macOS: brew install gdal. On Ubuntu: apt install gdal-bin libgdal-dev.

CLI Usage

The raquet CLI provides commands for inspecting, converting, and exporting Raquet files.

Inspect a Raquet file

# Display metadata and statistics
raquet-io inspect landcover.parquet

# With verbose output
raquet-io inspect landcover.parquet -v

Convert to Raquet

From GeoTIFF

# Basic conversion
raquet-io convert geotiff input.tif output.parquet

# With custom options
raquet-io convert geotiff input.tif output.parquet \
  --resampling bilinear \
  --block-size 512 \
  -v

Options:

Option	Description
--zoom-strategy	Zoom level strategy: auto, lower, upper (default: auto)
--resampling	Resampling algorithm: near, bilinear, cubic, etc. (default: near)
--block-size	Block size in pixels (default: 256)
--target-size	Target size for auto zoom calculation
-v, --verbose	Enable verbose output

From ArcGIS ImageServer

# Basic conversion
raquet-io convert imageserver https://server/arcgis/rest/services/dem/ImageServer dem.parquet

# With bounding box filter
raquet-io convert imageserver https://server/.../ImageServer output.parquet \
  --bbox "-122.5,37.5,-122.0,38.0"

# With specific resolution
raquet-io convert imageserver https://server/.../ImageServer output.parquet \
  --resolution 12 \
  -v

Options:

Option	Description
--token	ArcGIS authentication token
--bbox	Bounding box filter in WGS84: xmin,ymin,xmax,ymax
--block-size	Block size in pixels (default: 256)
--resolution	Target QUADBIN pixel resolution (auto if not specified)
--no-compression	Disable gzip compression for block data
-v, --verbose	Enable verbose output

Export from Raquet

To GeoTIFF

# Export to GeoTIFF
raquet-io export geotiff input.parquet output.tif

# With verbose output
raquet-io export geotiff input.parquet output.tif -v

Legacy Commands

For backwards compatibility, standalone commands are also available:

geotiff2raquet input.tif output.parquet
raquet2geotiff input.parquet output.tif

Python API

from raquet import geotiff2raquet, raquet2geotiff

# Convert GeoTIFF to Raquet
geotiff2raquet.main(
    "input.tif",
    "output.parquet",
    geotiff2raquet.ZoomStrategy.AUTO,
    geotiff2raquet.ResamplingAlgorithm.NEAR,
    block_zoom=8,  # 256px blocks
    target_size=None,
)

# Convert Raquet to GeoTIFF
raquet2geotiff.main("input.parquet", "output.tif")

ImageServer Conversion

from raquet.imageserver import imageserver_to_raquet

# Convert ImageServer to Raquet
result = imageserver_to_raquet(
    "https://server/arcgis/rest/services/dem/ImageServer",
    "output.parquet",
    bbox=(-122.5, 37.5, -122.0, 38.0),  # Optional WGS84 bounds
    block_size=256,
    target_resolution=12,  # Optional, auto-calculated if not specified
)

print(f"Created {result['num_blocks']} blocks with {result['num_bands']} bands")

Querying with DuckDB

Raquet files can be queried directly with DuckDB:

-- Load Raquet file
SELECT * FROM read_parquet('raster.parquet') WHERE block != 0 LIMIT 10;

-- Get metadata
SELECT metadata FROM read_parquet('raster.parquet') WHERE block = 0;

-- Query specific tiles using QUADBIN
SELECT block, band_1
FROM read_parquet('raster.parquet')
WHERE block = quadbin_from_tile(x, y, z);

Online Viewer

Try the RaQuet Viewer - a client-side viewer powered by DuckDB-WASM that runs entirely in your browser. Load any publicly accessible RaQuet file and explore it interactively.

Performance Tips

For optimal remote query performance:

1. Block sorting: Blocks are automatically sorted by QUADBIN ID during conversion, enabling Parquet row group pruning
2. Row group size: Use smaller row groups (default: 200) for cloud storage access
3. Zoom splitting: For large datasets, use raquet-io split-zoom to create per-zoom-level files

# Convert with optimized settings for remote access
raquet-io convert geotiff input.tif output.parquet --row-group-size 200

See the full documentation for more details.

Specification

See format-specs/raquet.md for the full specification.

Examples

See examples/example_metadata.json for an example of the metadata.

See examples/example_data.parquet for an example of the data.

License

See LICENSE for the license.