geoparquet-lite

Read, write and validate GeoParquet 1.0 and 1.1 with nothing but pyarrow.

No geopandas, shapely, GDAL or PROJ is required. That makes it a good fit for data pipelines, serverless functions, DuckDB and Spark interop, and small containers where a full geospatial stack is too heavy. geopandas and shapely remain available as optional extras for interop.

pip install geoparquet-lite                 # pyarrow only
pip install "geoparquet-lite[geopandas]"    # + GeoDataFrame bridges

Features

• Metadata: read, write and validate the geo key of GeoParquet 1.0.0 and 1.1.0. The validator applies every constraint of the official JSON schemas plus the rules that only the spec text states. It needs no jsonschema dependency.
• Encodings: WKB, and the 1.1 native GeoArrow encodings point, linestring, polygon, multipoint, multilinestring and multipolygon (separated struct/list layout). Columns can be converted in both directions.
• Pure-Python WKB codec: 2D, Z, M and ZM, all seven OGC types, both byte orders, and ISO WKB as well as PostGIS EWKB input. It converts between WKB, GeoJSON-like dicts and GeoArrow arrays.
• Spatial filtering: writes a 1.1 covering.bbox column. On read, row groups are pruned using the Parquet min/max statistics of that column, or of the x/y leaves of a native encoding. The remaining rows are then filtered exactly by bounding box, or by true intersection when shapely is installed. Query boxes that cross the antimeridian are supported.
• Hilbert sort on write, so row groups are spatially compact and pruning skips more of the file.
• CRS as PROJJSON, following the spec's defaults: no key means OGC:CRS84 and null means unknown. PROJJSON for OGC:CRS84, EPSG:4326, EPSG:3857, EPSG:4979 and EPSG:4258 is bundled, and pyproj is used for other codes if it is installed.
• edges (planar/spherical), orientation (counterclockwise, with ring rewinding), epoch, geometry_types and the column bbox.
• Column projection, batch iteration and fsspec/pyarrow filesystems.
• A validate(path) report with clear messages, and info(path).
• A CLI: geoparquet-lite info | validate | to-geojson | from-geojson | add-bbox | convert.

Quick start

import geoparquet_lite as gpq

fc = {
    "type": "FeatureCollection",
    "features": [
        {"type": "Feature", "geometry": {"type": "Point", "coordinates": [29.0, 41.0]},
         "properties": {"name": "Istanbul"}},
        {"type": "Feature", "geometry": {"type": "Point", "coordinates": [32.85, 39.93]},
         "properties": {"name": "Ankara"}},
    ],
}

# GeoParquet 1.1, WKB, bbox covering column, zstd. Returns the written `geo` metadata.
gpq.write(fc, "cities.parquet", crs="EPSG:4326", sort_hilbert=True, row_group_size=50_000)

# Native GeoArrow encoding instead of WKB:
gpq.write(fc, "cities_native.parquet", encoding="geoarrow")

# Read only what intersects a box. Row groups outside it are never read.
table = gpq.read_table("cities.parquet", bbox=(28, 40, 30, 42), columns=["name", "geometry"])

# Stream batches, or GeoJSON features:
for batch in gpq.iter_batches("cities.parquet", batch_size=10_000):
    ...
for feature in gpq.read_features("cities.parquet", bbox=(28, 40, 30, 42)):
    print(feature["properties"]["name"], feature["geometry"])

report = gpq.validate("cities.parquet")
print(report)            # "cities.parquet: valid GeoParquet (0 error(s), 0 warning(s), ...)"
assert report.ok

print(gpq.info("cities.parquet"))          # rows, row groups, encodings, CRS, row-group bboxes

Converting and upgrading files

gpq.convert("in.parquet", "native.parquet", encoding="geoarrow")   # WKB -> native
gpq.convert("native.parquet", "wkb.parquet", encoding="WKB")       # native -> WKB
gpq.add_bbox_covering("v1_0.parquet", "v1_1.parquet", sort_hilbert=True)

table = gpq.read_table("in.parquet", to_encoding="geoarrow")       # convert while reading
table = gpq.convert_table(table, "WKB")
gpq.write(table, "out.parquet")       # tables keep their CRS / edges / orientation metadata

WKB and GeoArrow helpers

from geoparquet_lite import wkb, geoarrow

blob = wkb.dumps({"type": "Polygon", "coordinates": [[[0, 0], [1, 0], [1, 1], [0, 0]]]})
wkb.loads(blob)                         # -> GeoJSON-like dict
wkb.read_header(blob)                   # WKBHeader(Polygon XY, ISO, little)

# XYM has no GeoJSON form, so a "dims" key marks it:
wkb.dumps({"type": "Point", "coordinates": [1, 2, 7], "dims": "XYM"})

arr = geoarrow.from_geometries([...])           # narrowest native encoding that fits
geoarrow.to_geometries(arr, "polygon")
geoarrow.to_wkb(arr, "polygon"); geoarrow.from_wkb(wkb_array, "multipolygon")

geopandas / shapely (optional)

from geoparquet_lite import interop

gdf = interop.to_geopandas(gpq.read_table("cities.parquet", bbox=(28, 40, 30, 42)))
interop.from_geopandas(gdf, "out.parquet", encoding="geoarrow")
geoms = interop.to_shapely(table.column("geometry"), "WKB")

CLI

geoparquet-lite info data.parquet [--json]
geoparquet-lite validate data.parquet [--metadata-only] [--max-rows N] [--strict] [--json]
geoparquet-lite to-geojson data.parquet [-o out.geojson] [--bbox xmin,ymin,xmax,ymax] [--columns a,b] [--seq]
geoparquet-lite from-geojson in.geojson out.parquet [--encoding WKB|geoarrow|polygon|...]
        [--crs EPSG:4326|unknown|crs.json] [--edges spherical] [--orientation]
        [--geoparquet-version 1.0.0|1.1.0] [--no-bbox] [--sort] [--row-group-size N]
geoparquet-lite add-bbox in.parquet out.parquet [--name bbox] [--sort]
geoparquet-lite convert in.parquet out.parquet --encoding geoarrow

validate exits with status 1 when the file has errors, or with --strict when it has warnings. Any other failure exits with status 2. from-geojson also reads newline-delimited GeoJSON and RFC 8142 text sequences, and reads stdin when the input is -.

Spec coverage

validate() also checks the following:

• the physical Arrow type of each geometry column matches its declared encoding (and interleaved layouts are reported);
• every geometry decodes;
• each geometry lies inside the column bbox and inside its covering bbox;
• the covering column has min/max statistics.

How the validator compares with the official JSON schemas. It gives the same verdict as the official schema.json for v1.0.0 and v1.1.0 (with the PROJJSON v0.7 schema) on 106 metadata objects. Those objects come from files written by this package, geopandas 1.2, GDAL 3.13 and DuckDB 2.0. It is stricter in two places:

• A native encoding must agree with geometry_types. The specification text requires this, but a JSON schema cannot express it.
• A crs object must be a CRS. The PROJJSON schema would also accept, for example, a bare ellipsoid.

The official schema files are not bundled.

Interoperability (tested)

• geopandas 1.2 / pyarrow 26. geopandas.read_parquet reads the files we write, in both WKB and native encodings, including with a bbox= filter and with the CRS kept. We read files from GeoDataFrame.to_parquet in both WKB and geometry_encoding="geoarrow", with or without write_covering_bbox, and they pass validate().
• GDAL 3.13 Parquet driver. ogrinfo reads our WKB and native files: feature count, geometry type, extent, CRS and -spat filtering all work. Files from GDAL's ogr2ogr -f Parquet, in WKB and in GEOMETRY_ENCODING=GEOARROW, read and validate here.
• DuckDB 2.0 spatial. DuckDB reads our WKB files as GEOMETRY with the CRS, and ST_Intersects and the bbox covering struct work in SQL. GeoParquet written by DuckDB COPY ... TO reads and validates here. DuckDB 2.0 exposes columns with a native encoding as their raw STRUCT/LIST type rather than as GEOMETRY. Use WKB when DuckDB is the consumer.

Limitations

• Geometry work is plain Python, roughly 30–70k simple features per second for writing and validation. Reading and bbox filtering use pyarrow compute and are fast. This package is not a replacement for GDAL in bulk ETL.
• predicate="bbox" (the default) keeps rows whose geometry bounding box intersects the query. For exact geometric intersection, pass predicate="intersects", which requires shapely.
• Bounding boxes are computed on the raw coordinates. For edges="spherical", the box of an edge that bends poleward of its vertices is not widened.
• Covering columns are written in 2D only (no zmin/zmax).
• Parquet's native GEOMETRY/GEOGRAPHY logical types (GeoParquet 2.0 / Parquet 2.11) are not handled.

Development

pip install -e ".[dev,geopandas]"
pytest --cov=geoparquet_lite
ruff check . && ruff format --check src tests && mypy src

The GDAL interop tests run when ogrinfo is on PATH, or when GEOPARQUET_LITE_OGRINFO points to it. The DuckDB tests run when duckdb is installed.

Data credits

The bundled PROJJSON definitions in geoparquet_lite/data/projjson.json were exported with PROJ from the EPSG Geodetic Parameter Dataset (https://epsg.org), © IOGP, and from the OGC CRS84 definition.

License

MIT © Yusuf Eminoğlu