geobuf 1.1.0

Geobuf is a compact binary geospatial format for lossless compression of GeoJSON and TopoJSON data.

Geobuf is a compact binary geospatial format for lossless compression of GeoJSON and TopoJSON data.

Advantages over using GeoJSON and TopoJSON directly (in this revised version):

• Very compact: typically makes GeoJSON 6-8 times smaller and TopoJSON 2-3 times smaller.

• Smaller even when comparing gzipped sizes: 2-2.5x compression for GeoJSON and 20-30% for TopoJSON.

• Easy incremental parsing — you can get features out as you read them, without the need to build in-memory representation of the whole data.

• Partial reads — you can read only the parts you actually need, skipping the rest.

• Trivial concatenation: you can concatenate many Geobuf files together and they will form a valid combined Geobuf file.

• Potentially faster encoding/decoding compared to native JSON implementations (i.e. in Web browsers).

• Can still accommodate any GeoJSON and TopoJSON data, including extensions with arbitrary properties.

Think of this as an attempt to design a simple, modern Shapefile successor that works seamlessly with GeoJSON and TopoJSON.

Unlike Mapbox Vector Tiles, it aims for lossless compression of datasets — without tiling, projecting coordinates, flattening geometries or stripping properties.

pygeobuf

This repository is the first encoding/decoding implementation of this new major version of Geobuf (in Python). It serves as a prototyping playground, with faster implementations in JS and C++ coming in future.

Sample compression sizes

| normal    | gzipped

us-zips.json	101.85 MB	26.67 MB
us-zips.pbf	12.24 MB	10.48 MB
us-zips.topo.json	15.02 MB	3.19 MB
us-zips.topo.pbf	4.85 MB	2.72 MB
idaho.json	10.92 MB	2.57 MB
idaho.pbf	1.37 MB	1.17 MB
idaho.topo.json	1.9 MB	612 KB
idaho.topo.pbf	567 KB	479 KB

Usage

Command line:

geobuf encode < example.json > example.pbf
geobuf decode < example.pbf > example.pbf.json

As a module:

import geobuf

pbf = geobuf.encode(my_json) # GeoJSON or TopoJSON -> Geobuf string
my_json = geobuf.decode(pbf) # Geobuf string -> GeoJSON or TopoJSON

The encode function accepts a dict-like object, for example the result of json.loads(json_str).

Both encode.py and geobuf.encode accept two optional arguments:

• precision — max number of digits after the decimal point in coordinates, 6 by default.

• dimensions — number of dimensions in coordinates, 2 by default.

Tests

py.test -v

The tests run through all .json files in the fixtures directory, comparing each original GeoJSON with an encoded/decoded one.