pygndc

Geographic Neural Data Cube — a Python SDK for reading and analyzing .gndc compressed geospatial time-series data.

What is GeoNDC?

GeoNDC is a continuous-time, AI-ready representation of Earth observation archives. Unlike traditional Analysis-Ready Data (cloud-corrected raster files) or geospatial foundation model embeddings (abstract feature vectors), GeoNDC preserves the original physical observables — surface reflectance, vegetation indices, biophysical variables — while enabling millisecond-level random-access queries at any (x, y, t) coordinate.

Each archive (MODIS, Sentinel-2, Landsat, HiGLASS, …) is encoded into a single self-contained .gndc file (typically 0.5–2 GB) that runs on a laptop, a server, or directly in a browser via WebGPU. Data providers train the model once and publish the file; users download it and run inference locally — the compressed form is the analysis-ready form. No hosted runtime, no API quota, no vendor lock-in.

Key Capabilities

• Continuous-time reconstruction — query data at any moment, not just original observation times
• Millisecond random access — point time series in ~7 ms, full-frame reconstruction in ~2 s on a consumer GPU
• Analytic gradients — compute spatial/temporal derivatives directly from the neural network
• Compact storage — typically ~100:1 versus Int16 raster baselines, up to ~400:1 versus raw float archives
• Portable Native encoder and decoder — the default CPU/WGPU path needs no PyTorch, Numba, CUDA toolkit, or runtime JIT; WGPU targets Vulkan, DX12, and Metal, with an optional PyTorch/tiny-cuda-nn integration
• Implicit gap-filling — cloud-occluded surfaces are reconstructed from the learned spatiotemporal field
• Multi-sensor support — Sentinel-2, Landsat, MODIS, HiGLASS, and more

Online Viewer & Sample Data

• Web Viewer: Browse .gndc files directly in the browser via WebGPU at geondc.org/viewer — no installation required, GPU-accelerated, runs entirely client-side.
• Sample Data: Download .gndc datasets from Hugging Face.

Querying a Collection

Many independently encoded geographic products can be exposed as one logical cube without repacking their neural payloads:

from pygndc import GNDCCollection

collection = GNDCCollection(mode="native_wgpu")
collection.add(r"Y:\HiGLASS\*.gndc")
collection.save("higlass_china.gndc-collection.json")

collection = GNDCCollection.load("higlass_china.gndc-collection.json")
values = collection.query(
    points=[(116.4, 39.9), (118.2, 36.5)],
    time="2023-07-15",
    bands=[0, 1],
)

The collection index loads without opening its members. Queries are grouped by member and decoded through the existing GNDCDataset runtime with a bounded LRU cache. Spatiotemporal windows support in-memory results, stream=True, or direct output to a NumPy .npy file.

Documentation

• TUTORIAL.md — Installation, quick-start, CLI commands, end-to-end usage examples.
• API_Reference.md — Full Python API for pygndc.open(), GNDCDataset, GNDCReader, analysis functions.

Installation Extras

Prebuilt packages support Python 3.10–3.13 on Windows x86-64 and Linux x86-64 (glibc 2.28 or newer). They include the encoder, CPU/GPU decoder, GeoTIFF I/O, and Python API. GPU acceleration requires a compatible graphics driver; CPU decoding is available without a GPU. The desktop viewer is optional:

pip install pygndc[viewer]

The desktop viewer caches up to nine chunk readers by default, enough for a 3×3 HiGLASS product, and keeps up to nine model slots resident on WGPU. Set PYGNDC_VIEWER_MAX_CACHED_CHUNKS to tune the host-side LRU and PYGNDC_VIEWER_WGPU_SLOTS to tune device-resident model slots, or change the reader limit at runtime from Tools > Chunk Cache.

Encoding large datasets

GPU training reuses data already loaded into device memory across training rounds. Set train.train_dtype: float16 to store training targets in half precision, or float32 to retain full-precision targets. Model calculations use float32 in either case. The default training-data budget is 4096 MB; set PYGNDC_NATIVE_PERSISTENT_MAX_MB to adjust it for your GPU.

For tiled inputs, the encoder can read the next tile while training the current one. Set tiled.prefetch_chunks: false when available RAM is limited.

Reading large archives

Masks and frame-indexed residual corrections are loaded for requested dates. Repeated queries to the same date reuse recently loaded data. Existing .gndc files can use these improvements without being re-encoded. Older residual formats without a date index continue to use their existing decoding behavior.

GPU loading transfers supported quantized weights directly and restores them on the device. This reduces work during model loading; it does not reduce the memory required by the loaded float32 model. CPU decoding remains available when GPU loading is unavailable.

For applications that process many chunks in sequence, GNDCDataset provides optional prefetch budgets. See the API reference for the parameters and memory limits. Ordinary read() calls do not enable background prefetch automatically.

Changes in 1.0.14

Encoding and decoding APIs now include type declarations for editor completion, parameter hints, and static type checking. Implementation source code is not included in the prebuilt packages.

• Faster encoder startup by reusing hardware identification within a process.
• More efficient reuse of GPU training data across epochs.
• Lower memory use for queries with frame-indexed residual corrections.
• Faster loading of supported quantized models and repeated queries to a date.
• Clear errors when a file declares residual corrections that cannot be read.

Existing file formats remain supported. See release notes.

License

GeoNDC is proprietary binary software. Reading and decoding do not require an encoder license. Creating .gndc files requires a valid encoder license. Contact jianboqi@126.com for evaluation or commercial use. See LICENSE for the complete terms.

The GeoNDC file-format specification is published separately to support interoperability. Previously released copies remain governed by the license that accompanied those copies.

Citation

@misc{qi2026geondcqueryableneuraldata,
  title={GeoNDC: A Queryable Neural Data Cube for Planetary-Scale Earth Observation},
  author={Jianbo Qi and Mengyao Li and Baogui Jiang and Yidan Chen and Qiao Wang},
  year={2026},
  eprint={2603.25037},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2603.25037},
}

Contact

• Author: Jianbo Qi
• Email: jianboqi@126.com
• Issues: GitHub Issues