pycopdem

Cached Copernicus GLO-30 elevation with on-read terrain derivatives — download once per chunk, never twice. Every elevation pixel this machine ever downloads lands in one sparse, chunk-indexed store on the DEM's native 1-arc-second grid; slope, aspect, flow accumulation, TWI and Heat Load Index are computed on read and never stored. Part of the Borevitz Lab ecosystem.

How it works

{data_root}/copdem_store/
├── index.db      # SQLite ledger: which chunks are populated
└── dem.zarr      # global sparse array; only written 1200×1200-px chunks exist

• The GLO-30 DEM is served as one COG per 1° × 1° cell on a 1-arc-second EPSG:4326 lattice. The store uses the same lattice globally, chunked at 1200 px (1/3°) — so 3 × 3 chunks nest exactly inside every tile, and fetching a chunk is a single integer-aligned windowed read from one COG. No resampling, ever.
• Any bbox maps deterministically to a set of chunk ids. Store.get_ds(bbox) diffs them against the ledger and downloads only the missing chunks. Elevation is time-invariant, so there's no time axis and no date bookkeeping.
• A 1° tile absent from S3 is genuinely all ocean: its chunks are stored as nodata and marked complete.
• Derivatives are a read-time transform (like everything derived in this ecosystem): request them per call, pay compute not disk.

Usage

The core API is troi-agnostic — just a bbox:

from pycopdem.store import Store

store = Store()
bbox = [148.36265, -33.52606, 148.38265, -33.50606]  # [W, S, E, N]

ds = store.get_ds(bbox)                              # elevation (lat, lon)
ds = store.get_ds(bbox, derivatives=('slope', 'aspect', 'twi', 'hli'))

store.fill(bbox)                                     # → 0: already local

Derivatives: slope and aspect (degrees), accumulation (pysheds fill-pits → fill-depressions → resolve-flats → flowdir → accumulation), twi (ln(accumulation / tan slope)), hli (McCune & Keon 2002). Dependencies resolve automatically — asking for twi computes slope and accumulation internally.

Pipelines that speak the shared troi.troi.Troi use the adapters (dates on the troi are ignored — elevation doesn't change):

ds = store.get_ds_troi(troi, derivatives=('slope',))

download_terrain(troi) remains as a thin wrapper.

Performance

Live measurements against the Copernicus S3 bucket — a ~2 × 2 km AOI (one chunk = 1200 × 1200 px ≈ 37 × 30 km):

Store footprint: ~10 MB for two chunks (~2 200 km² of elevation). Absolute times vary with network; the zero is the point — it's a ledger lookup, no network involved.

Install

pip

pip install git+https://github.com/thestochasticman/pycopdem.git

Dependencies (the troi core included, pulled from GitHub) are declared in pyproject.toml and installed automatically.

From source

git clone https://github.com/thestochasticman/pycopdem.git
cd pycopdem
pip install -e .

Package design (shared across the lab's packages — no inheritance, composition only):

• Troi (from troi) — identity: what region.
• CopernicusDEM (pycopdem.copdem) — config: tile source.
• Paths (pycopdem.paths) — derived locations of the store for a given Config.
• grid — the fixed 1-arc-second grid and tile/chunk nesting (pure, offline-testable math).
• derive — the terrain derivatives (pure array math + pysheds).
• Store (pycopdem.store) — ties them together.

Test

# offline (pure math + synthetic store):
python pycopdem/grid.py     # True
python pycopdem/paths.py    # True
python pycopdem/derive.py   # True
python pycopdem/store.py    # True

# live (small real reads from the Copernicus S3 bucket):
python pycopdem/download_terrain.py  # True