Skip to main content

envlib

A distributed catalogue for environmental data

build codecov PyPI version


Documentation: https://envlib.github.io/envlib/

Source Code: https://github.com/envlib/envlib


Overview

envlib is a distributed database and catalogue for environmental datasets — gridded model output and station time series alike. Datasets are stored as cfdb files on S3-compatible object storage, hosted by whoever owns the data; envlib provides the shared layer on top: standardized metadata, controlled vocabularies, deterministic identifiers, and a catalogue you can query to discover and open any registered dataset.

Key features

  • Distributed by design — the catalogue is an index, not a data silo: each dataset stays on its owner's storage, and one catalogue can span many owners.
  • Standardized metadata — eleven identity fields (drawing on ODM2, CF conventions, and lessons from tethys) describe what every dataset is, validated against controlled vocabularies at creation time.
  • Deterministic, permanent identifiers — a dataset's identity metadata hashes to stable ids (dataset_id, dataset_version_id, station_id), so the same data gets the same id everywhere, forever.
  • Queryable catalogue — filter by any identity field, spatial extent (including across the antimeridian), and time range; browse what a catalogue holds with cat.variables, cat.owners, and friends.
  • CF standard names derived for you — envlib curates the mapping from its variables to CF standard_names and applies it automatically at registration.
  • cfdb storage — every dataset version is a cfdb file: chunked, compressed, partially readable, S3-syncable.

Installation

pip install envlib

Quick example

import envlib

# The public envlib catalogue — zero config, no credentials
# (until it's hosted: Catalogue(remotes=['https://.../catalogue.rcg']))
cat = envlib.Catalogue()

# What's in it?
cat.variables       # ['precipitation', 'streamflow', 'temperature']
cat.owners          # ['ecan', 'ecmwf', 'niwa']

# Find the latest version of each matching dataset
results = cat.query(
    variable='temperature',
    feature='atmosphere',
    bbox=[166, -47, 179, -34],
    start_date='2020-01-01',
)

# Open one as a cfdb dataset and read it
ds = results[0].open()
temp = ds['temperature']

Producing data is the same library in the other direction: build a cfdb file, attach envlib.Metadata, and cat.publish(...) it — see the documentation for the full guide.

Changes between releases are tracked in the changelog.

Development

Setup environment

We use UV to manage the development environment and production build.

uv sync

Run unit tests

You can run all the tests with:

uv run pytest

Format the code

Execute the following commands to apply linting and check typing:

uv run ruff check .
uv run black --check --diff .
uv run mypy --install-types --non-interactive envlib

To auto-format:

uv run black .
uv run ruff check --fix .

License

This project is licensed under the terms of the Apache Software License 2.0.

Metadata

Release files for envlib 0.1.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for envlib 0.1.8
File Size Uploaded Provenance Actions
envlib-0.1.8.tar.gz 89.5 kB Without provenance Details for envlib-0.1.8.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for envlib 0.1.8
File Interpreter ABI Platform Actions
envlib-0.1.8-py3-none-any.whl Python 3 none any Details for envlib-0.1.8-py3-none-any.whl

Total release size: 182.2 kB

Release files / envlib-0.1.8.tar.gz

Download URL envlib-0.1.8.tar.gz
Size 89.5 kB
Tags Source
SHA-256 checksum
How to use checksums
ae1648e088bf752511c1d808cf1b14ec774528f4569d7e75312d6d4730f77468
BLAKE2b-256 checksum
How to use checksums
2db6377d44105166ceb0d47c47e52e1f938a47c2c265a66638e1cc30bc81e4a0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.7

Release files / envlib-0.1.8-py3-none-any.whl

Download URL envlib-0.1.8-py3-none-any.whl
Size 92.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c496f61104913428fa9f53a113cb04d18680cba6e1dd5647b47c6630d374e848
BLAKE2b-256 checksum
How to use checksums
2faa882ac68b33e2bcb3ab2e04f9a0b276c76e4c42a6ef370ddaed8082d495a4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.7

Security for envlib 0.1.8

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page