caeli-data

The Python client of the Caeli data repository API (https://data.caeli.space). One door for every producer and reader of datasets: the inventory platform, pipelines and colleagues all use this library; nothing writes to the tables or the buckets directly.

pip install caeli-data            # from the CodeArtifact repository caeli-pypi (see docs/library.md)

from caeli_data import Repository, TableRejected, IngestFailed

repo = Repository()               # CAELI_API_URL (default https://data.caeli.space) and CAELI_API_KEY
repo.whoami()                     # {"name": "tim", "role": "admin", ...}

# a prepared Parquet file of a table dataset: split per partition region and month, load every part
for part in repo.split_table_file("prepared.parquet", "out/"):
    job = repo.load_table(part.path, "cdse-sentinel-5p-l2-no2", part.area, part.period,
                          software={"my-pipeline": "1.0"}, source_ref="download 4711")
    print(job["message"])         # "200000 rows appended in cdse_sentinel_5p_l2_no2, ... item N_EU__2026-09__v1 now ..."

A service takes its key from Secrets Manager: Repository.from_secret("caeli/data-platform/api-key-inventory-platform").

Rasters, the read side (catalogue, versions, downloads) and the command line follow in the next versions. The rules a table file must meet are in Caeli_data_repository_conventions (section 6); the library checks them before uploading and says which rule a file breaks.