Intake-esm
Badges
| CI |    |
|---|---|
| Docs |  |
| Package |    |
| License |  |
| Citation |  |
Motivation
Computer simulations of the Earth’s climate and weather generate huge amounts of data. These data are often persisted on HPC systems or in the cloud across multiple data assets of a variety of formats (netCDF, zarr, etc...). Finding, investigating, loading these data assets into compute-ready data containers costs time and effort. The data user needs to know what data sets are available, the attributes describing each data set, before loading a specific data set and analyzing it.
Finding, investigating, loading these assets into data array containers such as xarray can be a daunting task due to the large number of files a user may be interested in. Intake-esm aims to address these issues by providing necessary functionality for searching, discovering, data access/loading.
Overview
intake-esm is a data cataloging utility built on top of intake, pandas, polars and xarray, and it's pretty awesome!
-
Opening an ESM catalog definition file: An Earth System Model (ESM) catalog file is a JSON file that conforms to the ESM Collection Specification. When provided a link/path to an esm catalog file,
intake-esmestablishes a link to a database (CSV file) that contains data assets locations and associated metadata (i.e., which experiment, model, the come from). The catalog JSON file can be stored on a local filesystem or can be hosted on a remote server.In [1]: import intake In [2]: import intake_esm In [3]: cat_url = intake_esm.tutorial.get_url("google_cmip6") In [4]: cat = intake.open_esm_datastore(cat_url) In [5]: cat Out[5]: <GOOGLE-CMIP6 catalog with 4 dataset(s) from 261 asset(s>
-
Search and Discovery:
intake-esmprovides functionality to execute queries against the catalog:In [5]: cat_subset = cat.search( ...: experiment_id=["historical", "ssp585"], ...: table_id="Oyr", ...: variable_id="o2", ...: grid_label="gn", ...: ) In [6]: cat_subset Out[6]: <GOOGLE-CMIP6 catalog with 2 dataset(s) from 67 asset(s)>
-
Access: when the user is satisfied with the results of their query, they can load data assets (netCDF and/or Zarr stores) into xarray datasets:
In [7]: dset_dict = cat_subset.to_dataset_dict() --> The keys in the returned dictionary of datasets are constructed as follows: 'activity_id.institution_id.source_id.experiment_id.table_id.grid_label' |███████████████████████████████████████████████████████████████| 100.00% [2/2 00:18<00:00]
See documentation for more information.
Installation
Intake-esm can be installed from PyPI with pip:
python -m pip install intake-esm
It is also available from conda-forge for conda installations:
conda install -c conda-forge intake-esm
3 maintainers
Metadata
3 maintainers
Release files for intake-esm 2025.12.12
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| intake_esm-2025.12.12.tar.gz | 119.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| intake_esm-2025.12.12-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 154.5 kB
Release files / intake_esm-2025.12.12.tar.gz
| Download URL | intake_esm-2025.12.12.tar.gz |
|---|---|
| Size | 119.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b99d2d07347a9ea98261277960ccce8a737f683521472bc4ca879b8f71bc7ed2
|
|
BLAKE2b-256 checksum How to use checksums |
7c79e29e52150367c1f07dd10b61bfbf7886c5cb282c537e826cf7bd5db5545c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 12, 2025.
Transparency logRelease files / intake_esm-2025.12.12-py3-none-any.whl
| Download URL | intake_esm-2025.12.12-py3-none-any.whl |
|---|---|
| Size | 35.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
aabc5ffdb195235eabf8ea6833bab9d9d04d159f7da2aa716c7550ba7078f6fe
|
|
BLAKE2b-256 checksum How to use checksums |
f85cc3665d907704eee62841330fa7771ba38c0bd54b0b268eb6c01612fac53e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 12, 2025.
Transparency log
