pythia-datasets 2021.9.21

Provides utility functions for accessing data repository for Project Pythia examples/notebooks

CI
Docs
Package
License

pythia-datasets

Data repository for Project Pythia examples/notebooks

Sample data sets

These files are used as sample data in Pythia Project examples/notebooks and are downloaded by pythia_datasets package:

• NARR_19930313_0000.nc
• enso_data.csv
• jan-17-co-asos.txt.xz
• CESM2_sst_data.nc
• CESM2_grid_variables.nc

Adding new datasets

To add a new dataset file, please follow these steps:

1. Add the dataset file to the data/ directory
2. From the command line, run python make_registry.py script to update the registry file residing in pythia_datasets/registry.txt
3. Commit and push your changes to GitHub

Using datasets in notebooks and/or scripts

• Ensure the pythia_datasets package is installed in your environment

  python -m pip install pythia-datasets
  
  # or
  
  python -m pip install git+https://github.com/ProjectPythia/pythia-datasets
  

• Import DATASETS and inspect the registry to find out which datasets are available

  In [1]: from pythia_datasets import DATASETS
  
  In [2]: DATASETS.registry_files
  Out[2]: ['jan-17-co-asos.txt.xz', 'NARR_19930313_0000.nc']
  

• To fetch a data file of interest, use the .fetch method and provide the filename of the data file. This will

  • download and cache the file if it doesn't exist already.
  • retrieve and return the local path

  In [4]: filepath = DATASETS.fetch('jan-17-co-asos.txt.xz')
  
  In [5]: filepath
  Out[5]: '/Users/abanihi/Library/Caches/pythia-datasets/jan-17-co-asos.txt.xz'
  

• Once you have access to the local filepath, you can then use it to load your dataset into pandas or xarray or your package of choice:

  In [6]: df = pd.read_csv(filepath)