swepy 1.9.2

A python package for obtaining and manipulating Tb files from the MEaSUREs database

SWEpy

Quick Start Guide

For Full Documentation, Please see the Read The Docs!

SWEpy is a Python library designed to simplify access to a passive microwave brightness temperature dataset available at the National Snow and Ice Data Center (NSIDC). This dataset contains Northern and Southern hemisphere imagery along with Equatorial imagery, and can be quite useful in analyzing snow water equivalent (SWE) over large spatial extents. SWEpy contains tools to web scrape, geographically subset, and concatenate files into time cubes. There is an automated workflow to scrape long time series while periodically stopping to geographically subset and concatenate files in order to reduce disk impact.

Setup:

1. Setup Earthdata Login

Create an Earthdata account to be able to download data: https://urs.earthdata.nasa.gov/

2. Setup conda environment from yaml (Recommended)

The libraries used in this analysis, namely pynco, can be finicky with the channels that dependencies are installed with. Thus, using the provided yaml file to build an environment for this project will make your life simpler. You can add more packages on top of the provided environment as long as you install with the conda-forge channel.

Using the yaml file (.yml) create a new conda environment

conda env create -f swepy_env.yml

Alternative: Install SWEpy Using Conda or pip:

SWEpy is available from anaconda, and will install all dependencies when installed. It is also available from pip (Pypi), but will not install all the dependencies automatically.

** Important ** conda-forge must be the first channel in your .condarc file.

channels:
  - conda-forge
  - defaults

conda install swepy

3. Install ipykernel (if using jupyter and conda environments)

source activate swepy_env
python -m ipykernel install --user --name <env name> --display-name "<display name>"

Do not include the brackets <>

Using SWEpy for analyzing SWE:

1. Import the Library:

from swepy.swepy import swepy

2. Instantiate the class with working directory and bounding coordinates

• To scrape data you must also set your Earthdata login and desired date range

   from swepy import pipeline
  
   upper_left = [lon_upleft, lat_upleft]
   lower_right = [lon_lowright, lat_lowright]
  
   start = datetime.date(startY, startM, startD)
   end = datetime.date(endY, endM, endD)
  
   path = os.getcwd()
  
   username = "username"
   password = "password"
  
   swe = pipeline.Swepy(path, upper_left, lower_right, high_res = True)
   swe.set_login(username, password)
   swe.set_dates(start, end)
  

3. Don't forget to orient your upper-left and lower-right bounding coordinates with the EASE Grid 2.0 Orientation:

• By default, the high_res parameter is set to True, meaning it will scrape high resolution images. If it is passed as 'False' then it will scrape 25km images instead of the 6.25km high resolution images.

5. Get Files

   a. Use desired functionality, either separate or individually:

   swe.scrape()
   swe.subset()
   swe.concatenate()
   
   swe.concatenate(swepy.subset(swepy.scrape()))
   

   b. Or, use scrape_all to avoid massive file sizes:

   swe.scrape_all()
   

   This limits the number of full-size images on your disk at one time.

6. If you need to give the class more information, or change information it already has, use one of the set_ functions:

   swe.set_grid(ul = [-145,66], lr = [-166, -16])
   

Using SWEpy's Web Scraper Alone:

• The web scraper is enabled automatically in the scrape_all workflow, however it can also be used as a standalone function!

from swepy.nsidcDownloader import nsidcDownloader

## Ways to instantiate nsidcDownloader
nD = nsidcDownloader.nsidcDownloader(username="user", password="pass", folder=os.getcwd())


## Download a file:

file = {
    "resolution": "3.125km",
    "platform": "F17",
    "sensor": "SSMIS",
    "date": datetime(2015,10,10),
    "channel": "37H"
}

nD.download_file(**file)

Main Dependencies:

• gdal
• affine
• requests
• scikit-image
• pynco
• netCDF4
• datetime
• tqdm
• pandas

Troubleshooting:

1. Missing image error when loading in swepy or when calling swepy functions

   • These are channel dependency errors and likely arise due to some of your packages being on conda-forge and others being on other channels. Namely, pynco struggles with this.
   • Make sure conda-forge is at the top of your .condarc file and then run a conda update --all.
   • https://conda-forge.org/docs/conda-forge_gotchas.html#using-multiple-channels

2. Importing SWEpy fails, or pandas fails to find numpy.

   • This seems to be an issue caused by numpy v1.15.0. I reverted back to 1.14.5 and reinstalled everything and it worked again.

If you experience any other issues, do not hesitate to open an issue in this repo!

Citations:

This library is designed to work with the MEaSUREs CETB dataset:

Brodzik, M. J., D. G. Long, M. A. Hardman, A. Paget, and R. Armstrong. 2016. MEaSUREs Calibrated Enhanced-Resolution Passive Microwave Daily EASE-Grid 2.0 Brightness Temperature ESDR, Version 1. [Indicate subset used]. Boulder, Colorado USA. NASA National Snow and Ice Data Center Distributed Active Archive Center. doi: https://doi.org/10.5067/MEASURES/CRYOSPHERE/NSIDC-0630.001. [June 2018].

Acknowledgements:

I want to thank Davey Lovin and David Nyberg for their help in conceiving the original project and developing the original prototype!

I also want to thank Carson Farmer and Earth Lab, CU Boulder for their support throughout the development of this project.