A python package for obtaining and cleaning Tb files
Project description
Disclaimer
-
Currently this script is not supported in Windows due to pynco only supporting Mac OS or Unix
-
Web scrapping currently only working on the northern hemisphere
-
Requires python 3.6 and Anaconda 3
1. Setup Earthdata Login
Create an Earthdata account to be able to download data: https://urs.earthdata.nasa.gov/
Setup your username and password in a .netrc file
Run this command in the directory you will be working in
echo "machine urs.earthdata.nasa.gov login <uid> password <password>" >> ~/.netrc
chmod 0600 ~/.netrc
is your Earthdata username. Do not include the brackets <>.
2. Build Conda Environment
Using the yaml file (.yml) create a new conda environment
conda env create -f swepy_env.yml
3. Install ipykernel (if using jupyter)
source activate swepy_env
python -m ipykernel install --user --name swepy_env --display-name "Python (swepy_env)"
4. Run Script
source activate myenv
python full_workflow.py
Troubleshooting
If encountering ‘image not found’ errors then one possible fix is to add theconda-forge channel on top of the defaults in your .condarc file. This is a hidden file, show hidden files and then edit the .condarc file and make your file look like this:
$ cat .condarc
channels:
- conda-forge
- defaults
After saving this file, update conda:
conda update all
https://conda-forge.org/docs/conda-forge_gotchas.html#using-multiple-channels
- If getting HDF5 errors, try deleting all the netCDF files in your directories.
Function Summaries
Descriptions of included functions
get_xy(ll_ul, ll_lr)
- Parameters: lists of latitude/longitude upper left, latitude/longitude lower right
- Uses NSIDC scripts to convert user inputted lat/lon into Ease grid 2.0 coordinates
- Returns: Ease grid 2.0 coordinates of inputted lat/longs
subset(list6, path)
- Parameters: coordinates of area of interest, current working directory
- Subset will get the files from wget directory and subset them geographically
- Returns: subsetted file
concatenate(path, outfile_19, outfile_37, final=False)
- Parameters: current working directory, output file for 19Ghz, output file for 37Ghz
- The concatenate function merges all netCDF files into one large file
- Returns: concatenated netCDF file
file_setup(path)
- Parameters: current working directory
- setup files needed for other functions
- Returns: create correct folders for use by other functions
scrape_all(start, end, list3, path=None)
- Parameters: start date, end date, list, current working directory(optional)
- Complete function that downloads, concatenates, and subsets data
- Returns: file names of concatenated 19/37 time cubes
plot_a_day(file1, file2, path, token)
- Parameters: 19Ghz files, 37Ghz files, current working directory, mapbox token
- Plots a day of data using Mapbox Jupyter
- Returns: interactive map of inputted data
Object Oriented Functionality: (Added May 18, 2018)
- Instantiate the class with bounding coordinates, date range, and working directory
upper_left = [float(lat_ul), float(lon_ul)]
lower_right = [float(lat_lr), float(lon_lr)]
start = datetime.date(int(startY), int(startM), int(startD))
end = datetime.date(int(endY), int(endM), int(endD))
path = os.getcwd()
swepy = swepy(path, start, end, upper_left, lower_right)
- Use desired functionality, either separate or individually:
swepy.scrape()
swepy.subset()
swepy.concatenate()
swepy.concatenate(swepy.subset(swepy.scrape()))
Web Scraper Example Use
- Note: Web scraper is enabled automatically in the scrape_all workflow, however it can also be used as a stanalone function!
from nsidcDownloader import nsidcDownloader
## Ways to instantiate nsidcDownloader
nD = nsidcDownloader(username="user", password="pass", folder=".") ## user/pass combo and folder
nD = nsidcDownloader(sensor="SSMIS") ## user/pass combo from .netrc and default folder, setting the default key of sensor
## Download a file:
file = {
"resolution": "3.125km",
"platform": "F17",
"sensor": "SSMIS",
"date": datetime(2015,10,10),
"channel": "37H"
}
nD.download_file(**file)
nD.download_range(sensor="SSMIS", date=[datetime(2014,01,01), datetime(2015,01,01)])
-
Authentication will work if the user/pass combo is saved in
~/.netrc
, or if it is passed in the nsidcDownloader instance -
The class formats the following string:
"{protocol}://{server}/{datapool}/{dataset}.{version}/{date:%Y.%m.%d}" \
"/{dataset}-{projection}_{grid}{resolution}-{platform}_{sensor}" \
"-{date:%Y%j}-{channel}-{pass}-{algorithm}-{input}-{dataversion}.nc"
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.