pyvpp 0.1.6

Python package to download phenological data from Wekeo (HR-VPP datsets from Sentinel 2)[

PyVPP

Python package to download data from the Pan European High Resolution Vegetation Phenology and Productivity part of the Copernicus Land Monitoring Service (CLMS). This package have been developed within the framework of the eLTER H2020 and SUMHAL projects, as a tool aimed at scientists and managers of the sites integrated in the eLTER network, for which long-term phenology monitoring can be assessed.

What we use

The package is based in these magnificient python libraries:

• HDA python package to download data from Wekeo.
• Deims is a python package to get data from the Dynamic Ecological Information Management System - Site and dataset registry (DEIMS-SDR). We use this library to quickly get the spatial boundaries of the eLTER sites.
• PyProj, Geopandas and Rasterio are used (as always :blush:) to do the spatial parts related with getting extents, reprojecting, mosaicking and clipping.

What we do

The script accepts both as input, a shapefile or a deims id (please remember to add "deimsid:yoursiteid" in case you use this option), then it will read it as a Geodataframe, and it will download all the tiles that intersect the Bounding Box of the input boundaries. Once the tiles are downloaded it will mosaic them gruping by dates and products and it will crop the mosaics with the boundaries. Keeping in the ouptu folder (it will create a /pyhda folder in your current working directory) just the "mosaic_selectedvariable_rec.tif" files.

At the moment we offer 4 datasets:

1. VPP_Index (Vegetation Indices): EO:EEA:DAT:CLMS_HRVPP_VI

2. VPP_Pheno (Phenology and Productivity): EO:EEA:DAT:CLMS_HRVPP_VPP

3. VPP_Seasonal Trajectories (Vegetation Indices): EO:EEA:DAT:CLMS_HRVPP_ST

4. Land Surface Temperature Radiometer (SLSTR: EO:ESA:DAT:SENTINEL-3:SL_2_LST___

Just a little of the whole catalog offered in Wekeo (you can have a look here), but they are the ones that we are interested in.

It's very likely that we will add more datasets in the near future. Even better, if you find this package helpful and you want to add some more datasets, please feel free to do it (and maybe share that with us with a Pull Requests :wink:)

Why a package

As we said, this is intended to be used for eLTER sites managers and/or researchers, but it could be used to download data for any part of the globe (where data exists, of course). So we consider that this script could be profited for people moreover eLTER network. Plus, we think that have a python package to import it, is an approach more "friendly to use" than having a long script to copy and paste in different enviroments.

How to use it

Very easy, just import the library, select your input shape or deims id site, select the dates and the variables and launch the process with .run(). The typical workflow, once installed, would be like that:

import pyvpp

# Just replace the second parameter (deimsid for /path/to/yor/shapefile.shp) for your local shapefile in case you want to use a shape.
MyWekeo = pyvpp.wekeo_download("VPP_Index", "deimsid:https://deims.org/bcbc866c-3f4f-47a8-bbbc-0a93df6de7b2", ['2018-01-01', '2018-06-30'], ['LAI', 'FAPAR'])
MyWekeo.run()

The upper code will do the whole process for Doñana Long-Term Socio-ecological Research Platform (you can paste the link in a browser if you are curious) and just remain the LAI and FAPAR variables cropped with Doñana boundaries in the /pyhda folder. There's also the possibility to keep the original Sentinel 2 tiles (this product are based in Sentinel 2 images, sorry, maybe I have should said this before) in your folder.

In that case you maybe want to run the process once by once, what can be done in the next way:

import pyvpp

MyWekeo = pyvpp.wekeo_download("VPP_Index", "deimsid:https://deims.org/bcbc866c-3f4f-47a8-bbbc-0a93df6de7b2", ['2018-01-01', '2018-06-30'], ['LAI', 'FAPAR'])
# Do the download
MyWekeo.download()
# Do the mosaics and cropping
MyWekeo.mosaic()
# Cleaning folder of original tiles and whole mosaics
MyWekeo.clean()

You can find the list of variables availables in these 2 datasets in Wekeo, but we work with the 4 vegetation indices:

1. PPI, Plant Phenology Index
2. NDVI, Normalized Difference vegetation Index
3. LAI, Leaf Area Index,
4. FAPAR, Fraction of Absorbed Photosynthetically Active Radiation

And just with these phenometrics (but you could download any of the availables, just be sure to type correctly their names):

1. SOSD, Start of the season Day of the Year
2. SOSV, Start of the season Value of Vegetation Index
3. MAXD, Maximun of the season Day of the Year
4. MAXV, Maximun of the season Value of Vegetation Index
5. EOSD, End of the season Day of the Year
6. EOSV, End of the season Value of the Vegetation Index

For the Seasonal Trajectories the only one product is the 10 days improved PPI index:

1. PPI, Plant Phenology Index

And last, for the SLSTR the only product available is the Land Surface Temperature:

1. SL_2_LST___, Land Surface Temperature

Important info! (specially for Datalab users)

The download from Wekeo needs a token generated taken into account a config file where yuour user and password are stored (please, read this). If you are running this in your local PC, just cerate your .hdarc file with your personal info and forget about it. But, if you run this in the Datalab (This is just for eLTER users), maybe you don't feel comfortable with the idea of having your credentials stored in the Datalab. Plus, if you don't delete this file and someone runs the script, the data will be downloaded with your account. Not a big deal, but it better to solve that.

So, we have add a couple of functions to fix this. fillHda() and delHdaInfo(). These functions are imported when you import the script, so the only thing you need to do is this:

# This will create or rewrite in case that it already exists, the .hdarc with your credentials in the home folder.
# This should be the first thing that you do before run the WekeoDownload process
fillHda("youruser", "yourpassword")

# And just be sure to run the del funciton when you are done. This will left the .hdarc but without any credentials in it, just white spaces waiting for the next fillHda() call.
# This should be the last thing that you do before exit your datalab session.
delHdaInfo()  

ToDo

To do for you: Please, don't forget download and delete the Pyhda folder in case you are in the Datalab.