A data visualization tool for PyCCD time-series model results and Landsat ARD
Project description
Abstract
The Timeseries Analysis and Plotting (TAP) Tool is being developed to provide exploratory data analysis for the Land Change Monitoring, Assessment, and Projection (LCMAP) project at USGS EROS. TAP is an open-source project written in python that uses PyQt5 and matplotlib for generating interactive GUIs and plots. Currently, it provides the following functionality:
Plot a timeseries of Landsat Analysis Ready Data (ARD)
Landsat 4-7 Bands 1, 2, 3, 4, 5, 7 Surface Reflectance
Landsat 8 Bands 2-7 Surface Reflectance
Landsat 4-7 Band 6 Brightness Temperature
Landsat 8 Band 10 Brightness Temperature
Plot model parameters and curve-fits from the Continuous Change Detection and Classification (CCDC) algorithm
Calculate indices on the fly for plotting and visualization
NDVI
MSAVI
SAVI
EVI
NDMI
NBR
NBR-2
Display RGB visualizations of Landsat ARD
Export plotted Landsat ARD surface reflectance, brightness temperature, and index values to a .csv file
Save a plot figure as a .png file
Save a Landsat ARD RGB display as a .png file
Save ESRI point shapefile for plotted coordinates
Installation
These instructions work for both Windows and Linux systems.
lcmap_tap has been installed and tested successfully on Windows 7 & 10, and CentOS Linux (version 7).
Note: some functionality could be unstable or intermittent as development of LCMAP operations is ongoing.
System Requirements
python >= 3.6
GDAL == 2.2.2
poppler == 0.60.1 (Linux only)
PyQt5 == 5.10.1*
cython
cytoolz
matplotlib
numpy
pandas
pyyaml
requests
* Note on PyQt5: For development any version of PyQt5>=5.9 is currently suitable. For running TAP, it is recommended to use PyQt5==5.10.1 because of an issue in later versions in which Qtwebengineprocess does not close on TAP exit. If building a stand-alone executable with PyInstaller, PyQt5==5.10.1 seems to work best at the moment.
The tool currently requires the following data inputs:
Serialized change and cover results generated by PyCCD.
Landsat ARD (obtained via HTTP request)
Landsat ARD and the PyCCD algorithm are publicly available. However, TAP currently requires access to ARD via a web-service that is currently available to on-site personnel at USGS EROS only.
In addition, the pre-generated PyCCD results must be stored locally and are expected to be in a specific format.
For these reasons, TAP is not intended for general public use until these datasets become widely available at a future date.
Note
It is recommended to use an Anaconda or Miniconda virtual environment since it provides an easier method of installation for GDAL, cython, and cytoolz. Otherwise, information for installing GDAL manually can be found here.
Create a Virtual Environment
Install Anaconda or Miniconda
Download here
Create a Conda virtual environment
Linux
conda env create --name tap python=3.6.7 gdal=2.2.2 poppler=0.60.1 cython cytoolz
Windows
conda env create --name tap python=3.6.7 gdal=2.2.2 cython cytoolz
If you experience SSL cert errors, ssl cert verification can be temporarily disabled for conda via:
conda config --set ssl_verify false
Activate the environment
conda activate tap
Install TAP
Ensure the virtual environment is activated from previous creation steps.
Use pip to install TAP and the remaining dependencies
pip install lcmap-tap
If you are having trusted host/certificate issues during pip install, you can explicitly trust pypi
pip install lcmap-tap --trusted-host python.pypi.org --trusted-host pypi.org --trusted-host files.pythonhosted.org
Create the configuration file required by tap
Linux Example
Create: “$HOME/.conda/envs/tap/lib/python3.6/site-packages/lcmap_tap/config.yaml”
URL: http://lcmap-test.cr.usgs.gov/ARD_CU_C01_V01 CCD: /lcmap_data/bulk/tiles
Windows Example
Create: “$HOME\AppData\Local\Continuum\miniconda3\envs\tap\Lib\site-packages\lcmap_tap\config.yaml”
URL: http://lcmap-test.cr.usgs.gov/ARD_CU_C01_V01 CCD: Z:\bulk\tiles
Run the Tool
Once installed, lcmap_tap can be executed directly from the command line if the virtual environment is activated:
Linux
lcmap_tap
Windows
python.exe $HOME\AppData\Local\Continuum\miniconda3\envs\tap\Scripts\lcmap_tap.exe
Packaging
Packaging tap-tool using PyInstaller for distribution of an executable binary.
Pre-Reqs
The following instructions assume the above Installation steps have already been completed.
Conda virtual environment is activated
Download a ZIP archive of the TAP source code for the target tag/branch.
Extract the zipped folder.
From the command line, cd into the extracted folder.
Current working directory is inside the extracted lcmap-tap ZIP archive
Instructions
Install the most recent version of PyInstaller with pip directly from github:
pip install --trusted-host python.pypi.org --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host github.com --trusted-host codeload.github.com https://github.com/pyinstaller/pyinstaller/archive/develop.zip
Create a new file - a short script that calls lcmap-tap, save as “run_lcmap_tap.py”
from lcmap_tap.__main__ import main main()
Freeze the script – this will recursively search all imported modules in lcmap_tap to include in the build environment:
pyinstaller run_lcmap_tap.py
Copy the file “config.yaml” (created in previous Installation section) from the conda environment into the downloaded/extracted lcmap-tap top level directory (should contain setup.py).
Once complete, open the run_lcmap_tap.spec file to edit.
We need to tell PyInstaller to include certain non-python data files. Add the following to the ‘datas’ list.
datas=[( '.\\config.yaml', 'lcmap_tap'), ('.\\lcmap_tap\\Auxiliary\\icon.png', 'lcmap_tap/Auxiliary'), ('.\\lcmap_tap\\MapCanvas\\UseWebEngineView', 'lcmap_tap/MapCanvas/UseWebEngineView'), ('.\\lcmap_tap\\MapCanvas\\UseWebView', 'lcmap_tap/MapCanvas/UseWebView')],
Note: the spec file is read as a python file. Datas is a list of tuples, each tuple contains two strings, the first is the path to the data file or folder, the second is the destination for the data file in the build module.
Update the hidden imports in the run_lcmap_tap.spec file to include:
hiddenimports=['PyQt5.sip', 'gdal', 'ogr', 'osr', 'merlin'],
Note: Doing this seems to fix an issue with not being able to import gdal later on. I think it also helps avoid errors with importing these other modules as well.
Re-run PyInstaller, but point to the spec file rather than the script file
pyinstaller run_lcmap_tap.spec
The resulting ‘dist’ folder contains the stand-alone TAP tool and this is what should be distributed on to users’ systems.
Zip up this directory for distribution: dist/run_lcmap_tap
Run TAP simply by double-clicking the executable file, or a shortcut that points to this executable:
dist/run_lcmap_tap/run_lcmap_tap.exe
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for lcmap_tap-1.2.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 036ea4376687c04d596cc7f4c173fe93ba50243d53ea319f6ca0bc7c17206824 |
|
MD5 | cd59064679ca57d890e1048aaa4f9296 |
|
BLAKE2b-256 | 2fdc0059ee9a8cff1981618518e8f9998af2c98c06b867e820538162722b5c9d |