A data visualization tool for PyCCD time-series model results and Landsat ARD
The Timeseries Analysis and Plotting (TAP) Tool is being developed to provide exploratory data analysis for the Land Change Monitoring, Assessment, and Projection (LCMAP) project at USGS EROS. TAP is an open-source project written in python that uses PyQt5 and matplotlib for generating interactive GUIs and plots. Currently, it provides the following functionality:
- Plot a timeseries of Landsat Analysis Ready Data
- Landsat 4-7 Bands 1, 2, 3, 4, 5, 7 Surface Reflectance
- Landsat 8 Bands 2-7 Surface Reflectance
- Landsat 4-7 Band 6 Brightness Temperature
- Landsat 8 Band 10 Brightness Temperature
- Plot model parameters and curve-fits from the Continuous Change Detection and Classification (CCDC) algorithm
- Calculate indices on the fly for plotting and visualization
- Display RGB visualizations of Landsat ARD
- Export plotted Landsat ARD surface reflectance, brightness temperature, and index values to a .csv file
- Save a plot figure as a .png file
- Save a Landsat ARD RGB display as a .png file
- Save ESRI point shapefile for plotted coordinates
These instructions work for both Windows and Linux systems.
lcmap_tap has been installed and tested successfully on Windows 7 & 10, and CentOS Linux (version 7).
Note: some functionality could be unstable or intermittent as development of LCMAP operations is ongoing.
- python >= 3.6
- GDAL == 2.2.2
- poppler == 0.60.1 (Linux only)
- PyQt5 == 5.10.1*
* Note on PyQt5: For development any version of PyQt5>=5.9 is currently suitable. For running TAP, it is recommended to use PyQt5==5.10.1 because of an issue in later versions in which Qtwebengineprocess does not close on TAP exit. If building a stand-alone executable with PyInstaller, PyQt5==5.10.1 seems to work best at the moment.
The tool currently requires the following data inputs:
- Serialized change and cover results generated by PyCCD.
- Landsat ARD (obtained via HTTP request)
Landsat ARD and the PyCCD algorithm are publicly available. However, TAP currently requires access to ARD via a web-service that is currently available to on-site personnel at USGS EROS only.
In addition, the pre-generated PyCCD results must be stored locally and are expected to be in a specific format.
For these reasons, TAP is not intended for general public use until these datasets become widely available at a future date.
Create a Virtual Environment
- Install Anaconda or Miniconda
- Download here
- Create a Conda virtual environment
Linuxconda env create --name tap python=3.6.7 gdal=2.2.2 poppler=0.60.1 cython cytoolz
Windowsconda env create --name tap python=3.6.7 gdal=2.2.2 cython cytoolz
If you experience SSL cert errors, ssl cert verification can be temporarily disabled for conda via:
conda config --set ssl_verify false
Activate the environment
conda activate tap
Ensure the virtual environment is activated from previous creation steps.
Use pip to install TAP and the remaining dependencies
pip install lcmap-tap
If you are having trusted host/certificate issues during pip install, you can explicitly trust pypi
pip install lcmap-tap --trusted-host python.pypi.org --trusted-host pypi.org --trusted-host files.pythonhosted.org
Create the configuration file required by tap
Create: “$HOME/.conda/envs/tap/lib/python3.6/site-packages/lcmap_tap/config.yaml”URL: http://lcmap-test.cr.usgs.gov/ARD_CU_C01_V01 CCD: /lcmap_data/bulk/tiles
- Create: “$HOME\AppData\Local\Continuum\miniconda3\envs\tap\Lib\site-packages\lcmap_tap\config.yaml”URL: http://lcmap-test.cr.usgs.gov/ARD_CU_C01_V01 CCD: Z:\bulk\tiles
Run the Tool
Once installed, lcmap_tap can be executed directly from the command line if the virtual environment is activated:
Packaging tap-tool using PyInstaller for distribution of an executable binary.
- The following instructions assume the above Installation steps have already been completed.
- Conda virtual environment is activated
- Download a ZIP archive of the TAP source code for the target tag/branch.
- Extract the zipped folder.
- From the command line, cd into the extracted folder.
- Current working directory is inside the extracted lcmap-tap ZIP archive
Install the most recent version of PyInstaller with pip directly from github:
pip install --trusted-host python.pypi.org --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host github.com --trusted-host codeload.github.com https://github.com/pyinstaller/pyinstaller/archive/develop.zip
Create a new file - a short script that calls lcmap-tap, save as “run_lcmap_tap.py”
from lcmap_tap.__main__ import main main()
Freeze the script – this will recursively search all imported modules in lcmap_tap to include in the build environment:
Copy the file “config.yaml” (created in previous Installation section) from the conda environment into the downloaded/extracted lcmap-tap top level directory (should contain setup.py).
Once complete, open the run_lcmap_tap.spec file to edit.
We need to tell PyInstaller to include certain non-python data files. Add the following to the ‘datas’ list.
datas=[( '.\\config.yaml', 'lcmap_tap'), ('.\\lcmap_tap\\Auxiliary\\icon.png', 'lcmap_tap/Auxiliary'), ('.\\lcmap_tap\\MapCanvas\\UseWebEngineView', 'lcmap_tap/MapCanvas/UseWebEngineView'), ('.\\lcmap_tap\\MapCanvas\\UseWebView', 'lcmap_tap/MapCanvas/UseWebView')],
- Note: the spec file is read as a python file. Datas is a list of tuples, each tuple contains two strings, the first is the path to the data file or folder, the second is the destination for the data file in the build module.
Update the hidden imports in the run_lcmap_tap.spec file to include:
hiddenimports=['PyQt5.sip', 'gdal', 'ogr', 'osr', 'merlin'],
- Note: Doing this seems to fix an issue with not being able to import gdal later on. I think it also helps avoid errors with importing these other modules as well.
Re-run PyInstaller, but point to the spec file rather than the script file
The resulting ‘dist’ folder contains the stand-alone TAP tool and this is what should be distributed on to users’ systems.
- Zip up this directory for distribution: dist/run_lcmap_tap
Run TAP simply by double-clicking the executable file, or a shortcut that points to this executable:
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size lcmap_tap-1.2.1-py3-none-any.whl (112.4 kB)||File type Wheel||Python version py3||Upload date||Hashes View hashes|
|Filename, size lcmap_tap-1.2.1.tar.gz (95.6 kB)||File type Source||Python version None||Upload date||Hashes View hashes|
Hashes for lcmap_tap-1.2.1-py3-none-any.whl