lcmap-tap 1.2.3

A data visualization tool for PyCCD time-series model results and Landsat ARD

Abstract

The Timeseries Analysis and Plotting (TAP) Tool is being developed to provide exploratory data analysis for the Land Change Monitoring, Assessment, and Projection (LCMAP) project at USGS EROS. TAP is an open-source project written in python that uses PyQt5 and matplotlib for generating interactive GUIs and plots. Currently, it provides the following functionality:

• Plot a timeseries of Landsat Analysis Ready Data (ARD)

  • Landsat 4-7 Bands 1, 2, 3, 4, 5, 7 Surface Reflectance

  • Landsat 8 Bands 2-7 Surface Reflectance

  • Landsat 4-7 Band 6 Brightness Temperature

  • Landsat 8 Band 10 Brightness Temperature

• Plot model parameters and curve-fits from the Continuous Change Detection and Classification (CCDC) algorithm

• Calculate indices on the fly for plotting and visualization

  • NDVI

  • MSAVI

  • SAVI

  • EVI

  • NDMI

  • NBR

  • NBR-2

• Display RGB visualizations of Landsat ARD

• Export plotted Landsat ARD surface reflectance, brightness temperature, and index values to a .csv file

• Save a plot figure as a .png file

• Save a Landsat ARD RGB display as a .png file

• Save ESRI point shapefile for plotted coordinates

---

The tool currently requires the following data inputs:

• Serialized change and cover results generated by PyCCD.

• Landsat ARD (obtained via HTTP request)

Landsat ARD and the PyCCD algorithm are publicly available. However, TAP currently requires access to ARD via a web-service that is currently available to on-site personnel at USGS EROS only.

In addition, the pre-generated PyCCD results must be stored locally and are expected to be in a specific format.

For these reasons, TAP is not intended for general public use until these datasets become widely available at a future date.

---

Installation

Note on PyQt5: It is recommended to use PyQt5==5.10.1 because of an issue in later versions in which Qtwebengineprocess does not close on TAP exit.

---

Create a Virtual Environment

• Install Miniconda3

• Create a Conda virtual environment

  • Linux

    conda env create --name tap python=3.6.7 gdal=2.2.2 poppler=0.60.1 cython cytoolz

  • Windows

    conda create --name tap python=3.6.7 gdal=2.2.2 cython cytoolz

  • If you experience SSL cert errors, disable ssl verification and try to create the env again.

    conda config --set ssl_verify false

• Activate the environment

  conda activate tap

Install TAP

• Ensure the virtual environment is activated from previous creation steps.

• Use pip to install TAP and the remaining dependencies

  pip install lcmap-tap

  • If you are having trusted host/certificate issues during pip install, you can explicitly trust pypi

    pip install lcmap-tap --trusted-host python.pypi.org --trusted-host pypi.org --trusted-host files.pythonhosted.org

• Create the configuration file required by tap

  • Linux Example

    • Create and edit as needed

      cd $HOME/.conda/envs/tap/lib/python3.6/site-packages/lcmap_tap/
      cp config-linux.yaml.example config.yaml

  • Windows Example

    • Create and edit as needed

      cd $HOME\AppData\Local\Continuum\miniconda3\envs\tap\Lib\site-packages\lcmap_tap
      cp config-windows.yaml.example config.yaml

Run the Tool

Once installed, lcmap_tap can be executed directly from the command line if the virtual environment is activated:

• Linux

  lcmap_tap

• Windows

  python.exe $HOME\AppData\Local\Continuum\miniconda3\envs\tap\Scripts\lcmap_tap.exe

Packaging

Packaging tap-tool using PyInstaller for distribution of an executable binary.

Note: These packaging steps must be done from a Windows system.

Pre-Reqs

• The following instructions assume the above Installation steps have already been completed.

• Conda virtual environment is activated

• Download a ZIP archive of the TAP source code for the target tag/branch.

  • Master

  • Develop

• Extract the zipped folder.

• From the command line, cd into the extracted folder.

• Current working directory is inside the extracted lcmap-tap ZIP archive

Instructions

• Install the exe package requirements

  pip install -r requirements-exe.txt

  • If you are having trusted host/certificate issues during pip install, you can explicitly trust pypi

    pip install -r requirements-exe.txt --trusted-host python.pypi.org --trusted-host pypi.org --trusted-host files.pythonhosted.org

• Copy the windows example config

  cp .\lcmap_tap\config-windows.yaml.example .\lcmap_tap\config.yaml

• Build the executable – this will recursively search all imported modules in lcmap_tap to include in the packaged application:

  pyinstaller --noconfirm `
  --add-data ".\lcmap_tap\config.yaml;lcmap_tap" `
  --add-data ".\lcmap_tap\Auxiliary\icon.PNG;lcmap_tap/Auxiliary" `
  --add-data ".\lcmap_tap\MapCanvas\UseWebEngineView;lcmap_tap/MapCanvas/UseWebEngineView" `
  --add-data ".\lcmap_tap\MapCanvas\UseWebView;lcmap_tap/MapCanvas/UseWebView" `
  --hidden-import "PyQt5.sip" `
  --hidden-import "gdal" `
  --hidden-import "ogr" `
  --hidden-import "osr" `
  --hidden-import "merlin" `
  --hidden-import "pkg_resources.py2_warn" `
  run_lcmap_tap.py

• The resulting ‘dist’ folder contains the stand-alone TAP tool

• Zip up this directory for distribution: dist/run_lcmap_tap

• Run TAP simply by double-clicking the executable file, or a shortcut that points to this executable:

  dist\run_lcmap_tap\run_lcmap_tap.exe