Py-Rate 0.4.3

A Python tool for estimating velocity and cumulative displacement time-series from Interferometric Synthetic Aperture Radar (InSAR) data.

Python tool for InSAR Rate and Time-series Estimation

PyRate is a Python tool for estimating the average displacement rate (velocity) and cumulative displacement time-series of surface movements for every pixel in a stack of geocoded unwrapped interferograms generated by Interferometric Synthetic Aperture Radar (InSAR) processing. PyRate uses a “Small Baseline Subset” (SBAS) processing strategy and currently supports input data in the GAMMA or ROI_PAC software formats.

The PyRate project started in 2012 as a partial Python translation of “Pirate”, a Matlab tool developed by the University of Leeds and the Guangdong University of Technology.

The full PyRate documentation is available at http://geoscienceaustralia.github.io/PyRate

Dependencies

The following dependencies need to be on your system prior to PyRate installation:

• Python, versions 3.6, 3.7 or 3.8.

• GDAL, versions 3.0.2 or 3.0.4

• Open MPI, versions 2.1.6, 3.0.4, 3.1.4 or 4.0.2

The versions of each package stated above have been tested to work.

Python dependencies are:

Cython==0.29.16
Pillow==7.1.1
glob2==0.7
joblib==0.14.1
mpi4py==3.0.3
networkx==2.4
numpy==1.18.3
pyproj==2.6.0
scipy==1.4.1
numexpr==2.7.1

Install

Details of all install options are given in the PyRate documentation.

PyRate and its Python dependencies can be installed directly from the Python Package Index (PyPI):

pip install Py-Rate

To install from source in Linux, enter these commands in a terminal:

cd ~
git clone https://github.com/GeoscienceAustralia/PyRate.git
python3 -m venv ~/PyRateVenv
source ~/PyRateVenv/bin/activate
cd ~/PyRate
python3 setup.py install

This will install the above-listed Python dependencies and compile the executable program pyrate:

>> pyrate --help
usage: pyrate [-h] [-v {DEBUG,INFO,WARNING,ERROR}]
              {conv2tif,prepifg,process,merge,workflow} ...

PyRate workflow:

    Step 1: conv2tif
    Step 2: prepifg
    Step 3: process
    Step 4: merge

Refer to https://geoscienceaustralia.github.io/PyRate/usage.html for
more details.

positional arguments:
  {conv2tif,prepifg,process,merge,workflow}
    conv2tif            Convert interferograms to geotiff.
    prepifg             Perform multilooking and cropping on geotiffs.
    process             Main processing workflow including corrections, time series and stacking computation.
    merge               Reassemble computed tiles and save as geotiffs.
    workflow            Run all the PyRate processes

optional arguments:
  -h, --help            show this help message and exit
  -v {DEBUG,INFO,WARNING,ERROR}, --verbosity {DEBUG,INFO,WARNING,ERROR}
                        Increase output verbosity

Documentation

The full documentation is at http://geoscienceaustralia.github.io/PyRate/

Release History

0.4.3 (2020-08-04)

Added

• Ability to define the order of steps in the process workflow (default order unchanged).

Fixed

• Nil

Changed

• prepifg output interferograms are saved as read-only files.

• process makes a writable copy of the prepifg output data at the beginning of each run.

• The selected reference pixel is saved to disk and re-used on subsequent process runs.

• Saving of incremental time series (tsincr) products is optional, controlled by the savetsincr configuration parameter (default is on).

Removed

• Removed obsolete InSAR terminology from code, docs and test data (changed to first and second images).

• Stopped using unittest unit test framework in favour of exclusively using pytest.

0.4.2 (2020-06-26)

Added

• Save full-res coherence files to disk in conv2tif step if cohmask = 1.

• Save multi-looked coherence files to disk in prepifg step if cohmask = 1.

• Additional DATA_TYPE geotiff header metadata for above coherence files.

• conv2tif and prepifg output files have a tag applied to filename dependent on data type, i.e. _ifg.tif, _coh.tif, _dem.tif.

• Metadata about used reference pixel is added to interferogram geotiff headers: lat/lon and x/y values; mean and standard deviation of reference window samples.

• Quicklook PNG and KML files are generated for the Stack Rate error map by default.

Fixed

• Ensure prepifg treats input data files as read only.

• Fix the way that the reference phase is subtracted from interferograms during process step.

• Manual entry of refx/y converted to type int.

Changed

• User supplies latitude and longitude values when specifying a reference pixel in the config file. Pixel x/y values are calculated and used internally.

• Move Stack Rate masking to a standalone function pyrate.core.stack.mask_rate, applied during the merge step and add unit tests.

• Skip Stack Rate masking if threshold parameter maxsig = 0.

• Provide log message indicating the percentage of pixels masked by pyrate.core.stack.mask_rate.

• Refactor pyrate.core.stack module; expose two functions in documentation: i) single pixel stacking algorithm, and ii) loop function for processing full ifg array.

• Refactor pyrate.merge script; remove duplicated code and create reusable generic functions.

• Colourmap used to render quicklook PNG images is calculated from min/max values of the geotiff band.

• Updated test and dev requirements.

Removed

• Deprecate unused functions in pyrate.core.config and corresponding tests.

• Static colourmap utils/colourmap.txt that was previously used to render quicklook PNG images is removed.

0.4.1 (2020-05-19)

Added

• Python 3.8 support.

• Algorithm to automatically calculate rows and columns for tiling. User no longer specifies these as part of the CLI, but can optionally specify rows and cols in the configuration file.

• Improvements to the test suite, including systems-wide tests.

• Improved logging.

Fixed

• Fixed bug in resampling/multi-looking when coherence masking is used. This bugfix will result in significantly fewer nan pixels in the outputs.

• Fixed a bug in how NaNs are handled during coherence masking and multi-looking. Output rasters will contain nan as the nodata value.

Changed

• Linear Rate algorithm has been renamed Stack Rate.

• User supplies full paths to input files in respective file lists.

• All files generated by PyRate saved to user-defined outdir directory.

• Renamed slcfilelist parameter to hdrfilelist.

• Log files are generated in the outdir and every PyRate step produces independent log files.

Removed

• Deprecate the use of obsdir, slcfiledir and cohdir configuration variables.

• Deprecate parallel = 2 option; splitting image via rows for parallelisation.

0.4.0 (2019-10-31)

Added

• Python 3.7 support.

• Optional conv2tif step.

• Building of docs integrated with Travis CI.

• Coherence masking, view coherence masking section in input_parameters.conf for options.

• Input parameter validation.

• SLC and coherence file lists for file discovery.

• Create quick view png for rate map product.

• Add support for reading interferogram in Geotiff format.

• Add detailed validation and hints for configuration parameters

• Add system tests for all 3 types of input formats

Changed

• linrate step has been renamed to process.

• postprocess step has been renamed to merge.

• converttogeotiff step has been renamed to conv2tif.

• CLI structure: config files now need to be provided with -f flag.

• Reduced console output, default verbosity setting is now INFO.

• Restructure of code layout, src modules now in PyRate/pyrate/core directory and scripts at PyRate/scripts.

• Reference pixel values are expected to be in latitude and longitude values.

Removed

• Unused luigi code.

• References to Matlab.

• Unused tests for legacy api.

0.3.0 (2019-07-26)

Added

• utils/apt_install.sh script that lists Ubuntu/apt package requirements.

• utils/load_modules.sh script that sets up NCI Raijin HPC environment.

Fixed

• Errors being caused by newer version of networkx; v2.3 now supported.

Removed

• Unused Python and OS packages.

• environment.yml - conda env will now be installed using requirements.txt.

• HPC directory - hpc README.rst moved to docs.

• setup.cfg - no longer needed.

• Luigi functionality - hasn’t been operational and is reported as vulnerable. Single machine parallelism is achieved with joblib.

Changed

• Requirements now managed by requirements.txt file, parsed by setup.py.

• Requirements now split across base requirements.txt and separate files for dev (requirements-dev.txt) and testing (requirements-test.txt).

• Moved default config files to top level source directory.

• Pinned Python dependencies to specific versions.

• Travis build now installs GDAL from apt.

• Travis only builds on master, develop and *-travis branches.

• Consolidated documentation into PyRate/docs.

• Updated install instructions for Ubuntu and NCI.

0.2.0 (2017-05-22)

• Stable beta release.

0.1.0 (2017-01-31)

• First release on PyPI.