pyxccd 0.1.6

A PYthon library for latest and eXtended COntinuous Change Detection algorithms

PYXCCD

A PYthon library for latest and eXtended Continuous Change Detection

Author: Su Ye ([e-mail])

The Continuous Change Detection and Classification (CCDC) algorithm has been popular for processing satellite-based time series datasets, particularly for Landsat-based datasets. As a CCDC user, you may already be familiar with the existing CCDC tools such as pyccd and gee ccdc.

Wait.. so why does the pyxccd package still exist?

We developed pyxccd mainly for the below purposes:

1. Near real-time monitoring: Implements the unique S-CCD algorithm, which recursively updates model coefficients and enables timely change detection.

2. Latest CCDC (COLD): Integrates the advanced COLD algorithm, offering the highest retrospective breakpoint detection accuracy to date, validated against Zhe’s MATLAB version.

3. Efficient Large-scale time-series processing: The core of pyxccd is written in C language, ensuring high computational efficiency and low memory usage in the desktop as well as HPC environments.

4. Flexible multi-sensor support: Supports arbitrary band combinations from diverse sensors (e.g., Sentinel-2, MODIS, GOSIF, and SMAP) in addition to Landsat.

5. State-space model incoporation: S-CCD allows modeling trend and seasonal signals as time-varying variables (namely “states”) guided by break detection, enabling (a) characterization of subtle inter-segment variations (e.g., phenological shifts) and (b) gap filling that accounts for land cover conversions (temporal breaks).

1. Installation

pip install pyxccd

Note: the installation has been cross-platform (Windows, Linux and MacOS). Contact the author ([e-mail]) if you have problems for installation

2. Using pyxccd for pixel-based processing

COLD:

from pyxccd import cold_detect
import pandas as pd
data = pd.read_csv('tutorial/datasets/1_hls_sc.csv')
dates, blues, greens, reds, nirs, swir1s, swir2s, thermals, qas, sensor = data.to_numpy().copy().T
cold_result = cold_detect(dates, blues, greens, reds, nirs, swir1s, swir2s, thermals, qas)

COLD algorithm for any combination of band inputs from any sensor:

from pyxccd import cold_detect_flex
# input a user-defined array instead of multiple lists
cold_result = cold_detect_flex(dates, np.stack((reds, nirs, swir1s), axis=1), qas, lambda=20,tmask_b1_index=1, tmask_b2_index=2)

S-CCD:

# require offline processing for the first time
from pyxccd import sccd_detect, sccd_update
sccd_pack = sccd_detect(dates, blues, greens, reds, nirs, swir1s, swir2s, qas)

# then use sccd_pack to do recursive and short-memory NRT update
sccd_pack_new = sccd_update(sccd_pack, dates, blues, greens, reds, nirs, swir1s, swir2s, qas)

S-CCD for outputting continuous seasonal and trend states:

# open state output (state_ensemble) by setting state_intervaldays as a non-zero value
sccd_result, state_ensemble = sccd_detect(dates, blues, greens, reds, nirs, swir1s, swir2s, qas, state_intervaldays=1)

3. Documentation

API documents: readthedocs

Tutorial: under development

4. Citations

If you make use of the algorithms in this repo (or to read more about them), please cite (/see) the relevant publications from the following list:

[S-CCD] Ye, S., Rogan, J., Zhu, Z., & Eastman, J. R. (2021). A near-real-time approach for monitoring forest disturbance using Landsat time series: Stochastic continuous change detection. Remote Sensing of Environment, 252, 112167.

[COLD] Zhu, Z., Zhang, J., Yang, Z., Aljaddani, A. H., Cohen, W. B., Qiu, S., & Zhou, C. (2020). Continuous monitoring of land disturbance based on Landsat time series. Remote Sensing of Environment, 238, 111116.

The recent applications of S-CCD could be found in CONUS Land Watcher

Q&A

Q1: Has pyxccd been verified?

Re: Multiple rounds of verification have been conducted. A comparison based on two testing tiles indicates that differences between pyxccd and the MATLAB implementation are minimal, with discrepancies of less than 2% in both breakpoint detection and harmonic coefficients. Furthermore, the accuracy of pyxccd was evaluated against the same reference dataset used in the original COLD study (Zhu et al., 2020). The results demonstrate that COLD in pyxccd achieves equivalent accuracy (27% omission and 28% commission), confirming that the observed discrepancies do not compromise performance. The primary source of the discrepancy stems from numerical precision: MATLAB employs float64, whereas pyxccd uses float32 to reduce memory consumption and improve computational efficiency.

Q2: how much time for production of a tile-based disturbance map (5000*5000 pixels) using pyxccd?

Re: I tested COLD in UCONN HPC environment (200 EPYC7452 cores): for processing a 40-year Landsat ARD tile (1982-2021), the stacking typically takes 15 mins; per-pixel COLD processing costs averagely 1 hour, while per-pixel S-CCD processing costs averagely 0.5 hour; exporting maps needs 7 mins.