PySDS: Python Soil Dielectric Simulator

An automatic framework for generating benchmark datasets of spatially coherent soil dielectric constant (DC) maps from soil organic carbon (SOC) maps, using the temperature-dependent multi-relaxation spectroscopic dielectric model (TD-MRSDM).

About the project

Surface soil moisture (SSM) retrieval from synthetic aperture radar (SAR) imagery can be posed as a two-dimensional inverse scattering problem, which needs spatially coherent soil DC maps as priors, training targets or inputs. PySDS generates such maps automatically. It projects the point-scale TD-MRSDM, in the version of Mironov and Savin (2019), onto SOC maps, so that the spatial structure of SOC carries into the simulated real (ε′) and imaginary (ε″) parts of the soil DC.

The workflow (figure above):

1. Inputs: a SOC map (g/kg) as the only spatial input, plus user-defined ranges of SSM and soil surface temperature (SST) and a radar frequency.
2. Derived soil properties: soil organic matter (SOM) is derived from SOC with the van Bemmelen factor (SOM = 1.724 × SOC), and soil bulk density (SBD) with a SOC-based pedotransfer function (PTF).
3. Simulation: the TD-MRSDM, implemented in the companion tdmrsdm package (PyTD-MRSDM) with constants optimized against the dielectric database of organic Arctic soils (DDOAS), computes the soil DC for each SSM, SST and frequency scenario.
4. Output: soil DC maps (ε′ and ε″), saved as patches in HDF5 or NPZ datasets.

The underlying model covers radar frequencies from 10 MHz to 16 GHz, SST from −30 °C to +25 °C, and moisture up to 0.6 cm³/cm³. The accompanying paper evaluates its reliability across frequency, SSM, SST and SBD: ε′ is reliable under thawed conditions and at operational microwave bands, whereas ε″ is less certain at low frequencies, near saturation, in frozen soils and near the freeze–thaw transition. Because the model was developed for Arctic organic soils and SOM and SBD are derived from SOC through empirical relations, transferability to other soils is not guaranteed. PySDS does not retrieve SSM itself; it provides the dielectric data that SAR inversion methods need.

Key features

• Spatially coherent soil DC maps: ε′ and ε″ maps that keep the SOC-driven soil heterogeneity of the input map.
• Frozen and thawed soils: bound, transient and unbound/ice water forms, through the TD-MRSDM.
• Multi-condition datasets: several (SST, SSM) conditions per geographic patch, at a chosen radar frequency.
• Stratified sampling (optional): balances the dataset across the ε′–ε″ space.
• Satellite segmentation (optional): segments each patch from satellite imagery with SAM (Segment Anything) through segment-geospatial.
• Command-line tools: pysdsim for dataset generation and pysdsim-report for statistical reports.

Installation

PySDS requires Python ≥ 3.8.

pip install pysdsim

or the latest version from GitHub:

pip install "git+https://github.com/Morteza-Khazaei/PySDS.git"

PySDS requires tdmrsdm ≥ 2026.10.0, which ships the optimized v3 TD-MRSDM constants; pip installs it automatically.

Usage

As a library

import numpy as np
import rasterio
from rasterio.transform import from_origin
from pysdsim import Simulator

# 1. Create a small synthetic SOC map (g/kg); replace it with your own SOC raster
soc_map_path = "dummy_soc_map.tif"
soc = (10 + 50 * np.random.rand(256, 256)).astype(np.float32)
profile = {"driver": "GTiff", "height": 256, "width": 256, "count": 1, "dtype": "float32",
           "crs": "EPSG:32632", "transform": from_origin(500000, 5500000, 10, 10)}
with rasterio.open(soc_map_path, "w", **profile) as dst:
    dst.write(soc, 1)

# 2. Configure the simulator
sim = Simulator(
    workspace="pysds_output",       # folder created in your home directory (Google Drive on Colab)
    dataset_size=5,                 # number of patches
    patch_size=64,                  # 64 x 64 pixel patches
    soc_path=soc_map_path,
    freq_GHz=5.405,                 # C-band (Sentinel-1)
    sst_range=(-30, 30, 5),         # SST (°C): -30, -25, ..., 25 (stop value excluded)
    ssm_range=(0.05, 0.65, 0.05),   # SSM (cm³/cm³): 0.05, 0.10, ..., 0.60 (stop value excluded)
    no_data_threshold=0.1,          # maximum fraction of NoData pixels per patch
    enable_segmentation=False,      # True downloads satellite imagery and needs SAM
)

# 3. Generate the dataset
sim.run()

Ranges are given as (start, stop, step) and expanded with numpy.arange, so the stop value is not included.

The dataset is written to ~/pysds_output/PySDS_output/, together with an HTML statistical report. In the library, stratified_sampling is off by default, so each patch gets one randomly drawn (SST, SSM) condition; set stratified_sampling=True to simulate several conditions per patch, as the command-line tool does by default.

From the command line

pysdsim path/to/soc_map.tif \
    --workspace MyPySDS_Workspace \
    --nsize 1000 \
    --patch-size 128 \
    --frequency 5.405 \
    --sst_range -30 30 5 \
    --ssm_range 0.05 0.65 0.05

This simulates 1000 patches of 128 × 128 pixels at C-band (5.405 GHz), with SST from −30 °C to 25 °C in 5 °C steps and SSM from 0.05 to 0.60 cm³/cm³ in 0.05 steps. Several (SST, SSM) conditions are simulated per patch. By default, conditions are selected by stratified sampling, which fills the ε′–ε″ space evenly by keeping a quota of samples per dielectric bin (--samples-per-bin) and a maximum number of conditions per patch (--max-per-patch); use --no-stratified for uniform random sampling.

Full list of options (pysdsim --help):

usage: pysdsim [-h] [-n DL_DB_NAME] [-w WORKSPACE] [-s NSIZE] [-p PATCH_SIZE]
               [-o OVERLAP OVERLAP] [-f FREQUENCY]
               [-t SST_RANGE SST_RANGE SST_RANGE]
               [-m SSM_RANGE SSM_RANGE SSM_RANGE] [-l NO_DATA_THRESHOLD]
               [-b MAX_SEARCH_DISTANCE] [-i SMOOTHING_ITERATIONS]
               [--save-format {hdf5,npz}] [--stratified] [--no-stratified]
               [--samples-per-bin SAMPLES_PER_BIN]
               [--max-per-patch MAX_PER_PATCH] [--enable-segmentation]
               [--sam-checkpoint SAM_CHECKPOINT]
               [--sam-model-type {vit_h,vit_l,vit_b}] [--gs-zoom GS_ZOOM]
               [--gs-source GS_SOURCE] [-v] [--version]
               soc_path

Output files

Datasets are named automatically from their parameters:

SynthSoilDiel_f<freq>GHz_p<size>x<count>_multiCond_SST<start><stop>s<step>C_SSMp<start>p<stop>s<step>_<seg>_v1.<ext>

The command-line example above produces SynthSoilDiel_f5p405GHz_p128x1000_multiCond_SSTm30p30s5C_SSMp05p65s05_withSeg_v1.h5. The SST and SSM parts record the arguments as given, including the excluded stop value.

Each patch stores the complex soil DC for every simulated condition, the SOC map, the SST and SSM values, the patch extent (WGS 84), summary statistics of the inputs and outputs, and the pedotransfer functions used, so that SOM and SBD can be reproduced from SOC.

Statistical report

pysdsim-report <path_to_h5_file> [--bins <percentiles>] [--output-dir <directory>]

• --bins: percentile bins for the distribution report (default 0 10 25 50 75 90 100).
• --output-dir: output directory (default pysdsim_report_output).

For example,

pysdsim-report ~/MyPySDS_Workspace/PySDS_output/SynthSoilDiel_f5p405GHz_p128x1000_multiCond_SSTm30p30s5C_SSMp05p65s05_withSeg_v1.h5 --output-dir my_report

writes my_report/SynthSoilDiel_f5p405GHz_p128x1000_multiCond_SSTm30p30s5C_SSMp05p65s05_withSeg_v1_report.html, with descriptive statistics, percentile analysis, outlier detection (IQR and z-score) and plots (histograms, scatter plots, Q–Q plots and a heatmap of the patch distribution).

Large SOC rasters are read in blocks of rows, so maps larger than the available memory can be processed.

Example datasets

Two example soil DC datasets generated by PySDS, with patch sizes of 64 and 128 pixels, are available on Zenodo: https://doi.org/10.5281/zenodo.15998448.

Contributing

Contributions are welcome:

1. Fork the project.
2. Create a feature branch (git checkout -b feature/AmazingFeature).
3. Commit your changes (git commit -m 'Add some AmazingFeature').
4. Push the branch (git push origin feature/AmazingFeature).
5. Open a pull request.

License

Distributed under the MIT License. See LICENSE for details.

Citation

If you use PySDS or the datasets it generates, please cite the accompanying paper (submitted to Big Earth Data):

@article{khazaei_pysds,
  title   = {An Automatic Framework for Generating Benchmark Datasets of Soil Dielectric Constant Maps},
  author  = {Khazaei, Morteza and Magagi, Ramata and Go{\"i}ta, Kalifa and Karavayskiy, Andrey Y.},
  journal = {Big Earth Data},
  note    = {Submitted}
}

References

• Mironov, V., & Savin, I. (2015). A temperature-dependent multi-relaxation spectroscopic dielectric model for thawed and frozen organic soil at 0.05–15 GHz. Physics and Chemistry of the Earth, Parts A/B/C, 83, 57–64. https://doi.org/10.1016/j.pce.2015.02.011
• Mironov, V., & Savin, I. (2016). Temperature-dependent spectroscopic dielectric model at 0.05–16 GHz for a thawed and frozen Alaskan organic soil. In Satellite Soil Moisture Retrieval (pp. 169–186). Elsevier. https://doi.org/10.1016/B978-0-12-803388-3.00009-7
• Mironov, V., & Savin, I. (2019). Spectroscopic multirelaxation dielectric model of thawed and frozen Arctic soils considering the dependence on temperature and organic matter content. Izvestiya, Atmospheric and Oceanic Physics, 55, 986–995. https://doi.org/10.1134/S0001433819090305
• Savin, I., Mironov, V., Muzalevskiy, K., Fomin, S., Karavayskiy, A., Ruzicka, Z., & Lukin, Y. (2020). Dielectric database of organic Arctic soils (DDOAS). Earth System Science Data, 12, 3481–3487. https://doi.org/10.5194/essd-12-3481-2020