Skip to main content

SOBA reference TEST datasets

CI Tests Build conda-forge build Python License: MIT pre-commit Docs

Create Sentinel-1 Wave Mode (WV) reference TEST/TARGET Parquets and a SOBA report from co-aligned catalogues. SWOT runs apply the existing 13 filters. HSCAT runs support wind speed and wind direction; wind-speed runs apply the ECMWF/HSCAT speed-difference filter.

Install

Install the published package with pip:

python -m pip install soba_reference_repo

Or, once the first conda-forge build is published, install it with conda:

conda install -c conda-forge soba_reference_repo

For development from a checkout, install it from the repository root instead:

python -m pip install -e .

Run a SWOT recipe

  1. Copy the example. Keep your edited recipe in runs/ so it stays out of Git:

    mkdir -p runs
    cp examples/swot-curation.toml runs/recipe.toml
    
  2. Edit runs/recipe.toml: replace the S1A, S1C, and S1D path values with your WV SWOT Parquet files. Check production_date, version, and output_dir. The output directory must be new; the tool will not overwrite a run. Set compile = true if you have pdflatex or Tectonic and want a PDF. Otherwise, it writes the editable LaTeX report without compiling it.

  3. Run:

    soba_create_test_dataset --recipe runs/recipe.toml
    

The recipe selects SWOT WV or HSCAT. The old soba_reference_repo command remains an alias. The Python client is now soba_reference_repo.create_test_dataset.

The example writes to runs/swot-WV-20260930-0.1/ (paths resolve relative to the recipe). The run contains:

recipe.toml                 Copy of the run settings
curated/                    One Curated Parquet per satellite
datasets/                   Paired TEST and TARGET Parquets
report/                     TeX, optional PDF, and images_<report-stem>/ assets
manifest.json               Source files, rules, counts, and output paths

Curated filenames retain the source catalogue coverage dates, polarization, and reference product; production date and version come from the recipe.

Reports describe Curated observations, keep sequential filter counts, and document the generated observation schema without assigning a final split. TeX and image-folder names stay versionless; only PDFs carry the recipe version. See report details.

Each satellite starts with 13 predefined SWOT filters. A [[catalogues.rules]] entry changes a default filter when its id matches a default rule; a new id adds a filter after the defaults. The example includes commented overrides and additions. Filters run in order: report counts read remaining / removed after each filter. Curated counts can exceed TEST counts because the exporter also checks required fields and duplicate keys. A satellite can contribute zero rows if no input row passes its filters. Recipe rules support UTC date_gt and date_lt comparisons: they select timestamps strictly later or earlier than midnight UTC on the stated date; invalid or missing timestamps do not match. SCAT rule clauses also support abs_diff_le for absolute differences between two columns.

Run an HSCAT recipe

Copy examples/hscat-curation.toml to runs/, update the paths, and choose windspeed or winddirection. The example rejects imagettes whose eligible SCAT matchup directions have circular STD above 10°. Keep this configurable rule last. Existing duplicate-key handling is unchanged. See usage details.

Build a CHALLENGER

Copy examples/cci-seastate-challenger.toml, set its paths and run settings, then run:

soba_produce_challenger --recipe path/to/challenger.toml

Validate existing Parquets

Validate supported TEST, TARGET, and CHALLENGER files individually:

soba_validate_parquets --type test --file path/to/SWOT_TEST.parquet
soba_validate_parquets --type target --file path/to/SWOT_TARGET.parquet
soba_validate_parquets --type test --file path/to/SCAT_TEST.parquet --reference scat
soba_validate_parquets --type target --file path/to/SCAT_TARGET.parquet --reference scat
soba_validate_parquets --type challenger --file path/to/CHALLENGER.parquet

--reference swot is the default. Use --reference scat for SCAT TEST/TARGET files.

The default report shows the filename, row and column counts, and a [PASS] checklist of completed checks. It also identifies skipped checks and single-file limitations. Failed validation reports the error without printing a success checklist.

The validator accepts one file at a time through --type and --file. Each report includes a Schema checked: line with the checked column names and accepted types. Single-file validation does not verify alignment with another file.

Development

Run python -m pytest -q in an environment with the development dependencies. The SOBA report style and logo live in assets/latex/.

Licensed under MIT.

Metadata

Release files for soba-reference-repo 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for soba-reference-repo 0.3.0
File Size Uploaded Provenance Actions
soba_reference_repo-0.3.0.tar.gz 491.5 kB With provenance Details for soba_reference_repo-0.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for soba-reference-repo 0.3.0
File Interpreter ABI Platform Actions
soba_reference_repo-0.3.0-py3-none-any.whl Python 3 none any Details for soba_reference_repo-0.3.0-py3-none-any.whl

Total release size: 961.9 kB

Release files / soba_reference_repo-0.3.0.tar.gz

Download URL soba_reference_repo-0.3.0.tar.gz
Size 491.5 kB
Tags Source
SHA-256 checksum
How to use checksums
898db148b699adb5550271db14a967fde7f88071bee82baf7f85761c2e001add
BLAKE2b-256 checksum
How to use checksums
5ca87ab4bd4eb8b4914597425bc8177445bec6c933fd80beeeee567226402c03
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 15, 2026.

Transparency log

Release files / soba_reference_repo-0.3.0-py3-none-any.whl

Download URL soba_reference_repo-0.3.0-py3-none-any.whl
Size 470.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ed3a618276087aa86ae42923dfeb4977220521de2a2f398039daff396c10e09f
BLAKE2b-256 checksum
How to use checksums
c649bb2a5e15d9a5a0b6916f9ceed1b4c90d2a649b327f25a4bef1d53113d551
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 15, 2026.

Transparency log

Security for soba-reference-repo 0.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for soba_reference_repo-0.3.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 15, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page