Skip to main content

EIDA consistency checker: compare availability vs dataselect services and generate reports.

Project description

EIDA Consistency Checker

Run Tests Coverage Docs


A tool to evaluate the consistency between EIDA nodes' availability and dataselect web services.
Designed for use in quality control and monitoring tasks across the European Integrated Data Archive (EIDA).


🚀 Installation & Quick Start

You can use eida-consistency in two ways:

Option 1: Using uvx

uvx eida-consistency <command> [options]

Option 2: Global install

uv tool install eida-consistency

Now it’s available globally:

eida-consistency <command> [options]

Update it with:

uv tool upgrade eida-consistency

🔍 CLI Commands

Run Consistency Check

Check if availability spans align with dataselect results:

uvx eida-consistency consistency --node RESIF --epochs 10 --duration 600
# OR check 5% of available stations:
uvx eida-consistency consistency --node RESIF --epochs "5%"

Options:

  • --node: Node code (e.g. RESIF, NOA, ETH)
  • --epochs: Number of random test epochs (default: 10) OR percentage (e.g., "5%", 0.05)
  • --duration: Epoch length in seconds (≥600)
  • --seed: Reproducible seed
  • --delete-old: Keep only the most recent report
  • --stdout: Print JSON report to stdout
  • --report-dir: Save reports to a custom folder (default: reports/)
  • --log-level: Control verbosity (DEBUG, INFO, WARNING, ERROR)

Compare Reports

Compare results across two runs with the same seed:

uvx eida-consistency compare reports/resif_run1.json reports/resif_run2.json

Explore Inconsistencies

Drill down into inconsistencies with day-by-day exploration:

uvx eida-consistency explore reports/nodes/noa/*.json --index 7 --days 15 --verbose

You can also use:

uvx eida-consistency explore --latest

to automatically use the newest report.

Manage Node List

Reload routing cache:

uvx eida-consistency reload-nodes

List currently cached nodes:

uvx eida-consistency list-nodes

📂 Reports

Reports are stored in ./reports/ by default, or in a custom folder using --report-dir.

  • JSON reports: reports/resif_<seed>.json
  • Markdown reports: reports/resif_<seed>.md
  • Global summary: summary.md

📚 Library Usage

You can also use eida-consistency as a Python library to run checks programmatically or build custom tools.

Check a Single Candidate

from eida_consistency.core.checker import check_candidate

# Define a candidate (network, station, channel, starttime)
candidate = {
    "network": "GR",
    "station": "ATH",
    "channel": "BHZ",
    "starttime": "2023-01-01T00:00:00",
}

# Run the check
results, stats = check_candidate(
    base_url="http://node.eida.eu/fdsnws",
    candidate=candidate,
    epochs=5,
    duration=600
)

for res in results:
    url, available, start, end, loc, span = res
    print(f"Time: {start} -> Available: {available}")

Run a Full Consistency Check

from eida_consistency.runner import run_consistency_check

# Run check for a specific node and get the report path
report_path = run_consistency_check(
    node="NOA",
    epochs=10,
    duration=600
)
print(f"Report generated at: {report_path}")

For full API documentation, please visit our Documentation Site or run:

uv run mkdocs serve

🧪 Example Workflow

1. Run a check for NOA:

uvx eida-consistency consistency --seed 1234 --node NOA --epochs 20 --duration 600 --report-dir reports/test_noa

2. Explore incosistencies:

If inconsistencies are found:

uvx eida-consistency explore reports/nodes/resif/*.json --verbose

3. Apply fixes

Investigate the reported service inconsistencies and fix them at node level. Maybe use dmtri command suggested.

4. Re-run Consistency

uvx eida-consistency consistency --seed 1234 --node NOA --epochs 20 --duration 600 --report-dir reports/test_noa

5.Compare Before/After

uv run eida-consistency compare reports/test_noa/old.json reports/test_noa/new.json

📊 Global Consistency Summary

Node Epochs Requested Epochs Usable Total Checks Consistent Inconsistent Score
BGR 20 20 20 15 5 75.0 %
BGS 20 11 11 6 5 54.55 %
ETH 20 20 20 19 1 95.0 %
GEOFON 20 20 20 20 0 100.0 %
KOERI 20 20 20 10 10 50.0 %
LMU 20 20 20 20 0 100.0 %
NIEP 20 20 20 15 5 75.0 %
NOA 20 20 20 16 4 80.0 %
RESIF 20 20 20 20 0 100.0 %
UIB-NORSAR 20 20 20 18 2 90.0 %

📖 Full details: summary.md


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

eida_consistency-0.4.0.tar.gz (293.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

eida_consistency-0.4.0-py3-none-any.whl (37.1 kB view details)

Uploaded Python 3

File details

Details for the file eida_consistency-0.4.0.tar.gz.

File metadata

  • Download URL: eida_consistency-0.4.0.tar.gz
  • Upload date:
  • Size: 293.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.8

File hashes

Hashes for eida_consistency-0.4.0.tar.gz
Algorithm Hash digest
SHA256 331634cd7b9bbeffdf0cba92b1cc1ea3db3e99826eeb60c6e271766edfa52ade
MD5 68b5c95da8f0fee50a33d9eb57c9d7ac
BLAKE2b-256 ca56daf3bbb18234a36c7b2795521d8894183916fc377ad4513b797b8434a50f

See more details on using hashes here.

Provenance

The following attestation bundles were made for eida_consistency-0.4.0.tar.gz:

Publisher: publish.yml on EIDA/eida-consistency

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file eida_consistency-0.4.0-py3-none-any.whl.

File metadata

File hashes

Hashes for eida_consistency-0.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 d3f5218e3ed9c4668715318df1d4dcc7dd3cb20f68567e572f83ae02b31e010b
MD5 97d6290780f92a97c8931f62e0d62706
BLAKE2b-256 9d356d22f78d2863eb21ab379fcea0f35f4ed017bcebaa8b9ed88eb5ec92a0e2

See more details on using hashes here.

Provenance

The following attestation bundles were made for eida_consistency-0.4.0-py3-none-any.whl:

Publisher: publish.yml on EIDA/eida-consistency

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page