Versions
Status
Tools
License
Paper

ELaborative Particle Analysis from Satellite Observations (EL-PASO)

EL-PASO is a Python framework designed to streamline the download, processing, and saving of satellite particle observation data.

Its primary purpose is to prepare and standardize particle data for use in radiation belt modeling.

Features

• Format Flexibility: Capable of handling different input formats including cdf, netcdf, h5, ascii, and json
• Integrated Processing: Provides a comprehensive set of functions for common particle data analysis tasks
• Supports Metadata: Stores all processing and metadata alongside the data, ensuring full traceability and reproducibility.
• Standardized output files: Saving processed data in different standards (e.g. PRBEM) to enable easy loading and sharing of processed data

Full documentation can be viewed here.

Available processing scripts

• Arase
  • MEPe
  • XEP (archived and real-time)
  • PWE density
• DMSP
  • SSJ (electrons)
• GOES-R
  • MPS-High real-time
  • MPS-High
• GPS
  • GPS processing
• ESA
  • NGRM satellites
• POES
  • MEPED (electrons)
  • TED (electrons)
• PROBA-V
  • EPT (electrons and protons)
• Van Allen Probes
  • HOPE (electrons and protons)
  • MagEIS (electrons and protons)
  • RBSPICE (protons)
  • ECT-combined
  • EMFISIS waves
  • EMFISIS and EFW density
• DMSP
  • SSJ (electrons)

Every recipe shares one command line interface; see Running a recipe below.

Installation

Step 1: Clone the Repository

Begin by cloning the EL-PASO repository and navigating into its directory.

git clone https://github.com/GFZ/EL_PASO.git
cd EL_PASO

Step 2: Set up a Python Virtual Environment

It is highly recommended to use a virtual environment to manage dependencies.

uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install .

The custom setup.py script will automatically download and compile the IRBEM Fortran library during this step.

Verifying the Installation

You can validate your installation by running the minimal example located in examples:

python examples/minimal_example.py

[!TIP]

Using the Apptainer Image

Instead of setting up a Python environment yourself, you can pull a prebuilt Apptainer el_paso image:

apptainer pull -F elpaso.sif oras://ghcr.io/gfz/el_paso:latest

Run a command inside the image with apptainer exec or apptainer run, e.g.:

apptainer exec elpaso.sif python examples/minimal_example.py

Available tags mirror the CI build: latest (most recent build on main), a specific commit SHA, or a released package version (e.g. oras://ghcr.io/gfz/el_paso:2.1.2).

Running a recipe

Installing EL-PASO provides the el-paso command, which exposes every recipe under a common set of options:

el-paso list                 # show every available recipe
el-paso poes meped --help    # options for one recipe

el-paso poes meped \
    --start-time 2013-03-16 --end-time 2013-03-16T23:59:59 \
    --satellite noaa15 --mag-field T89 --bin-cadence 10s \
    --raw-data-path ./raw --processed-data-path ./processed

Recipes can equally be run as modules, which is convenient inside job scripts:

python -m el_paso.recipes.poes.process_poes_meped \
    --start-time 2013-03-16 --end-time 2013-03-16T23:59:59

Both forms accept the same options, because both are generated from the recipe function's own signature. --start-time and --end-time are required; the rest have per-recipe defaults. Common options are --satellite (repeat it to process several), --mag-field, --bin-cadence (10s, 5min, 1h), --raw-data-path, --processed-data-path and --num-cores. Add --dry-run to print the resolved arguments without processing anything, and --install-completion to set up shell completion. Which options a given recipe accepts depends on what it actually supports, so check --help.

See the command line documentation for the full list.

Testing

Step 1: Download the Test Data

Most tests rely on reference/system test data hosted on Zenodo. Download it by running the following script from the repository root:

bash download_data_for_tests.sh

This fetches the dataset archive and extracts it into tests/system/. You only need to do this once (rerun it if the data changes upstream).

pytest is installed as part of the regular dependencies (see Installation), so no separate test install step is needed.

Step 2: Run the Tests

Run the full test suite with pytest:

pytest tests

Tests are grouped using pytest markers, defined in pytest.ini:

• basic: quick tests suitable for fast, everyday verification of the code. This is what CI runs on every push/PR:

  pytest tests -m basic
  

• visual: tests that produce plots or other visual output which must be checked manually rather than being asserted automatically:

  pytest tests -m visual
  

You can combine or exclude markers using standard pytest marker expressions, e.g. to run everything except visual tests:

pytest tests -m "not visual"

Some system tests compare against previously stored reference solutions. Pass --renew_solution to regenerate and overwrite those reference solutions instead of comparing against them:

pytest tests --renew_solution=true

Use this only when you intend to intentionally update the stored reference outputs.

Contributing

Please read our comprehensive contribution guide before submitting a pull request.

Citation

If you use EL-PASO in your research, please cite the associated preprint:

Haas, B., Drozdov, A. Y., and Jhawar, S. EL-PASO: An Open-Source Python Library for Processing and Standardizing Particle Measurements Taken in Space. ESS Open Archive. https://essopenarchive.org/doi/full/10.22541/essoar.15002644/v1

@article{
doi:10.22541/essoar.15002644/v1,
author = {Bernhard Haas  and Alexander Y. Drozdov  and Sahil Jhawar },
title = {EL-PASO: An Open-Source Python Library for Processing and Standardizing Particle Measurements Taken in Space},
journal = {ESS Open Archive},
volume = {2026},
number = {0502},
pages = {},
year = {2026},
doi = {10.22541/essoar.15002644/v1},
URL = {https://essopenarchive.org/doi/abs/10.22541/essoar.15002644/v1},
eprint = {https://essopenarchive.org/doi/pdf/10.22541/essoar.15002644/v1}}

To cite this repository you can use CITATION.cff.

Acknowledgements

This work has been funded by the German Research Foundation (NFDI4Earth, DFG project no. 460036893, https://www.nfdi4earth.de/). The authors acknowledge the work of Mátyás Szabó-Roberts who led the foundation for the EL-PASO framework.

The thank the authors of the IRBEM library for providing their code.