Skip to main content

Data loader for Solar Orbiter/EPD energetic charged particle sensors EPT, HET, and STEP.

Project description

pypi Version conda version python version pytest codecov license zenodo doi

Python data loader for Solar Orbiter’s (SolO) Energetic Particle Detector (EPD). At the moment provides level 2 (l2) and low latency (ll) data (more details on data levels here) obtained through CDF files from ESA’s Solar Orbiter Archive (SOAR) for the following sensors:

  • Electron Proton Telescope (EPT)

  • High Energy Telescope (HET)

  • SupraThermal Electrons and Protons (STEP)

Current caveats:

  • Only the standard rates data products are supported (i.e., no burst or high cadence data).

  • For EPT and HET, only electrons, protons and alpha particles are processed (i.e., for HET He3, He4, C, N, O, Fe are omitted at the moment).

  • For STEP, electron data needs to be calculated manually.

  • The Suprathermal Ion Spectrograph (SIS) is not yet included.

Disclaimer

This software is provided “as is”, with no guarantee. It is no official data source, and not officially endorsed by the corresponding instrument teams. Please always refer to the official EPD data description before using the data!

Installation

solo_epd_loader requires python >= 3.6.

It can be installed either from PyPI using:

pip install solo-epd-loader

or from Anaconda using:

conda install -c conda-forge solo-epd-loader

Usage

The standard usecase is to utilize the epd_load function, which returns Pandas dataframe(s) of the EPD measurements and a dictionary containing information on the energy channels.

from solo_epd_loader import epd_load

df_1, df_2, energies = epd_load(sensor, startdate, enddate=None, level='l2', viewing=None, path=None,
                                autodownload=False, only_averages=False)

Input

  • sensor: 'ept', 'het', or 'step' (string)

  • startdate, enddate: Datetime object (e.g., dt.date(2021,12,31) or dt.datetime(2021,4,15)) or integer of the form yyyymmdd with empty positions filled with zeros, e.g. 20210415 (if no enddate is provided, enddate = startdate will be used)

  • level: 'l2' or 'll' (string); defines level of data product: level 2 ('l2') or low-latency ('ll'). By default 'l2'.

  • viewing: 'sun', 'asun', 'north', 'south', 'omni' (string) or None; not needed for sensor = 'step'. 'omni' is just calculated as the average of the other four viewing directions: ('sun'+'asun'+'north'+'south')/4

  • path: directory in which Solar Orbiter data is/should be organized; e.g. '/home/userxyz/solo/data/' (string). See Data folder structure for more details.

  • autodownload: if True, will try to download missing data files from SOAR (bolean)

  • only_averages: If True, will for STEP only return the averaged fluxes, and not the data of each of the 15 Pixels. This will reduce the memory consumption. By default False.

Return

  • For sensor = 'ept' or 'het':

    1. Pandas dataframe with proton fluxes and errors (for EPT also alpha particles) in ‘particles / (s cm^2 sr MeV)’

    2. Pandas dataframe with electron fluxes and errors in ‘particles / (s cm^2 sr MeV)’

    3. Dictionary with energy information for all particles:

      • String with energy channel info

      • Value of lower energy bin edge in MeV

      • Value of energy bin width in MeV

  • For sensor = 'step':

    1. Pandas dataframe with fluxes and errors in ‘particles / (s cm^2 sr MeV)’

    2. Dictionary with energy information for all particles:

      • String with energy channel info

      • Value of lower energy bin edge in MeV

      • Value of energy bin width in MeV

SupraThermal Electron Proton (STEP) sensor electron measurements

Please note that the STEP electron measurements are not directly provided in the publically released data, but need to be calculated from them. This process is not straightforward, and the resulting data is prone to uncertainties (like contamination). Thus it should only be used scientifically with caution! Please refer to the official EPD data description before using the data!

Data folder structure

The path variable provided to the module should be the base directory where the corresponding cdf data files should be placed in subdirectories. First subfolder defines the data product level (l2 or low_latency at the moment), the next one the instrument (so far only epd), and finally the sensor (ept, het or step).

For example, the folder structure could look like this: /home/userxyz/solo/data/l2/epd/het. In this case, you should call the loader with path='/home/userxyz/solo/data'; i.e., the base directory for the data.

You can use the (automatic) download function described in the following section to let the subfolders be created initially automatically. NB: It might be that you need to run the code with sudo or admin privileges in order to be able to create new folders on your system.

Data download within Python

While using epd_load() to obtain the data, one can choose to automatically download missing data files from SOAR directly from within python. They are saved in the folder provided by the path argument (see above). For that, just add autodownload=True to the function call:

from solo_epd_loader import epd_load

df_protons, df_electrons, energies = \
    epd_load(sensor='het', level='l2', startdate=20200820,
             enddate=20200821, viewing='sun',
             path='/home/userxyz/solo/data/', autodownload=True)

# plot protons and alphas
ax = df_protons.plot(logy=True, subplots=True, figsize=(20,60))
plt.show()

# plot electrons
ax = df_electrons.plot(logy=True, subplots=True, figsize=(20,60))
plt.show()

Note: The code will always download the latest version of the file available at SOAR. So in case a file V01.cdf is already locally present, V02.cdf will be downloaded nonetheless.

Example 1 - low latency data

Example code that loads low latency (ll) electron and proton (+alphas) fluxes (and errors) for EPT NORTH telescope from Apr 15 2021 to Apr 16 2021 into two Pandas dataframes (one for protons & alphas, one for electrons). In general available are ‘sun’, ‘asun’, ‘north’, ‘south’, and ‘omni’ viewing directions for ‘ept’ and ‘het’ telescopes of SolO/EPD.

from matplotlib import pyplot as plt
from solo_epd_loader import epd_load

df_protons, df_electrons, energies = \
    epd_load(sensor='ept', level='ll', startdate=20210415,
             enddate=20210416, viewing='north',
             path='/home/userxyz/solo/data/')

# plot protons and alphas
ax = df_protons.plot(logy=True, subplots=True, figsize=(20,60))
plt.show()

# plot electrons
ax = df_electrons.plot(logy=True, subplots=True, figsize=(20,60))
plt.show()

Example 2 - level 2 data

Example code that loads level 2 (l2) electron and proton (+alphas) fluxes (and errors) for HET SUN telescope from Aug 20 2020 to Aug 20 2020 into two Pandas dataframes (one for protons & alphas, one for electrons).

from matplotlib import pyplot as plt
from solo_epd_loader import epd_load

df_protons, df_electrons, energies = \
    epd_load(sensor='het', level='l2', startdate=20200820,
             enddate=20200821, viewing='sun',
             path='/home/userxyz/solo/data/')

# plot protons and alphas
ax = df_protons.plot(logy=True, subplots=True, figsize=(20,60))
plt.show()

# plot electrons
ax = df_electrons.plot(logy=True, subplots=True, figsize=(20,60))
plt.show()

Example 3 - partly reproducing Fig. 2 from Gómez-Herrero et al. 2021 [1]

from matplotlib import pyplot as plt
from solo_epd_loader import epd_load
import numpy as np

# set your local path here
lpath = '/home/userxyz/solo/data'

# load ept sun viewing data
df_protons_ept, df_electrons_ept, energies_ept = \
   epd_load(sensor='ept', level='l2', startdate=20200708,
            enddate=20200724, viewing='sun', path=lpath, autodownload=True)

# load step data
df_step, energies_step = \
   epd_load(sensor='step', level='l2', startdate=20200708,
            enddate=20200724, path=lpath, autodownload=True)

# change time resolution to get smoother curve (resample with mean)
resample = '60min'

fig, axs = plt.subplots(2, sharex=True, figsize=(8, 10), dpi=200)
axs[0].set_prop_cycle('color', plt.cm.Oranges_r(np.linspace(0,1,7)))
axs[1].set_prop_cycle('color', plt.cm.winter(np.linspace(0,1,7)))

# plot selection of ept electron channels
for channel in [0, 8, 16, 26]:
   df_electrons_ept['Electron_Flux'][f'Electron_Flux_{channel}'].resample(resample).mean().plot(
      ax = axs[0], logy=True, label='EPT '+energies_ept["Electron_Bins_Text"][channel][0])

# plot selection of step ion channels
for channel in [8, 17, 33]:
   df_step[f'Magnet_Avg_Flux_{channel}'].resample(resample).mean().plot(
      ax = axs[1], logy=True, label='STEP '+energies_step["Bins_Text"][channel][0])

# plot selection of ept ion channels
for channel in [6, 22, 32, 48]:
   df_protons_ept['Ion_Flux'][f'Ion_Flux_{channel}'].resample(resample).mean().plot(
      ax = axs[1], logy=True, label='EPT '+energies_ept["Ion_Bins_Text"][channel][0])

axs[0].set_ylim([0.3, 4e6])
axs[1].set_ylim([0.01, 5e8])

axs[0].set_ylabel("Electron flux\n"+r"(cm$^2$ sr s MeV)$^{-1}$")
axs[1].set_ylabel("Ion flux\n"+r"(cm$^2$ sr s MeV)$^{-1}$")
axs[0].legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)
axs[1].legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)
plt.subplots_adjust(hspace=0)
fig.savefig("gh2021_fig_2.png", bbox_inches = "tight")
plt.close('all')

NB: This is just an approximate reproduction with different energy channels, different time resolution, and different viewing direction! Note also that the STEP data can not be used straightforwardly. Figure

Example 4 - partly reproducing Fig. 2e from Wimmer-Schweingruber et al. 2021 [2]

from matplotlib import pyplot as plt
from solo_epd_loader import epd_load
import datetime
import pandas as pd

# set your local path here
lpath = '/home/userxyz/solo/data'

# load data
df_protons_sun, df_electrons_sun, energies = \
    epd_load(sensor='ept', level='l2', startdate=20201210,
             enddate=20201211, viewing='sun',
             path=lpath, autodownload=True)
df_protons_asun, df_electrons_asun, energies = \
    epd_load(sensor='ept', level='l2', startdate=20201210,
             enddate=20201211, viewing='asun',
             path=lpath, autodownload=True)
df_protons_south, df_electrons_south, energies = \
    epd_load(sensor='ept', level='l2', startdate=20201210,
             enddate=20201211, viewing='south',
             path=lpath, autodownload=True)
df_protons_north, df_electrons_north, energies = \
    epd_load(sensor='ept', level='l2', startdate=20201210,
             enddate=20201211, viewing='north',
             path=lpath, autodownload=True)

# plot mean intensities of two energy channels; 'channel' defines the lower one
channel = 6
ax = pd.concat([df_electrons_sun['Electron_Flux'][f'Electron_Flux_{channel}'],
                df_electrons_sun['Electron_Flux'][f'Electron_Flux_{channel+1}']],
                axis=1).mean(axis=1).plot(logy=True, label='sun', color='#d62728')
ax = pd.concat([df_electrons_asun['Electron_Flux'][f'Electron_Flux_{channel}'],
                df_electrons_asun['Electron_Flux'][f'Electron_Flux_{channel+1}']],
                axis=1).mean(axis=1).plot(logy=True, label='asun', color='#ff7f0e')
ax = pd.concat([df_electrons_north['Electron_Flux'][f'Electron_Flux_{channel}'],
                df_electrons_north['Electron_Flux'][f'Electron_Flux_{channel+1}']],
                axis=1).mean(axis=1).plot(logy=True, label='north', color='#1f77b4')
ax = pd.concat([df_electrons_south['Electron_Flux'][f'Electron_Flux_{channel}'],
                df_electrons_south['Electron_Flux'][f'Electron_Flux_{channel+1}']],
                axis=1).mean(axis=1).plot(logy=True, label='south', color='#2ca02c')

plt.xlim([datetime.datetime(2020, 12, 10, 23, 0),
          datetime.datetime(2020, 12, 11, 12, 0)])

ax.set_ylabel("Electron flux\n"+r"(cm$^2$ sr s MeV)$^{-1}$")
plt.title('EPT electrons ('+str(energies['Electron_Bins_Low_Energy'][channel])
          + '-' + str(energies['Electron_Bins_Low_Energy'][channel+2])+' MeV)')
plt.legend()
plt.show()

NB: This is just an approximate reproduction; e.g., the channel combination is a over-simplified approximation! image1

References

License

This project is Copyright (c) Jan Gieseler and licensed under the terms of the BSD 3-clause license. This package is based upon the Openastronomy packaging guide which is licensed under the BSD 3-clause license. See the licenses folder for more information.

Acknowledgements

The development of this software has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101004159 (SERPENTINE).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

solo_epd_loader-0.3.7.tar.gz (2.1 MB view details)

Uploaded Source

Built Distribution

solo_epd_loader-0.3.7-py3-none-any.whl (447.7 kB view details)

Uploaded Python 3

File details

Details for the file solo_epd_loader-0.3.7.tar.gz.

File metadata

  • Download URL: solo_epd_loader-0.3.7.tar.gz
  • Upload date:
  • Size: 2.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.9.18

File hashes

Hashes for solo_epd_loader-0.3.7.tar.gz
Algorithm Hash digest
SHA256 85e61956ee06a3e5a4e4e12b79314b5056eabdb89302688c2668c918254c5a06
MD5 b433fee69dcc4be708f797646032c4ca
BLAKE2b-256 6b320af0210b2e91cc4d5ed264008edeeff2d820534f028f073e9aa0a29c7555

See more details on using hashes here.

File details

Details for the file solo_epd_loader-0.3.7-py3-none-any.whl.

File metadata

File hashes

Hashes for solo_epd_loader-0.3.7-py3-none-any.whl
Algorithm Hash digest
SHA256 5edc2928b676ed6ce001fbb091136ed32243a8291966bf6edd7591971d448f77
MD5 e7f90751492af412daa98f8857342398
BLAKE2b-256 30ec42b95cd12b0fd3f557c834018bcd53c2ad4c18749aa2dc574836c4b05153

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page