fetchsep 0.3.1

Identify SEP elevations above background in a time series (idsep) and analyze events individually (opsep)

Identify SEP elevations above background in a time series (idsep) and analyze events individually (opsep)

• Free software: MIT license

• Documentation: https://fetchsep.readthedocs.io.

Set Up

By default, idsep and opsep will create the necessary output directories in the current working directory where the command is executed. The directories that will be created are data, ouptut, plots, and lists. The directories will not be overwritten if they already exist. If you would like to choose another output location, generate a config file (fetchsep.cfg) with opsep –ExportConfig and edit the paths described there.

A configuration file may also be placed in your home directory with the name .fetchsep. The configuration file does not need to be complete; you may specify only the values for which you wish to override the default. The order of config value precidence is 1. current working directory fetchsep.cfg, 2. home directory .fetchsep, 3. fetchsep defaults.

Run

To run OpSEP to process individual SEP events:

opsep –StartDate 2012-05-16 –EndDate 2012-05-22 –Experiment GOES-13 –FluxType integral –showplot

IDSEP

The idsep code will read in a long time series and automatically identify increases above background. This is done by estimating a mean background level plus an expected level of variation (sigma). All flux less than mean + 3sigma are considered background while all points above mean + 3sigma are identified as increases.

By assigning a set of criteria, increases that are most likely due to SEP events are identified and an SEP event list is output for each energy channel.

The code also outputs a file containing every single high flux point above the mean + 3sigma threshold.

Note that in fetchsep/utils/config.py, number of sigma, the initial window used to estimate background levels, and the final sliding window used to estimate background levels can be adjusted.

idsep_nsigma = 3

init_win = 150 #days to average initial estimate of threshold

sliding_win = 27 #days in sliding window to calculate final threshold

percent_points = 0.9 #Percent of points that must be in the sliding

#window to calculate the background; otherwise use

#previous good value

OPSEP

The opsep code was previously supported at https://github.com/ktindiana/operational-sep and is now transitioned to this package going forward. Please see the operational-sep repository for extensive documentation until the documentation in this repository can be updated.

opsep is intended to assess each individual SEP event at a time, extracting information such as start and end times, peak fluxes, and event fluence.

The code will output various csv files and a json file with accompany txt files. The JSON file is in the same format as required by the SEP Scoreboard to submit forecasts.

opsep creates files from observations that can be directly compared to SEP model forecasts sent to the SEP Scoreboard.

For time profile SEP models, opsep may be used to create the JSON files that can be submitted to the SEP Scoreboard.

Running opsep for your own time series

Users may input their own time series into opsep by specifying some information in the config file:

##### DELIMETER between columns of file with time series

user_delim = “ “ #any string

##### COLUMNS containing the fluxes you want to analyze

user_col = arr.array(‘i’,[1,2,3,4,5,6,7,8])

err_col = arr.array(‘i’,[]) #set to [] if no uncertainties. err_col only used by idsep

##### ENERGY BINS associated with user file and columns

#For differential bins, use the format:

user_energy_bins = [[Elow1,Ehigh1],[Elow2,Ehigh2],etc]

#For integral bins, use the format:

user_energy_bins = [[Elow1,-1],[Elow2,-1],[Elow3,-1],etc]

Automatically generate a Processed SEP Event list

It is possible to run both codes with a single button push to create a preliminary SEP event list. The code:

fetchsep_prepare_obs

will first run idsep on a specified data set and identify all increases above background. Output files are created that are then used to automatically run opsep in batch mode to analyze each quiet and elevated period. This creates a set of json another other supporting files for each SEP event and quiet time period in the time series.

Note that manual intervention is required to get a truly good event list. The automated method is not perfect at identifying individual SEP events, but it will get you 80% of the way there.

Support

Do not hesitate to contact Katie Whitman at [e-mail] for support with this code.

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

History

0.3.1 (2024-05-02)

• opsep: modified onset peak fitting and estimate

• opsep: onset peak changed to max flux if automated algorithm identifies onset after max

• opsep: better workflow for fetchsep_prepare_obs when starting from BATCH

• opsep: updates to json output to comply with CCMC schema

• Bugfix: Fix crash in opsep and idsep on certain python installations when reading in GOES-R data files

0.3.0 (2023-11-02)

• Fixed bug preventing fluence calculation for differential fluxes

• Improved plots for differential energy channels

• Added capability to move output produced by opsep to a target dir

• Updated filenames for plots to contain full zulu string

• Added padding to SEP start and end times in fetchsep_prepare_obs workflow

• Removed unneeded asciitable dependency

0.2.2 (2023-09-22)

• Fix packaging bug that failed to include and correctly use the CCMC json templates

0.2.1 (2023-09-22)

• Fix packaging bug that failed to include the default configuration

0.2.0 (2023-09-22)

• Add GOES-18 differential and real time integral fluxes; changes GOES-R integral flux source to CCMC’s HAPI API

• Add configuration by user config file in the configparser INI format, replacing placement of config values in config.py. This fixes the pip deployment issue whereby a user could not easily update the code configuration.

• Add –ExportConfig option to opsep & idsep. Users may use this to prepare a custom configuration.

• Fix: Create necessary output directories instead of crashing if they don’t exist

0.1.4 (2023-08-22)

• Fix bug preventing command-line scripts from executing in Python interpreter

0.1.3 (2023-08-22)

• Update utils/read_datasets.py to add new version, v3-0-1, for GOES-R differential files

• Fix bug in ccmc_json_handler.py that would crash for model profiles with 0 values

• Fix deployment of command-line scripts

• Update README documentation

0.1.2 (2023-08-22)

• Fix README for PyPI distribution

• Update developer package requirements

0.1.1 (2023-08-08)

• Test bump2version procedure

0.1.0 (2023-06-01)

• First release on PyPI.