Redline All-Sky Imager Raw PGM Data Readfile (REGO)

Python library for reading REGO All-Sky Imager (ASI) stream0 raw PGM-file data. The data can be found at https://data.phys.ucalgary.ca.

Supported Datasets

• REGO raw: stream0 PGM files

Installation

The rego-imager-readfile library is available on PyPI:

$ python3 -m pip install rego-imager-readfile

Supported Python Versions

rego-imager-readfile officially supports Python 3.8+.

Examples

Example Python notebooks can be found in the "examples" directory. Further, some examples can be found in the "Usage" section below.

Usage

Import the library using import rego_imager_readfile

Warning: On Windows, be sure to put any read calls into a main() method. This is because we utilize the multiprocessing library and the method of forking processes in Windows requires it. Note that if you're using Jupyter or other IPython-based interfaces, this is not required.

Read a single file

>>> import rego_imager_readfile
>>> filename = "path/to/data/2020/01/01/fsmi_rego-654/ut06/20200101_0600_fsmi_rego-654_6300.pgm.gz"
>>> img, meta, problematic_files = rego_imager_readfile.read(filename)

Read multiple files

>>> import rego_imager_readfile, glob
>>> file_list = glob.glob("path/to/files/2020/01/01/fsmi_rego-654/ut06/*6300.pgm*")
>>> img, meta, problematic_files = rego_imager_readfile.read(file_list)

Read using multiple worker processes

>>> import rego_imager_readfile, glob
>>> file_list = glob.glob("path/to/files/2020/01/01/fsmi_rego-654/ut06/*6300.pgm*")
>>> img, meta, problematic_files = rego_imager_readfile.read(file_list, workers=4)

Read with no output

>>> import rego_imager_readfile, glob
>>> file_list = glob.glob("path/to/files/2020/01/01/fsmi_rego-654/ut06/*6300.pgm*")
>>> img, meta, problematic_files = rego_imager_readfile.read(file_list, workers=4, quiet=True)

Read only the first frame of each file

>>> import rego_imager_readfile, glob
>>> file_list = glob.glob("path/to/files/2020/01/01/fsmi_rego-654/ut06/*6300.pgm*")
>>> img, meta, problematic_files = rego_imager_readfile.read(file_list, first_frame=True)

Exclude reading the metadata

>>> import rego_imager_readfile, glob
>>> file_list = glob.glob("path/to/files/2020/01/01/fsmi_rego-654/ut06/*6300.pgm*")
>>> img, meta, problematic_files = rego_imager_readfile.read(file_list, no_metadata=True)

Development

Clone the repository and install dependencies using Poetry.

$ git clone https://github.com/ucalgary-aurora/rego-imager-readfile.git
$ cd rego-imager-readfile/python
$ make install

Testing

$ make test
[ or do each test separately ]
$ make test-flake8
$ make test-pylint
$ make test-pytest