wl-utilities 2022.0.2

Utility code to perform QAG and SV analysis

Utility code to be used in the WEAVE-QAG/SV environment

This package is called wl-utilities on PyPI and can be installed with pip install wl-utilities and then imported with import wl_utilities This package is a collaborative effort and will comprise the scientific code that allows the QAG tests and SV to run. wl_utilities will be a dependency of those projects. wl_utilities should not use weaveio or qag packages.

Please see setup for setting up github

Workflow

There will be 2 branches in use on this repository:

• main - where the production-ready version exists and which is uploaded automatically to PyPI for use by everyone.
• develop - where changes and merges all take place before merging into the main branch

You will likely have 2 versions of this code:

1. from PyPI (with pip install). This will be the version you should use when running science tests in production
2. from your own development repo. This will be your test and development version which must be separate from everything else

setup

To setup git to deal with collaboration here

1. On github click Fork. This creates a copy of wl_utilities for you to work on (WARNING: make sure your repo is called "wl_utilities" and not "utilities")
2. Check that your machine can use ssh key authentication with github
3. On the pc where you develop QAG tests, cd somewhere for development on this wl_utiltities repository. There is no need to create a wl_utilities directory.
4. git clone [e-mail]:YOURUSERNAME/wl-utilities.git (the link can be found under the green code button on your github page)
5. cd wl_utilities on your local machine.
6. git remote add upstream [e-mail]:WEAVE-LOFAR/wl-utilities.git
7. git fetch --all
8. Activate your development environment (maybe: conda activate weaveio or similar)
9. pip install -r requirements.txt --user && pip install -r dev-requirements.txt --user && pre-commit install --hook-type pre-commit --hook-type pre-push

To add/modify this repository with your own changes and improvements please

0. Activate your development environment (maybe: conda activate weaveio or similar)
1. Pull the latest updates: git fetch --all && git checkout main && git pull && git merge upstream/main
2. Create a new branch for your changes to live on: git checkout -b my-very-clever-edit
3. Make your changes on this branch, committing as you go.
4. Update requirements.txt and manifest.IN with your python package requirements and ancillary files that you used. For example, you would need to include the list of sky lines (write sky_lines.csv to manifest.IN and you would need to add astroquery == version.number to requirements.txt if you use that package).
5. View what has happened: git status
6. Add your changes to be commited: git add <file1> <file2> ...
7. View what has changed: git status
8. Save your changes: git commit -m "description of changes"
9. Push changes to your fork only: git push
10. Open a pull request on this github page, referencing myusername/my-very-clever-edit -> weave-lofar/develop

All changes therefore end up in the develop branch of the weave-lofar shared repo.

All pull requests will be reviewed before merging, so we can limit mistakes.

Structure

The structure of this package will be:

wl_utilities/
  misc/  # for random short but useful snippets
  spectrum/  # for anything that processes spectra
    e.g. cross-correlation.py
    e.g. reduction.py
  stats/  # for anything that looks like a statistical test
    e.g. zscores.py

We will not use separate folders for individual's code since the objective is to put them together and not repeat.

Rules:

• This repo will be autoformatted according to black on server-side
• No change is merged with the main branch until approved by "enough" people
• Keep the code as modular as possible