nowcastlib 3.0.0

🧙🔧 Utils that can be reused and shared across and beyond the ESO Nowcast project

Nowcast Library

🧙‍♂️🔧 Utils that can be reused and shared across and beyond the ESO Nowcast project

This is a public repository hosted on GitHub via a push mirror setup in the internal ESO GitLab repository

Installation

Simply run

pip install nowcastlib

Usage and Documentation

Nowcast Library (nowcastlib) consists in a collection of functions organized in submodules (API) and a tool accessible via the command line (CLI). The latter is primarily intended for accessing the Nowcast Library Pipeline, an opinionated yet configurable set of processing steps for wrangling data and evaluating models in a consistent and rigorous way. More information can be found on the nowcastlib pipeline index page (link to markdown and link to hosted docs)

Please refer to the examples folder on GitHub for usage examples.

API

Here is a quick example of how one may import nowcastlib and access to one of the functions:

"""Example showing how to access compute_trig_fields function"""
import nowcastlib as ncl
import pandas as pd
import numpy as np

data_df = pd.DataFrame(
    [[0, 3, 4, np.NaN], [32, 4, np.NaN, 4], [56, 8, 0, np.NaN]],
    columns=["A", "B", "C"],
    index=pd.date_range(start="1/1/2018", periods=4, freq="2min"),
)

result = ncl.datasets.compute_trig_fields(data_df, ["A", "C"])

More in-depth API documentation can be found here.

CLI

Some of the library's functionality is bundled in configurable subcommands accessible via the terminal with the command nowcastlib:

usage: nowcastlib [-h] [-v]
                  {triangulate,preprocess,sync,postprocess,datapipe} ...

positional arguments:
  {triangulate,preprocess,sync,postprocess,datapipe}
                        available commands
    triangulate         Run `nowcastlib triangulate -h` for further help
    preprocess          Run `nowcastlib preprocess -h` for further help
    sync                Run `nowcastlib sync -h` for further help
    postprocess         Run `nowcastlib postprocess -h` for further help
    datapipe            Run `nowcastlib datapipe -h` for further help

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         increase verbosity level from INFO to DEBUG

Development Setup

This repository relies on Poetry for tracking dependencies, building and publishing. It is therefore recommended that developers install poetry and make use of it throughout their development of the project.

Dependencies

Make sure you are in the right Python environment and run

poetry install

This reads pyproject.toml, resolves the dependencies, and installs them.

Deployment

The repository is published to PyPi, so to make it accessible via a pip install command as mentioned earlier.

To publish changes follow these steps. Ideally this process is automated via a CI tool triggered by a push/merge to the master branch:

1. Optionally run poetry version with the appropriate argument based on semver guidelines.

2. Update the documentation by running

   make document
   

3. Prepare the package by running

   poetry build
   

4. Ensure you have TestPyPi and PyPi configured as your poetry repositories:

   poetry config repositories.testpypi https://test.pypi.org/legacy/
   poetry config repositories.pypi https://pypi.org/
   

5. Publish the repository to TestPyPi, to see that everything works as expected:

   poetry publish -r testpypi
   

6. Stage, commit and push your changes (to master) with git.

7. Publish the repository to PyPi:

   poetry publish -r pypi
   

Upon successful deployment, the library should be available for install via pip