Skip to main content

Tools for performing common tasks on solar PV data signals

Project description

solar-data-tools

Latest Release latest release
License license
Build Status documentation build status Actions build status
Code Quality Language grade: Python Total alerts
Publications DOI
PyPI Downloads PyPI downloads
Conda Downloads conda-forge downloads
Test-Coverage test-coverage

Tools for performing common tasks on solar PV data signals. These tasks include finding clear days in a data set, common data transforms, and fixing time stamp issues. These tools are designed to be automatic and require little if any input from the user. Libraries are included to help with data IO and plotting as well.

There is close integration between this repository and the Statistical Clear Sky repository, which provides a "clear sky model" of system output, given only measured power as an input.

See notebooks folder for examples.

Install & Setup

3 ways of setting up, either approach works:

1) Recommended: Set up conda environment with provided .yml file

We recommend setting up a fresh Python virtual environment in which to use solar-data-tools. We recommend using the Conda package management system, and creating an environment with the environment configuration file named pvi-user.yml, provided in the top level of this repository. This will install the statistical-clear-sky package as well.

Creating the env:

$ conda env create -f pvi-user.yml

Starting the env:

$ conda activate pvi_user

Stopping the env

$ conda deactivate

Updating the env with latest

$ conda env update -f pvi-user.yml

Additional documentation on setting up the Conda environment is available here.

2) PIP Package

$ pip install solar-data-tools

Alternative: Clone repo from GitHub

Mimic the pip package by setting up locally.

$ pip install -e path/to/root/folder

3) Anaconda Package

$ conda install -c slacgismo solar-data-tools

Solvers

QSS & OSQP

By default, QSS and OSQP solvers are used for non-convex and convex problems, respectively. Both are supported by OSD, the modeling language used to solve signal decomposition problems in Solar Data Tools, and both are open source.

MOSEK

MOSEK is a commercial software package. It is more stable and offers faster solve times. The included YAML/requirements.txt file will install MOSEK for you, but you will still need to obtain a license. More information is available here:

Usage

Users will primarily interact with this software through the DataHandler class. If you would like to specify a solver, just pass the keyword argument solver to dh.pipeline with the solver of choice. Passing QSS will keep the convex problems solver as OSQP, unless solver_convex=QSS is passed as well. Setting solver=MOSEK will set the solver to MOSEK for convex and non-convex problems by default.

from solardatatools import DataHandler
from solardatatools.dataio import get_pvdaq_data

pv_system_data = get_pvdaq_data(sysid=35, api_key='DEMO_KEY', year=[2011, 2012, 2013])

dh = DataHandler(pv_system_data)
dh.run_pipeline(power_col='dc_power')

If everything is working correctly, you should see something like the following

total time: 24.27 seconds
--------------------------------
Breakdown
--------------------------------
Preprocessing              11.14s
Cleaning                   0.94s
Filtering/Summarizing      12.19s
    Data quality           0.25s
    Clear day detect       1.75s
    Clipping detect        7.77s
    Capacity change detect 2.42s

Contributors

Must enable pre-commit hook before pushing any contributions

pip install pre-commit
pre-commit install

Run pre-commit hook on all files

pre-commit run --all-files

Test Coverage

In order to view the current test coverage metrics, run:

coverage run --source solardatatools -m unittest discover && coverage html
open htmlcov/index.html

Versioning

We use Semantic Versioning for versioning. For the versions available, see the tags on this repository.

Authors

See also the list of contributors who participated in this project.

License

This project is licensed under the BSD 2-Clause License - see the LICENSE file for details

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

solar-data-tools-1.0.5.tar.gz (76.2 MB view hashes)

Uploaded source

Built Distribution

solar_data_tools-1.0.5-py3-none-any.whl (76.2 MB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page