api-24sea 1.3.0

The api-24sea package contains modules that are aimed at helping a user interact with the 24SEA API.

API 24SEA

api_24sea is a project designed to provide aid for the interaction with data from the 24SEA API.

Installation

pip install api_24sea

DataSignals Usage

The following example shows the classical usage of the datasignals module.

• The first step is to import the package and the necessary libraries.
• Then, the environment variables are loaded from a .env file.
• After that, the package is initialized and the user is authenticated with the API.
• Finally, the user can get data from the API.

Importing the package

# %%
# **Package Imports**
# - From the Python Standard Library
import logging
import os

# - From third party libraries
import pandas as pd
import dotenv  # <-- Not necessary to api_24sea per se, but useful for
                #     loading environment variables. Install it with
                #     `pip install python-dotenv`

# - API 24SEA
from api_24sea.version import __version__, parse_version
import api_24sea

Setting up the environment variables

This step assumes that you have a file structure similar to the following one:

.
├── env/
│   └── .env
├── notebooks/
│   └── example.ipynb
└── requirements.txt

The .env file should look like this:

24SEA_API_USERNAME=your_username
24SEA_API_PASSWORD=your_password

With this in mind, the following code snippet shows how to load the environment variables from the .env file:

# %%
_ = dotenv.load_dotenv("../env/.env")
if _:
    print("Environment Variables Loaded Successfully")
    print(os.getenv("24SEA_API_USERNAME"))
else:
    raise Exception("Environment Variables Not Loaded")

Initializing an empty dataframe

Initializing an empty dataframe is necessary to use the API, as here is where the data will be stored.

# %%
# **DataFrame initialization**
# The empty DataFrame is created beforehand because it needs to authenticate
# with the API to fetch the data.
df = pd.DataFrame()

# %%
# This is a test to check if the authentication system warns the users when
# they are not authenticated.
try:
    df.datasignals.get_metrics()
except Exception as e:
    print("API not available")
    print(e)
# It will raise an exception because the user is not authenticated

Authenticating with the API

The authentication step is performed automatically if the environment variables 24SEA_API_USERNAME and 24SEA_API_PASSWORD are loaded. The user can also authenticate manually by calling the authenticate method from the DataFrame.

# %%
# **Authentication**
df.datasignals.authenticate("some_other_username", "some_other_password")

Checking the available metrics after authentication

# %%
# **Metrics Overview**
# The metrics overview is a summary of the metrics available in the API and can
# be accessed from a hidden method in the DataSignals class.
df.datasignals._DataSignals__api.metrics_overview
# It will show all the available metrics with the corresponding units
# and the time window for which the user is allowed to get data

Getting sample data from the API

After loading the environment variables and authenticating with the API, the user can get data from 24SEA API endpoints.

The data is retrieved and stored in the DataFrame.

When the option as_dict is set to False, the response from each metric is joined on the timestamp which is then set as the index of DataFrame. This option will necessarily drop the location and site columns from the DataFrame, but they can still be retrieved from the metrics names.

The data retrieval is done by specifying the sites or the locations or both, the metrics, and timestamps.

• Sites: Case insensitive, it can either match site or site_id. It is an optional parameter.
• Locations: Case insensitive, it can either match location or location_id. It is an optional parameter.
• Metrics: Case insensitive, it can be a partial match of the metric name
• Timestamps: Timezone-aware datetime, strings in ISO 8601 format, or shorthand strings compatible with the shorthand_datetime package.

# %%
# **Data Retrieval**

sites = ["wf"]

locations = ["a01", "a02"]

metrics = ["mean WinDSpEed", "mean pitch", "mean-Yaw", "mean_power"]

start_timestamp = "2020-03-01T00:00:00Z"
end_timestamp = "2020-06-01T00:00:00Z"

df.datasignals.get_data(sites, locations, metrics,
                        start_timestamp, end_timestamp, as_dict=False)

Checking the metrics selected and the data

# %%
df.datasignals.selected_metrics
df

as_dict True

When as_dict is set to True, the response from each metric is stored in a dictionary with the site and location as keys.

# %%
# Data is a dictionary of dictionary of DataFrames in the shape of:
# {
#   "site1": {
#     "location1": DataFrame,
#     "location2": DataFrame,
#     ...
#   },
#   ...
# }
data = df.datasignals.get_data(sites, locations, metrics,
                    start_timestamp, end_timestamp, as_dict=True)
# %%
# Retrieve the DataFrame for the windfarm WFA01 only
data["windfarm"]["WFA02"]

Core API Usage

The core API module is designed to provide a more direct interaction with the 24SEA API as it is not implemented as a pandas accessor.

The following example shows the classical usage of the core API module, which can be integrated within other standalone classes or functions.

• The first step is to import the package and the necessary libraries.
• Then, the environment variables are loaded from a .env file.
• After that, the package is initialized and the user is authenticated with the API.
• Finally, the user can get data from the API.

The first two steps are the same as in the DataSignals module and will not be repeated here.

Authenticating with the API

The authentication step allows the user to access the API and check the available metrics.

  # %%
  # **Authentication**
  api = api_24sea.API()
  api.authenticate(
      os.getenv("24SEA_API_USERNAME"), os.getenv("24SEA_API_PASSWORD")
  )

Checking the available metrics after authentication

  # %%
  # **Metrics Overview**
  # The metrics overview is a summary of the metrics available in the API and
  # can be accessed from a hidden method in the DataSignals class.
  api._API__api.metrics_overview
  # It will show all the available metrics with the corresponding units
  # and the time window for which the user is allowed to get Data

Getting sample data from the API

After loading the environment variables and authenticating with the API, the user can get data from 24SEA API endpoints.

The retrieved data can be stored in a DataFrame or a dictionary:

• When the option as_dict is set to False the output is a dataframe. Besed on the option outer_join_on_timestamp, the response will be:

  • outer_join_on_timestamp = True: the response from each metric is joined on the timestamp which is then set as the index of DataFrame. This option will necessarily drop the location and site columns from the DataFrame, but they can still be retrieved from the metrics names.
  • outer_join_on_timestamp = false: the response from each metric is stored in a separate DataFrame and the site and location columns are kept. This means that the DataFrame will be "diagonal" with repeated timestamps (as many times as the number of queried (locations, sites) pairs).

• When the option as_dict is set to True, the dataframe is returned as a dictionary according to the following formula: dataframe.reset_index().to_dict('records').

  # %%
  # **Data Retrieval**
  sites = ["wf"]

  locations = ["a01", "a02"]

  metrics = ["mean WinDSpEed", "mean pitch", "mean-Yaw", "mean_power"]

  start_timestamp = "2020-03-01T00:00:00Z"
  end_timestamp = "2020-06-01T00:00:00Z"

  data = api.get_data(sites, locations, metrics,
                      start_timestamp, end_timestamp, as_dict=False,
                      outer_join_on_timestamp=True)

Data as Star Schema

Overview

The data as star schema feature is designed to provide a more user-friendly experience when getting data for BI purposes. It is implemented both for the core API module and the DataSignals module.

---

Usage

In this example, we will demonstrate how to use the normalization feature with the core API module.

Example

# %%
# **Star schema**
# The data normalization feature is designed to provide a more user-friendly
# experience when working with the API. It is implemented both for the core
# API module and the DataSignals module.

# %%
# **Data Retrieval**
sites = ["wf"]

locations = ["a01", "a02"]

metrics = ["mean WinDSpEed", "mean pitch", "mean-Yaw", "mean_power"]

start_timestamp = "2020-03-01T00:00:00Z"
end_timestamp = "2020-06-01T00:00:00Z"

star_schema = api.get_data(sites, locations, metrics,
                           start_timestamp, end_timestamp, as_star_schema=True)

This command is equivalent to the following:

# %%
# **Data Retrieval**
from api_24sea.core import to_star_schema, API

api = API()

sites = ["wf"]

locations = ["a01", "a02"]

metrics = ["mean WinDSpEed", "mean pitch", "mean-Yaw", "mean_power"]

start_timestamp = "2020-03-01T00:00:00Z"
end_timestamp = "2020-06-01T00:00:00Z"

data = api.get_data(sites, locations, metrics,
                    start_timestamp, end_timestamp)

star_schema = to_star_schema(data, api.metrics_overview)

# or alternatively

star_schema = api.get_data(sites, locations, metrics,
                           start_timestamp, end_timestamp, as_star_schema=True)

The star_schema variable will contain a dictionary with the following keys:

• DimCalendar: A DataFrame with the timestamps and their corresponding calendar information.
• DimWindFarm: A DataFrame with the wind farm information (site an locations)
• DimDataGroup: A DataFrame with the metric group information (e.g. TP, SCADA)
• DimMetric: A DataFrame with the metric information (e.g. mean pitch, mean power)
• FactData: A DataFrame with the data itself.

Data as Category-Value

Overview

The data as category-value feature is designed to reshape the data so that all the information for a metric is stored in a single row. It is implemented only for the core API module.

# %%
# **Data Retrieval**
from api_24sea.core import to_category_value
sites = ["wf"]

locations = ["a01", "a02"]

metrics = ["mean WinDSpEed", "mean pitch", "mean-Yaw", "mean_power"]

start_timestamp = "2020-03-01T00:00:00Z"
end_timestamp = "2020-06-01T00:00:00Z"

data = api.get_data(sites, locations, metrics,
                    start_timestamp, end_timestamp)

category_value = to_category_value(data)

---

Example table

The data is transformed from the following (example) shape:

timestamp	mean_WF_A01_TP_SG_LAT005_DEG000	mean_WF_A01_TP_SG_LAT005_DEG045	mean_WF_A01_TP_SG_LAT005_DEG090	mean_WF_A02_TP_SG_LAT015_DEG000	mean_WF_A02_TP_SG_LAT015_DEG045	mean_WF_A02_TP_SG_LAT015_DEG090
2020-03-01 00:00:00	1.0	2.0	3.0	4.0	5.0	6.0
2020-03-01 00:10:00	1.1	2.1	3.1	4.1	5.1	6.1
2020-03-01 00:20:00	1.2	2.2	3.2	4.2	5.2	6.2
2020-03-01 00:30:00	1.3	2.3	3.3	4.3	5.3	6.3

To the following (example) shape:

timestamp	full_metric_name	value	unit	statistic	short_hand	site_id	location_id	lat	heading	site	location	metric_group
2020-03-01 00:00:00	mean_WF_A01_TP_SG_LAT005_DEG000	1.0	unit	mean	TP_SG_LAT005_DEG000	WF	A01	5.0	0.0	WindFarm	WFA01	TP
2020-03-01 00:10:00	mean_WF_A01_TP_SG_LAT005_DEG000	1.1	unit	mean	TP_SG_LAT005_DEG000	WF	A01	5.0	0.0	WindFarm	WFA01	TP
2020-03-01 00:20:00	mean_WF_A01_TP_SG_LAT005_DEG000	1.2	unit	mean	TP_SG_LAT005_DEG000	WF	A01	5.0	0.0	WindFarm	WFA01	TP
2020-03-01 00:30:00	mean_WF_A01_TP_SG_LAT005_DEG000	1.3	unit	mean	TP_SG_LAT005_DEG000	WF	A01	5.0	0.0	WindFarm	WFA01	TP
2020-03-01 00:00:00	mean_WF_A01_TP_SG_LAT005_DEG045	2.0	unit	mean	TP_SG_LAT005_DEG045	WF	A01	5.0	45.0	WindFarm	WFA01	TP
2020-03-01 00:10:00	mean_WF_A01_TP_SG_LAT005_DEG045	2.1	unit	mean	TP_SG_LAT005_DEG045	WF	A01	5.0	45.0	WindFarm	WFA01	TP
2020-03-01 00:20:00	mean_WF_A01_TP_SG_LAT005_DEG045	2.2	unit	mean	TP_SG_LAT005_DEG045	WF	A01	5.0	45.0	WindFarm	WFA01	TP
2020-03-01 00:30:00	mean_WF_A01_TP_SG_LAT005_DEG045	2.3	unit	mean	TP_SG_LAT005_DEG045	WF	A01	5.0	45.0	WindFarm	WFA01	TP
2020-03-01 00:00:00	mean_WF_A01_TP_SG_LAT005_DEG090	3.0	unit	mean	TP_SG_LAT005_DEG090	WF	A01	5.0	90.0	WindFarm	WFA01	TP
2020-03-01 00:10:00	mean_WF_A01_TP_SG_LAT005_DEG090	3.1	unit	mean	TP_SG_LAT005_DEG090	WF	A01	5.0	90.0	WindFarm	WFA01	TP

Fatigue Extra

The extra is compatible with Python versions from 3.8 to 3.10, and installs the py-fatigue and swifter packages.

Installation

To install the extra, run the following command in your terminal:

pip install api_24sea[fatigue]

Usage

# %%
# Import the pandas fatigue accessor from the api_24sea package
from api_24sea.datasignals import fatigue

[!NOTE]

Suppose you have already authenticated with the API, loaded the environment variables, and initialized the DataFrame.

If your Metrics Overview table shows metrics whose name starts with CC_, then the fatigue extra will be available for use.

# %%
# **Data Retrieval**
# Besides SCADA data, we will query cycle-count metrics, which are available
# by looking for "CC" (cycle-count) and ["Mtn", "Mtl"] (i.e. Normal and
# Lateral Bending moment).

sites = ["wf"]
locations = ["a01", "a02"]
metrics = ["mean WinDSpEed", "mean pitch", "mean-Yaw", "mean power",
            "cc mtn", "cc mtl"] # <-- Cycle-count metrics

start_timestamp = "2020-03-01T00:00:00Z"
end_timestamp = "2020-06-01T00:00:00Z"

df.datasignals.get_data(sites, locations, metrics,
                        start_timestamp, end_timestamp, as_dict=False)

Analyzing cycle-count metrics

Converting the cycle-count JSON objects to py_fatigue.CycleCount objects is the first step in the fatigue analysis. This is done by calling the api_24sea.datasignals.fatigue.Fatigue.cycle_counts_to_objects method.

# %%
# **Fatigue Analysis**
# The fatigue analysis is done by calling the cycle_counts_to_objects() method
# from the fatigue accessor.
try:
    df.fatigue.cycle_counts_to_objects()
except ImportError as ie:
    print(f"\033[31;1mImportError\033[22m: {ie}")

At this point, you can treat your py_fatigue.CycleCount objects as you would normally do in py-fatigue.

For more information, check py-fatigue's beginner's guide and API documentation.

Project Structure

.
├── .azure/
├── api_24sea/
│   ├── __init__.py
│   ├── datasignals/
│   │   ├── __init__.py
│   │   ├── fatigue.py
│   │   └── schemas.py
│   ├── core.py
│   ├── exceptions.py
│   ├── singleton.py
│   ├── utils.py
│   └── version.py
├── tests/
├── docs/
├── notebooks/
├── pyproject.toml
├── LICENSE
├── VERSION
└── README.md

License

The package is licensed under the GNU General Public License v3.0.