The official dataload for http://www.nonlinearbenchmark.org/
Project description
nonlinear_benchmarks
The official dataloader of nonlinearbenchmark.org. This toolbox simplifies the process of downloading, loading, and splitting various datasets available on the website.
Usage Example
https://www.nonlinearbenchmark.org/benchmarks/wiener-hammerstein is loaded as:
import nonlinear_benchmarks
train_val, test = nonlinear_benchmarks.WienerHammerBenchMark()
print(train_val)
# prints : Input_output_data "train WH" u.shape=(100000,) y.shape=(100000,)
# sampling_time=1.953e-05
print(test)
# prints: Input_output_data "test WH" u.shape=(78800,) y.shape=(78800,)
# sampling_time=1.953e-05 state_initialization_window_length=50
sampling_time = train_val.sampling_time # in seconds
u_train, y_train = train_val # to unpack or use train_val.u, train_val.y
u_test, y_test = test # to unpack or use test.u, test.y
print(test.state_initialization_window_length)
#state_initialization_window_length = The number of samples that can be used at the
# start of the test set to initialize the model state.
print(train_val[:100]) # creates a slice of the train_val data from 0 to 100
Useful Options
When using the WienerHammerBenchMark
(or any other benchmark function), you can customize the behavior with the following options:
data_file_locations=True
: Returns the raw data file locations.train_test_split=False
: Retrieves the entire dataset without splitting.force_download=True
: Forces (re-)downloading of benchmark files.url=
: Allows manual override of the download link (contact maintainers if the default link is broken).atleast_2d=True
: Converts input/output arrays to at least 2D shape (e.g.,u.shape = (250,)
becomesu.shape = (250, 1)
).
Install
pip install nonlinear-benchmarks
Datasets
Multiple datasets have been implemented with an official train test split which are given below.
(p.s. datasets without an official train test split can be found in nonlinear_benchmarks.not_splitted_benchmarks
)
Electro-Mechanical Positioning System (EMPS)
train_val, test = nonlinear_benchmarks.EMPS()
print(test.state_initialization_window_length) # = 20
train_val_u, train_val_y = train_val
test_u, test_y = test
Coupled Electric Drives (CED)
train_val, test = nonlinear_benchmarks.CED()
print(test[0].state_initialization_window_length) # = 4
(train_val_u_1, train_val_y_1), (train_val_u_2, train_val_y_2) = train_val
(test_u_1, test_y_1), (test_u_2, test_y_2) = test
This dataset consists of two time series where the first has a low input amplitude (train_val_1
and test_1
) and the second a high input amplitude (train_val_2
and test_2
).
You can use both training sets in your training, and please report the RMSE values on both test sets separately.
Cascaded Tanks with Overflow (Cascaded_Tanks)
train_val, test = nonlinear_benchmarks.Cascaded_Tanks()
print(test.state_initialization_window_length) # = 4
train_val_u, train_val_y = train_val
test_u, test_y = test
Wiener-Hammerstein System (WienerHammerBenchMark)
train_val, test = nonlinear_benchmarks.WienerHammerBenchMark()
print(test.state_initialization_window_length) # = 50
train_val_u, train_val_y = train_val
test_u, test_y = test
Silverbox
train_val, test = nonlinear_benchmarks.Silverbox()
multisine_train_val = train_val
print(test[0].state_initialization_window_length) # = 50 (for all test sets)
test_multisine, test_arrow_full, test_arrow_no_extrapolation = test
Submission test RMSE template: submission_examples/silverbox.py
Note that the test arrow full and the test arrow no extrapolation have some overlap.
Error Metrics
We also provide error metrics in nonlinear_benchmarks.error_metrics
.
from nonlinear_benchmarks.error_metrics import RMSE, NRMSE, R_squared, MAE, fit_index
#generate example ouput data and prediction
y_true = np.random.randn(100)
y_model = y_true + np.random.randn(100)/100
print(f"RMSE: {RMSE(y_true, y_model)} (Root Mean Square Error)")
print(f"NRMSE: {NRMSE(y_true, y_model)} (Normalized Root Mean Square Error)")
print(f"R-squared: {R_squared(y_true, y_model)} (coefficient of determination R^2)")
print(f'MAE: {MAE(y_true, y_model)} (Mean Absolute value Error)')
print(f"fit index: {fit_index(y_true, y_model)} (https://arxiv.org/pdf/1902.00683.pdf page 31)")
Benchmark Result Submission
In the reporting of the benchmark result please use it as follows;
train_val, test = nonlinear_benchmarks.WienerHammerBenchMark()
n = test.state_initialization_window_length
# y_model = your model output using only test.u and test.y[:n]
RMSE_result = RMSE(test.y[n:], y_model[n:]) #skip the first n
print(RMSE_result) #report this number
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for nonlinear_benchmarks-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | a42376346b37f16849a0226b43a088875e430df02996f7b2bbb3bf8a134159d1 |
|
MD5 | 183d994b8a7907b3b8ca613254df8526 |
|
BLAKE2b-256 | e46671fe412b84beedce2aba22875103c9056571c3c5537e7f804e585163af27 |
Hashes for nonlinear_benchmarks-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 27e3b34498c79b72601aa66d2b94116857ed29d8a504546df472599162c92678 |
|
MD5 | fe9c8fc0d595d4694a5a7a61c4ac1f91 |
|
BLAKE2b-256 | 1ebad137ec2057b19f915de4749775409e0e0849db22c58792911f10140130fa |