The official dataload for http://www.nonlinearbenchmark.org/
Project description
nonlinear_benchmarks
The official dataloader of nonlinearbenchmark.org. This toolbox simplifies the process of downloading, loading, and splitting various datasets available on the website.
Usage Example
https://www.nonlinearbenchmark.org/benchmarks/wiener-hammerstein is loaded as:
import nonlinear_benchmarks
train_val, test = nonlinear_benchmarks.WienerHammerBenchMark()
print(train_val)
# prints : Input_output_data "train WH" u.shape=(100000,) y.shape=(100000,)
# sampling_time=1.953e-05
print(test)
# prints: Input_output_data "test WH" u.shape=(78800,) y.shape=(78800,)
# sampling_time=1.953e-05 state_initialization_window_length=50
sampling_time = train_val.sampling_time # in seconds
u_train, y_train = train_val # to unpack or use train_val.u, train_val.y
u_test, y_test = test # to unpack or use test.u, test.y
print(test.state_initialization_window_length)
#state_initialization_window_length = The number of samples that can be used at the
# start of the test set to initialize the model state.
print(train_val[:100]) # creates a slice of the train_val data from 0 to 100
Useful Options
When using the WienerHammerBenchMark
(or any other benchmark function), you can customize the behavior with the following options:
data_file_locations=True
: Returns the raw data file locations.train_test_split=False
: Retrieves the entire dataset without splitting.force_download=True
: Forces (re-)downloading of benchmark files.url=
: Allows manual override of the download link (contact maintainers if the default link is broken).atleast_2d=True
: Converts input/output arrays to at least 2D shape (e.g.u.shape = (250,)
becomesu.shape = (250, 1)
).always_return_tuples_of_datasets=True
: Even if there is only a single training or test set a list is still returned (i.e. adds[train] if not isinstance(train,list) else train
)
Install
pip install nonlinear-benchmarks
Datasets
Multiple datasets have been implemented with an official train test split which are given below.
(p.s. datasets without an official train test split can be found in nonlinear_benchmarks.not_splitted_benchmarks
)
Electro-Mechanical Positioning System (EMPS)
train_val, test = nonlinear_benchmarks.EMPS()
print(test.state_initialization_window_length) # = 20
train_val_u, train_val_y = train_val
test_u, test_y = test
Coupled Electric Drives (CED)
train_val, test = nonlinear_benchmarks.CED()
print(test[0].state_initialization_window_length) # = 4
(train_val_u_1, train_val_y_1), (train_val_u_2, train_val_y_2) = train_val
(test_u_1, test_y_1), (test_u_2, test_y_2) = test
This dataset consists of two time series where the first has a low input amplitude (train_val_1
and test_1
) and the second a high input amplitude (train_val_2
and test_2
).
You can use both training sets in your training, and please report the RMSE values on both test sets separately.
Cascaded Tanks with Overflow (Cascaded_Tanks)
train_val, test = nonlinear_benchmarks.Cascaded_Tanks()
print(test.state_initialization_window_length) # = 4
train_val_u, train_val_y = train_val
test_u, test_y = test
Wiener-Hammerstein System (WienerHammerBenchMark)
train_val, test = nonlinear_benchmarks.WienerHammerBenchMark()
print(test.state_initialization_window_length) # = 50
train_val_u, train_val_y = train_val
test_u, test_y = test
Silverbox
train_val, test = nonlinear_benchmarks.Silverbox()
multisine_train_val = train_val
print(test[0].state_initialization_window_length) # = 50 (for all test sets)
test_multisine, test_arrow_full, test_arrow_no_extrapolation = test
Submission test RMSE template: submission_examples/silverbox.py
Note that the test_arrow_no_extrapolation
is a subset of the test_arrow_full
.
Error Metrics
We also provide error metrics in nonlinear_benchmarks.error_metrics
.
from nonlinear_benchmarks.error_metrics import RMSE, NRMSE, R_squared, MAE, fit_index
#generate example ouput data and prediction
y_true = np.random.randn(100)
y_model = y_true + np.random.randn(100)/100
print(f"RMSE: {RMSE(y_true, y_model)} (Root Mean Square Error)")
print(f"NRMSE: {NRMSE(y_true, y_model)} (Normalized Root Mean Square Error)")
print(f"R-squared: {R_squared(y_true, y_model)} (coefficient of determination R^2)")
print(f'MAE: {MAE(y_true, y_model)} (Mean Absolute value Error)')
print(f"fit index: {fit_index(y_true, y_model)} (https://arxiv.org/pdf/1902.00683.pdf page 31)")
Benchmark Result Submission
In the reporting of the benchmark result please use it as follows;
train_val, test = nonlinear_benchmarks.WienerHammerBenchMark()
n = test.state_initialization_window_length
# y_model = your model output using only test.u and test.y[:n]
RMSE_result = RMSE(test.y[n:], y_model[n:]) #skip the first n
print(RMSE_result) #report this number
For example see the Silverbox submission template: submission_examples/silverbox.py
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for nonlinear_benchmarks-0.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee986f13e2aee7c2d421716ae88f49885aff1765445c82bbb764552191f4370e |
|
MD5 | 47318560afb627ca790d11b31647155b |
|
BLAKE2b-256 | e8086f3e25b7682f1ffe8d7542bf4e92fb9cec9583f19cecf140b0ea5f3b2ccb |
Hashes for nonlinear_benchmarks-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 78f0c4ed4313873f5ed85e7530409632cb4aa0aed507c40ee0f82969adc89939 |
|
MD5 | 1e719ba2a458b1c5e98430f1ee179eda |
|
BLAKE2b-256 | d492c2df145bca0d6fe4128b5bcefccfc13b8ac97836f08d707fefcc524d1a25 |