A benchmarking framework to run and evaluate changepoint detection algorithms.
Project description
The Changepoint-Detection Workbench (CPD-Bench)
This library is a performance and test benchmark for changepoint detection algorithms, especially created for the changepoynt project.
Important links
Installation
Simply install the cpd-bench via pip and include it into your library:
pip install cpdbench
Usage
Basic usage
- Import cpdbench.CPDBench and create a CPDBench object cpdb
- Use the decorators "dataset", "algorithm", and "metric" of this cpdb object to annotate your respective changepoint dataset function, your changepoint algorithms and validation metrics. The functions have to look like this:
- dataset: def dataset_funtion() -> dataset: cpdbench.dataset.CPDDataset
- algorithm: def algorithm_function(signal: ndarray) -> changepoints: list[int], confidences: list[float]
- metric: def metric_funtion(changepoints: list[int], confidences: list[float], ground_truths: list[int]) -> result: float
- Use cpdb.start() to start the workbench
A very basic configuration created with included example functions looks like this:
from cpdbench.CPDBench import CPDBench
import cpdbench.examples.ExampleDatasets as example_datasets
import cpdbench.examples.ExampleAlgorithms as example_algorithms
import cpdbench.examples.ExampleMetrics as example_metrics
cpdb = CPDBench()
@cpdb.dataset
def get_apple_dataset():
return example_datasets.dataset_get_apple_dataset()
@cpdb.dataset
def get_bitcoin_dataset():
return example_datasets.dataset_get_bitcoin_dataset()
@cpdb.algorithm
def execute_esst_test(signal):
return example_algorithms.algorithm_execute_single_esst(signal)
@cpdb.metric
def calc_accuracy(indexes, scores, ground_truth):
return example_metrics.metric_accuracy_in_allowed_windows(indexes, scores, ground_truth, window_size=25)
if __name__ == '__main__':
cpdb.start()
Configuration
You can configure multiple settings using a config.yml file. For this create a config.yml file with the syntax/commands given in cpdbench.examples.configs.parametersConfig.yml and enter the file path when running the bench: cpdb.start(config_file)
Use of parameters
Use parameters in your own functions as global placeholders (global parameters) or to run the function multiple
times with different configurations (runtime parameters).
To use parameters declare them in your function heads as keyword-only parameters, for example:
def algorithm_function(signal, *, example_param)
Then enter the values in your config file:
- global param: user -> "param name: value"
- runtime param: user -> dataset-executions/algorithm-executions/metric-executions -> list of "param name: value" for the amount of executions/run configurations. Example:
user:
global_param1: 242
global_param2: 353
algorithm_executions:
- runtime_param1: 2424
runtime_param2: 3
- runtime_param1: 345
runtime_param2: 3
For more examples please refer to the "examples" package.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file cpdbench-1.0.0.tar.gz
.
File metadata
- Download URL: cpdbench-1.0.0.tar.gz
- Upload date:
- Size: 1.4 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.9.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 78ef33836ebd6b466a8d768b1412086fd26ca2c44b32da736d5cb4198e0b37ec |
|
MD5 | 4165b20412b4e4388431c3dbab088901 |
|
BLAKE2b-256 | b7329c50b7448201efad2cec4ae07df9d94d01e6a1ff94b1aee3e638ad4cfa2e |
File details
Details for the file cpdbench-1.0.0-py3-none-any.whl
.
File metadata
- Download URL: cpdbench-1.0.0-py3-none-any.whl
- Upload date:
- Size: 404.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.9.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 861512e1261ba9ec56d10a104fab1d925a7a76b4dd9303111d629bc25e4923e7 |
|
MD5 | aa9de49edb8b4ec35524674b6efe1d1a |
|
BLAKE2b-256 | 15e593af1246fdb8131cc4ab07362aad47d1595438c1d0a57aaebd0754d4794d |