aqudem 0.3.0

Activity and Sequence Detection Performance Measures: A package to evaluate activity detection results, including the sequence of events given multiple activity types.

AquDeM

Activity and Sequence Detection Evaluation Metrics: A Comprehensive Tool for Event Log Comparison.

• Documentation: https://aqudem.readthedocs.io

Installation

pip install aqudem

Usage

import aqudem

aqu_context = aqudem.Context("ground_truth_log.xes", "detected_log.xes")

aqu_context.activity_names # get all activity names present in log
aqu_context.case_ids # get all case IDs present in log

aqu_context.cross_correlation() # aggregate over all cases and activites
aqu_context.event_analysis(activity_name="Pack", case_id="1") # filter on case and activity
ts = aqu_context.two_set(activity_name="Pack") # filter on activity, aggregate over cases
ts_f1_score = ts.f1 # get the F1 score of the Two Set metric

For a more detailed description of the available methods, please refer to the rest of the documentation.

Preface

• Metrics to evaluate activity detection results

• Input: two XES files, one with the ground truth and one with the detection results

• Output: a set of metrics to evaluate the detection results

• Prerequisites for the input files: the XES files must…

  • … have a sampling_freq in Hz associated with each case (only detected file), and each case must have the same sampling frequency

  • … have a concept:name attribute for each case (case ID), with a matching case ID in both files (ground truth and detected)

  • … have a time:timestamp attribute for each event

  • … have an concept:name attribute for each event (activity name)

  • … have a lifecycle:transition attribute for each event

  • … each start event must have a corresponding complete event; and only these two types of events are relevant for the analysis currently; activity executions with a duration of exactly zero are removed

An ACTIVITY_METRIC is a metric that is calculated for each activity type in each case separately. Available ACTIVITY_METRICs are:

• Cross-Correlation

• Event Analysis by Ward et al. (2011)

• Two Set Metrics by Ward et al. (2011)

A SEQUENCE_METRIC is a metric that is calculated for each case separately. Available SEQUENCE_METRICs are:

• Damerau-Levenshtein Distance

• Levenshtein Distance

All metrics are also available in appropriately normalized versions. For requests that span multiple cases, the results are aggregated. The default and only aggregation method is currently the mean. For more detailed definitions of the metrics, please refer to the documentation.

History

0.3.0 (2025-06-29)

Make it more clear that EA and TS derived metrics/rates are micro-averaged:

In order to clarify the nature of the rates and metrics such as F1 of the event analysis and two set metrics we decided to give sums of event and frame classification totals. Derived metrics are then more clearly understood to be micro-averaged. The average over all case-activity case-pairs was prone to misinterpretation. Furthermore we improved the documentation of the code to reflect this aspect.

More details on micro-averaging: https://doi.org/10.1016/j.ipm.2009.03.002

0.2.1 (2025-05-01)

Resolved some dependency issues (by updating all requirements to newest version).

0.2.0 (2024-10-11)

Added additional properties for the EventAnalysis and TwoSet classes, for a better overview of the performance of methods. The main additions are:

• The TwoSet class now offers the properties precision, recall, f1, and balanced_accuracy.

• The EventAnalysis class now offers the properties precision, recall, and f1 (balanced_accuracy does not make sense here, since there is no notion of true negative events).

0.1.1 (2024-08-14)

• Added additional validations and checks for the input logs, with helpful tips in errors in case of non-compliance.

• Minor bug fixes.

0.1.0 (2024-06-19)

• First release on PyPI.