ergminer 0.2.2

Event Relationship Graph mining from event logs

ERG Miner - Event Relationship Graph Mining

A Python package for mining Event Relationship Graphs (ERGs) from event logs using process mining, state variable identification, and statistical analysis.

Features

The ERG Miner implements a complete pipeline for extracting Event Relationship Graphs from event logs:

flowchart TB
    subgraph L1[Level 1 - Input & Discovery]
        direction LR
        A[Event Log CSV] -->|Load & Preprocess| B[Trace Cleaning]
        B -->|Remove Incomplete Traces| C[Process Mining]
        C -->|PM4PY Alpha Miner| D[Queueing Network Mapping]
        D -->|Map to Event Types| E[Synthetic Start/End Nodes]
    end

    subgraph L2[Level 2 - Behavioral Inference]
        direction LR
        F[State Variable Identification] -->|Server Busy, Queue Length| G[Conditional Guard Detection]
        G -->|ML Pattern Recognition| H[Delay Distribution Analysis]
        H -->|Exponential, Normal, Uniform| I[Immediate vs Delayed Arcs]
        I -->|Classify Transitions| J[Node State Equations]
    end

    subgraph L3[Level 3 - ERG Assembly]
        direction LR
        K[Complete ERG Structure]
        K --> L[Export to JSON/ERGML/DOT]
        K --> M[Visualizations]
    end

    subgraph L4[Level 4 - Validation & Reporting]
        direction LR
        N[ERG Simulation] -->|SimPy| O[Generate Synthetic Logs]
        O --> P[Conformance Testing]
        P -->|Token Replay, Activity, Routing, Timing| Q[Fitness Metrics & Reports]
    end

    E -->|Add Begin & End| F
    J -->|Update Equations| K
    K --> N
    
    style A fill:#e1f5ff
    style K fill:#c8e6c9
    style Q fill:#fff9c4

Key Capabilities:

• Process Mining - Uses PM4PY's Alpha Miner to discover process structure
• Queueing Network Mapping - Maps activities to queueing network event types (Arrive, Start Process, Run, End Process, Rework, Failure)
• Synthetic Start/End Nodes - Adds "Begin" and "End" nodes to mark the start and end of the ERG
• State Variable Identification - Identifies state variables (Server Busy, Queue Length) and their update equations
• Conditional Guard Detection - Uses pattern recognition and machine learning to detect conditional guards on arcs
• Delay Distribution Analysis - Determines probabilities and delay distributions (exponential, normal, uniform, etc.)
• Immediate vs Delayed Arcs - Distinguishes between immediate transitions and delayed transitions
• Trace Cleaning - Removes incomplete/partial traces before mining (configurable)
• Node State Equations - Displays state variable update equations on each non-terminal node
• Immediate Arc Fallback - Uses probability labels when no conditional guard can be found
• ERG Simulation - Simulates the ERG using SimPy to generate synthetic event logs
• Conformance Testing - Validates the mined ERG against the original event log using token replay, activity frequency, routing, and timing fitness metrics

Installation

From Source

# Clone the repository
git clone https://github.com/zeyde-eng/ERG_miner.git
cd ERG_miner

# Install in development mode (includes dev tools)
pip install -e .[dev]

Dependencies

The package requires:

• Python >= 3.9
• pandas >= 1.5
• numpy >= 1.23
• scikit-learn >= 1.2
• pm4py >= 2.7
• scipy >= 1.10
• matplotlib >= 3.7
• networkx >= 3.0
• graphviz >= 0.20 (Python bindings)

Graphviz System Installation

The graphviz Python package is only a binding — you must also install the Graphviz system executables (dot, etc.) and ensure they are on your PATH, otherwise ERG visualization/export to DOT/PNG/SVG will fail.

• Windows: download the installer from https://graphviz.org/download/ and check the option "Add Graphviz to the system PATH" during installation (or install via winget install graphviz / choco install graphviz).
• macOS: brew install graphviz
• Linux (Debian/Ubuntu): sudo apt-get install graphviz
• Linux (Fedora/RHEL): sudo dnf install graphviz

Verify the install with:

dot -V

Project Structure

ERG_miner/
├── examples/
│   └── basic_usage.py            # Complete pipeline example (configurable)
│   └── data/                     # Sample event logs 
├── tests/                        # pytest test suite
│   ├── test_read.py
│   ├── test_utils.py
│   ├── test_filtering.py
│   └── test_discovery.py
├── src/
│   └── ergminer/                 # Installable package
│       ├── __init__.py           # Public API
│       ├── py.typed              # PEP 561 type marker
│       ├── read.py               # read_csv, read_xes, read_dataframe
│       ├── utils.py              # format_dataframe, get_start/end_activities, etc.
│       ├── filtering.py          # filter_start/end_activities, filter_case_size, etc.
│       ├── discovery.py          # discover_erg
│       ├── write.py              # write_ergml, write_erg_json, write_dot
│       ├── sim.py                # play_out
│       ├── conformance.py        # conformance_erg
│       ├── vis.py                # save_vis_erg, view_erg
│       ├── erg_structure.py      # ERG, ERGNode, ERGArc, ERGStateVariable
│       ├── erg_miner.py          # ERGMiner class, DelayConfig
│       ├── erg_plotter.py        # matplotlib / graphviz rendering
│       ├── erg_playback.py       # ERG discrete-event simulator
│       ├── ERGML_converter.py    # ERG ↔ ERGML XML
│       ├── conformance_testing.py# ERGConformanceChecker, ConformanceResult
│       ├── event_log_loader.py   # Internal log loading / preprocessing
│       ├── process_miner.py      # PM4Py Alpha Miner integration
│       ├── state_variables.py    # State variable identification
│       ├── guard_detector.py     # ML-based conditional guard detection
│       └── delay_analyzer.py     # Delay distribution fitting
├── pyproject.toml                # Build config and dependencies
├── requirements.txt              # Dev shortcut: -e .[dev]
├── LICENSE                       # GPL-3.0-or-later
└── README.md

Quick Start

Run the Complete Pipeline

The easiest way to get started is to run the complete pipeline on the provided sample data:

python examples/basic_usage.py

This will:

1. Mine the ERG from the configured event log (default: examples/data/parallel_mmc.csv)
2. Generate visualizations (ERG graph with probabilities, guards, and distributions)
3. Export results to JSON, ERGML, and DOT formats
4. Run conformance testing using ERG simulation
5. Save all outputs to the output/ directory

You can configure the input file, column names, delay analysis parameters, and conformance testing settings by editing the CONFIG section at the top of examples/basic_usage.py.

Basic Usage

import ergminer

# Load event log
log = ergminer.read_csv(
    'data/event_log.csv',
    case_id_col='case_id',
    activity_col='activity_name',
    timestamp_col='timestamp',
    resource_col='resource_id',
)

# Mine ERG
erg = ergminer.discover_erg(log)

# Print summary
erg.print_summary()

# Export results
ergminer.write_ergml(erg,     'output/erg.ergml', sim_time=10000.0)
ergminer.write_erg_json(erg,  'output/erg.json')
ergminer.write_dot(erg,       'output/erg.dot')

Configuring Delay Analysis

Fine-tune delay detection and distribution fitting with DelayConfig:

import ergminer

delay_config = ergminer.DelayConfig(
    immediate_threshold=0.01,           # delays below this are instantaneous (seconds)
    immediate_fraction_min=0.02,        # min fraction of near-zero delays to be immediate
    resource_blocked_fraction_min=0.5,  # min fraction of blocked arcs to confirm immediacy
    min_sample_size=10,                 # min observations before fitting a distribution
    min_p_value=0.05,                   # min KS p-value to accept a fit
    ks_stat_tolerance=0.05,             # tolerance for preferring simpler distributions
)

log = ergminer.read_csv('data/event_log.csv')
erg = ergminer.discover_erg(log, delay_config=delay_config)

Simulation and Conformance Testing

The package includes ERG simulation and conformance testing to validate the mined ERG against the original event log:

import ergminer

# Simulate the ERG (returns a combined DataFrame of all runs)
sim_log = ergminer.play_out(erg, n=10, sim_time=10000.0, seed=42)

# Run conformance checking (pass sim_log to avoid re-simulating)
result = ergminer.conformance_erg(
    log, erg,
    sim_log=sim_log,
    n_simulations=10,
    sim_time=10000.0,
    seed=42,
    case_id_col='case_id',
    activity_col='activity_name',
    timestamp_col='timestamp',
    verbose=True,
)

# Print the full conformance report
result.print_report()

# Access group-level scores
scores = {grp: info['score'] for grp, info in result.group_scores.items()}
print("Group scores:", scores)

The conformance testing feature:

• Simulates the ERG multiple times to generate synthetic event logs
• Compares simulated logs with the original event log across twelve fitness metrics
• Groups results by: Structural Validity, Process Behaviour, Resource & Timing, and Routing
• Reports per-check and group-level scores with pass/fail thresholds

Visualization

The package includes powerful visualization capabilities using matplotlib and networkx.

Plot ERG Graph

import ergminer

# Save as PNG and SVG
ergminer.save_vis_erg(
    erg,
    'output/erg_graph.png',
    show_probabilities=True,
    show_distributions=True,
    show_guards=True,
    title='My ERG',
)

# Open in an interactive window
ergminer.view_erg(erg)

Visualization Features

• Node Colors: Different colors for each event type (Arrive, Run, End Process, etc.)
• Node Shapes: Diamonds for synthetic Begin/End nodes, circles for real events
• Edge Styles: Solid lines for immediate arcs, dashed lines for delayed arcs
• Labels: Probabilities, guard conditions, and distribution types on arcs
• Statistics: Bar charts, pie charts, and histograms showing ERG characteristics

Event Log Format

The event log should be in CSV format with at least the following columns:

• Case ID: Identifies the case/trace
• Activity: Name of the activity/event
• Timestamp: Time when the event occurred

Optional columns:

• Resource: Resource that executed the activity

Example Event Log

case_id,activity_name,timestamp,resource
1,Arrive,2023-01-01 10:00:00,
1,Start Process,2023-01-01 10:01:00,Server1
1,Run,2023-01-01 10:02:00,Server1
1,End Process,2023-01-01 10:05:00,Server1
2,Arrive,2023-01-01 10:03:00,
2,Start Process,2023-01-01 10:05:30,Server1
2,Run,2023-01-01 10:06:00,Server1
2,End Process,2023-01-01 10:08:00,Server1

ERG Structure

The mined ERG contains:

Synthetic Start/End Nodes

The ERG automatically includes two special synthetic nodes:

• Begin - Marks the start of the ERG (connects to all start activities from the event log)
• End - Marks the end of the ERG (all terminal activities connect to it)

These nodes are NOT mapped from actual events in the event log. They have:

• original_activity = '<synthetic>'
• frequency = 0
• They exist solely to provide clear entry/exit points for the ERG

Parameters

Parameters extract from the Event logs (used in simulations)

• Simulation Time
• Number of Job (not implemented)
• Start Events
• End Events
• Statistics

Nodes (Events)

Each node represents an event/activity with:

• Name
• Event type (queueing activity type)
• Resource association
• Frequency

Arcs (Transitions)

Each arc represents a transition between events with:

• Source and target nodes
• Transition probability
• Guard condition (if any)
• Delay distribution (immediate or with parameters)
• State variable updates

State Variables

Each state variable has:

• Name
• Type (Server_Busy or Queue_Length)
• Associated resource
• Initial value
• Update equations

Output Formats

JSON Export

Complete ERG structure in JSON format:

{
  "name": "My_ERG",
  "nodes": [...],
  "arcs": [...],
  "state_variables": [...],
  "start events": [...],
  "end events": [...],
  "statistics": [...]
}

ERGML Export

Event Relationship Graph Markup Language (ERGML) for simulation:

<erg name="My_ERG">
  <parameters>...</parameters>
  <state_variables>...</state_variables>
  <nodes>...</nodes>
  <arcs>...</arcs>
</erg>

The ERGML format is specifically designed for use with the ERG simulator and includes:

• Complete state variable definitions with update equations
• Node definitions with event types and frequencies
• Arc definitions with guards, probabilities, and delay distributions
• Simulation time parameter for conformance testing

GraphViz DOT Export

Visual representation for graphical rendering:

# Generate PNG image from DOT file
dot -Tpng output/erg.dot -o erg.png

API Reference

Reading

Function	Description
ergminer.read_csv(filepath, ...)	Load event log from a CSV file
ergminer.read_xes(filepath, ...)	Load event log from an XES file
ergminer.read_dataframe(df, ...)	Wrap an existing DataFrame as an event log

Utilities

Function	Description
ergminer.format_dataframe(df, ...)	Rename columns to ergminer standard names
ergminer.get_start_activities(log)	Return start activity counts
ergminer.get_end_activities(log)	Return end activity counts
ergminer.get_variants(log)	Return variant frequencies
ergminer.get_activity_labels(log)	Return list of unique activities

Filtering

Function	Description
ergminer.filter_start_activities(log, activities)	Keep cases starting with given activities
ergminer.filter_end_activities(log, activities)	Keep cases ending with given activities
ergminer.filter_case_size(log, min_size, max_size)	Keep cases within a size range
ergminer.filter_variants(log, variants)	Keep cases matching given variants

Discovery

Function	Description
ergminer.discover_erg(log, ...)	Mine an ERG from an event log

discover_erg key parameters:

• case_id_col, activity_col, timestamp_col, resource_col — column names
• delay_config — DelayConfig instance for tuning delay analysis
• remove_incomplete_traces — drop partial traces (default: True)
• min_events_per_trace — minimum trace length to keep (default: 2)
• erg_name — name for the resulting ERG
• verbose — print progress

Writing

Function	Description
ergminer.write_ergml(erg, file_path, sim_time)	Export to ERGML XML format
ergminer.write_erg_json(erg, file_path)	Export to JSON format
ergminer.write_dot(erg, file_path)	Export to GraphViz DOT format

Simulation

Function	Description
ergminer.play_out(erg, n, sim_time, seed, ...)	Simulate ERG for n runs; returns combined DataFrame

Conformance

Function	Description
ergminer.conformance_erg(log, erg, ...)	Run conformance checking; returns ConformanceResult

ConformanceResult attributes:

• overall_score — mean score across all checks (0–1)
• group_scores — {group_name: {'score': float, 'pass': bool, 'checks': int}}
• checker_results — per-check detailed results dict
• summary_df — DataFrame summary of all checks
• print_report() — print a formatted report to stdout

Visualisation

Function	Description
ergminer.save_vis_erg(erg, filepath, ...)	Save ERG visualisation to PNG or SVG
ergminer.view_erg(erg, ...)	Open ERG in an interactive window

Data Classes

DelayConfig — tune delay analysis:

• immediate_threshold (float, default 0.01) — delays below this (seconds) are instantaneous
• immediate_fraction_min (float, default 0.02) — min fraction of near-zero delays to classify as immediate
• resource_blocked_fraction_min (float, default 0.5) — min fraction blocked to confirm immediacy
• min_sample_size (int, default 10) — min observations before fitting a distribution
• min_p_value (float, default 0.05) — min KS p-value to accept a fit
• ks_stat_tolerance (float, default 0.05) — tolerance for preferring simpler distributions

ERG — the mined graph object:

• get_statistics() — returns node/arc/state-variable counts
• print_summary(verbose) — prints human-readable summary
• generate_graphviz_dot() — returns DOT string

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

License

This project is licensed under the GNU General Public License v3.0 or later (GPLv3+).

Author

Zach Eyde ([e-mail])

Citation

If you use this software in your research, please cite:

Eyde, Z. (2026). ERG Miner: Event Relationship Graph Mining from Event Logs.
GitHub repository: https://github.com/zeyde-eng/ERG_miner

Acknowledgments

This package builds upon:

• PM4PY for process mining algorithms
• scikit-learn for pattern recognition
• scipy for statistical distribution fitting
• networkx for graph analysis
• graphviz / matplotlib for visualisation