Metadata-Version: 2.4
Name: ergminer
Version: 0.2.2
Summary: Event Relationship Graph mining from event logs
Author-email: Zach Eyde <zeyde@asu.edu>
License-Expression: GPL-3.0-or-later
Project-URL: Homepage, https://github.com/zeyde-eng/ERG_miner
Project-URL: Documentation, https://zeyde-eng.github.io/ERG_miner/
Project-URL: Repository, https://github.com/zeyde-eng/ERG_miner
Project-URL: Issues, https://github.com/zeyde-eng/ERG_miner/issues
Keywords: event mining,process mining,event logs,relationship graph
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.5
Requires-Dist: numpy>=1.23
Requires-Dist: scikit-learn>=1.2
Requires-Dist: pm4py>=2.7
Requires-Dist: scipy>=1.10
Requires-Dist: matplotlib>=3.7
Requires-Dist: networkx>=3.0
Requires-Dist: graphviz>=0.20
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Requires-Dist: twine>=4.0; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.0; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.24; extra == "docs"
Dynamic: license-file

# ERG Miner - Event Relationship Graph Mining

A Python package for mining Event Relationship Graphs (ERGs) from event logs using process mining, state variable identification, and statistical analysis.

## Features

The ERG Miner implements a complete pipeline for extracting Event Relationship Graphs from event logs:

```mermaid
flowchart TB
    subgraph L1[Level 1 - Input & Discovery]
        direction LR
        A[Event Log CSV] -->|Load & Preprocess| B[Trace Cleaning]
        B -->|Remove Incomplete Traces| C[Process Mining]
        C -->|PM4PY Alpha Miner| D[Queueing Network Mapping]
        D -->|Map to Event Types| E[Synthetic Start/End Nodes]
    end

    subgraph L2[Level 2 - Behavioral Inference]
        direction LR
        F[State Variable Identification] -->|Server Busy, Queue Length| G[Conditional Guard Detection]
        G -->|ML Pattern Recognition| H[Delay Distribution Analysis]
        H -->|Exponential, Normal, Uniform| I[Immediate vs Delayed Arcs]
        I -->|Classify Transitions| J[Node State Equations]
    end

    subgraph L3[Level 3 - ERG Assembly]
        direction LR
        K[Complete ERG Structure]
        K --> L[Export to JSON/ERGML/DOT]
        K --> M[Visualizations]
    end

    subgraph L4[Level 4 - Validation & Reporting]
        direction LR
        N[ERG Simulation] -->|SimPy| O[Generate Synthetic Logs]
        O --> P[Conformance Testing]
        P -->|Token Replay, Activity, Routing, Timing| Q[Fitness Metrics & Reports]
    end

    E -->|Add Begin & End| F
    J -->|Update Equations| K
    K --> N
    
    style A fill:#e1f5ff
    style K fill:#c8e6c9
    style Q fill:#fff9c4
```

**Key Capabilities:**
- **Process Mining** - Uses PM4PY's Alpha Miner to discover process structure
- **Queueing Network Mapping** - Maps activities to queueing network event types (Arrive, Start Process, Run, End Process, Rework, Failure)
- **Synthetic Start/End Nodes** - Adds "Begin" and "End" nodes to mark the start and end of the ERG
- **State Variable Identification** - Identifies state variables (Server Busy, Queue Length) and their update equations
- **Conditional Guard Detection** - Uses pattern recognition and machine learning to detect conditional guards on arcs
- **Delay Distribution Analysis** - Determines probabilities and delay distributions (exponential, normal, uniform, etc.)
- **Immediate vs Delayed Arcs** - Distinguishes between immediate transitions and delayed transitions
- **Trace Cleaning** - Removes incomplete/partial traces before mining (configurable)
- **Node State Equations** - Displays state variable update equations on each non-terminal node
- **Immediate Arc Fallback** - Uses probability labels when no conditional guard can be found
- **ERG Simulation** - Simulates the ERG using SimPy to generate synthetic event logs
- **Conformance Testing** - Validates the mined ERG against the original event log using token replay, activity frequency, routing, and timing fitness metrics

## Installation

### From Source

```bash
# Clone the repository
git clone https://github.com/zeyde-eng/ERG_miner.git
cd ERG_miner

# Install in development mode (includes dev tools)
pip install -e .[dev]
```

### Dependencies

The package requires:
- Python >= 3.9
- pandas >= 1.5
- numpy >= 1.23
- scikit-learn >= 1.2
- pm4py >= 2.7
- scipy >= 1.10
- matplotlib >= 3.7
- networkx >= 3.0
- graphviz >= 0.20 (Python bindings)

### Graphviz System Installation

The `graphviz` Python package is only a binding — you must also install the
Graphviz **system executables** (`dot`, etc.) and ensure they are on your `PATH`,
otherwise ERG visualization/export to DOT/PNG/SVG will fail.

- **Windows:** download the installer from <https://graphviz.org/download/> and
  check the option *"Add Graphviz to the system PATH"* during installation
  (or install via `winget install graphviz` / `choco install graphviz`).
- **macOS:** `brew install graphviz`
- **Linux (Debian/Ubuntu):** `sudo apt-get install graphviz`
- **Linux (Fedora/RHEL):** `sudo dnf install graphviz`

Verify the install with:

```bash
dot -V
```


## Project Structure

```
ERG_miner/
├── examples/
│   └── basic_usage.py            # Complete pipeline example (configurable)
│   └── data/                     # Sample event logs 
├── tests/                        # pytest test suite
│   ├── test_read.py
│   ├── test_utils.py
│   ├── test_filtering.py
│   └── test_discovery.py
├── src/
│   └── ergminer/                 # Installable package
│       ├── __init__.py           # Public API
│       ├── py.typed              # PEP 561 type marker
│       ├── read.py               # read_csv, read_xes, read_dataframe
│       ├── utils.py              # format_dataframe, get_start/end_activities, etc.
│       ├── filtering.py          # filter_start/end_activities, filter_case_size, etc.
│       ├── discovery.py          # discover_erg
│       ├── write.py              # write_ergml, write_erg_json, write_dot
│       ├── sim.py                # play_out
│       ├── conformance.py        # conformance_erg
│       ├── vis.py                # save_vis_erg, view_erg
│       ├── erg_structure.py      # ERG, ERGNode, ERGArc, ERGStateVariable
│       ├── erg_miner.py          # ERGMiner class, DelayConfig
│       ├── erg_plotter.py        # matplotlib / graphviz rendering
│       ├── erg_playback.py       # ERG discrete-event simulator
│       ├── ERGML_converter.py    # ERG ↔ ERGML XML
│       ├── conformance_testing.py# ERGConformanceChecker, ConformanceResult
│       ├── event_log_loader.py   # Internal log loading / preprocessing
│       ├── process_miner.py      # PM4Py Alpha Miner integration
│       ├── state_variables.py    # State variable identification
│       ├── guard_detector.py     # ML-based conditional guard detection
│       └── delay_analyzer.py     # Delay distribution fitting
├── pyproject.toml                # Build config and dependencies
├── requirements.txt              # Dev shortcut: -e .[dev]
├── LICENSE                       # GPL-3.0-or-later
└── README.md
```

## Quick Start

### Run the Complete Pipeline

The easiest way to get started is to run the complete pipeline on the provided sample data:

```bash
python examples/basic_usage.py
```

This will:
1. Mine the ERG from the configured event log (default: `examples/data/parallel_mmc.csv`)
2. Generate visualizations (ERG graph with probabilities, guards, and distributions)
3. Export results to JSON, ERGML, and DOT formats
4. Run conformance testing using ERG simulation
5. Save all outputs to the `output/` directory

You can configure the input file, column names, delay analysis parameters, and conformance testing settings by editing the CONFIG section at the top of `examples/basic_usage.py`.

### Basic Usage

```python
import ergminer

# Load event log
log = ergminer.read_csv(
    'data/event_log.csv',
    case_id_col='case_id',
    activity_col='activity_name',
    timestamp_col='timestamp',
    resource_col='resource_id',
)

# Mine ERG
erg = ergminer.discover_erg(log)

# Print summary
erg.print_summary()

# Export results
ergminer.write_ergml(erg,     'output/erg.ergml', sim_time=10000.0)
ergminer.write_erg_json(erg,  'output/erg.json')
ergminer.write_dot(erg,       'output/erg.dot')
```

### Configuring Delay Analysis

Fine-tune delay detection and distribution fitting with `DelayConfig`:

```python
import ergminer

delay_config = ergminer.DelayConfig(
    immediate_threshold=0.01,           # delays below this are instantaneous (seconds)
    immediate_fraction_min=0.02,        # min fraction of near-zero delays to be immediate
    resource_blocked_fraction_min=0.5,  # min fraction of blocked arcs to confirm immediacy
    min_sample_size=10,                 # min observations before fitting a distribution
    min_p_value=0.05,                   # min KS p-value to accept a fit
    ks_stat_tolerance=0.05,             # tolerance for preferring simpler distributions
)

log = ergminer.read_csv('data/event_log.csv')
erg = ergminer.discover_erg(log, delay_config=delay_config)
```

### Simulation and Conformance Testing

The package includes ERG simulation and conformance testing to validate the mined ERG against the original event log:

```python
import ergminer

# Simulate the ERG (returns a combined DataFrame of all runs)
sim_log = ergminer.play_out(erg, n=10, sim_time=10000.0, seed=42)

# Run conformance checking (pass sim_log to avoid re-simulating)
result = ergminer.conformance_erg(
    log, erg,
    sim_log=sim_log,
    n_simulations=10,
    sim_time=10000.0,
    seed=42,
    case_id_col='case_id',
    activity_col='activity_name',
    timestamp_col='timestamp',
    verbose=True,
)

# Print the full conformance report
result.print_report()

# Access group-level scores
scores = {grp: info['score'] for grp, info in result.group_scores.items()}
print("Group scores:", scores)
```

The conformance testing feature:
- **Simulates** the ERG multiple times to generate synthetic event logs
- **Compares** simulated logs with the original event log across twelve fitness metrics
- **Groups** results by: Structural Validity, Process Behaviour, Resource & Timing, and Routing
- **Reports** per-check and group-level scores with pass/fail thresholds

## Visualization

The package includes powerful visualization capabilities using matplotlib and networkx.

### Plot ERG Graph

```python
import ergminer

# Save as PNG and SVG
ergminer.save_vis_erg(
    erg,
    'output/erg_graph.png',
    show_probabilities=True,
    show_distributions=True,
    show_guards=True,
    title='My ERG',
)

# Open in an interactive window
ergminer.view_erg(erg)
```

### Visualization Features

- **Node Colors**: Different colors for each event type (Arrive, Run, End Process, etc.)
- **Node Shapes**: Diamonds for synthetic Begin/End nodes, circles for real events
- **Edge Styles**: Solid lines for immediate arcs, dashed lines for delayed arcs
- **Labels**: Probabilities, guard conditions, and distribution types on arcs
- **Statistics**: Bar charts, pie charts, and histograms showing ERG characteristics

## Event Log Format

The event log should be in CSV format with at least the following columns:
- **Case ID**: Identifies the case/trace
- **Activity**: Name of the activity/event
- **Timestamp**: Time when the event occurred

Optional columns:
- **Resource**: Resource that executed the activity

### Example Event Log

```csv
case_id,activity_name,timestamp,resource
1,Arrive,2023-01-01 10:00:00,
1,Start Process,2023-01-01 10:01:00,Server1
1,Run,2023-01-01 10:02:00,Server1
1,End Process,2023-01-01 10:05:00,Server1
2,Arrive,2023-01-01 10:03:00,
2,Start Process,2023-01-01 10:05:30,Server1
2,Run,2023-01-01 10:06:00,Server1
2,End Process,2023-01-01 10:08:00,Server1
```

## ERG Structure

The mined ERG contains:

### Synthetic Start/End Nodes

The ERG automatically includes two special synthetic nodes:
- **Begin** - Marks the start of the ERG (connects to all start activities from the event log)
- **End** - Marks the end of the ERG (all terminal activities connect to it)

These nodes are NOT mapped from actual events in the event log. They have:
- `original_activity = '<synthetic>'`
- `frequency = 0`
- They exist solely to provide clear entry/exit points for the ERG

### Parameters
Parameters extract from the Event logs (used in simulations)
- Simulation Time
- Number of Job (not implemented)
- Start Events
- End Events
- Statistics

### Nodes (Events)
Each node represents an event/activity with:
- Name
- Event type (queueing activity type)
- Resource association
- Frequency

### Arcs (Transitions)
Each arc represents a transition between events with:
- Source and target nodes
- Transition probability
- Guard condition (if any)
- Delay distribution (immediate or with parameters)
- State variable updates

### State Variables
Each state variable has:
- Name
- Type (Server_Busy or Queue_Length)
- Associated resource
- Initial value
- Update equations

## Output Formats

### JSON Export
Complete ERG structure in JSON format:
```json
{
  "name": "My_ERG",
  "nodes": [...],
  "arcs": [...],
  "state_variables": [...],
  "start events": [...],
  "end events": [...],
  "statistics": [...]
}
```

### ERGML Export
Event Relationship Graph Markup Language (ERGML) for simulation:
```xml
<erg name="My_ERG">
  <parameters>...</parameters>
  <state_variables>...</state_variables>
  <nodes>...</nodes>
  <arcs>...</arcs>
</erg>
```

The ERGML format is specifically designed for use with the ERG simulator and includes:
- Complete state variable definitions with update equations
- Node definitions with event types and frequencies
- Arc definitions with guards, probabilities, and delay distributions
- Simulation time parameter for conformance testing

### GraphViz DOT Export
Visual representation for graphical rendering:
```bash
# Generate PNG image from DOT file
dot -Tpng output/erg.dot -o erg.png
```


## API Reference

### Reading

| Function | Description |
|---|---|
| `ergminer.read_csv(filepath, ...)` | Load event log from a CSV file |
| `ergminer.read_xes(filepath, ...)` | Load event log from an XES file |
| `ergminer.read_dataframe(df, ...)` | Wrap an existing DataFrame as an event log |

### Utilities

| Function | Description |
|---|---|
| `ergminer.format_dataframe(df, ...)` | Rename columns to ergminer standard names |
| `ergminer.get_start_activities(log)` | Return start activity counts |
| `ergminer.get_end_activities(log)` | Return end activity counts |
| `ergminer.get_variants(log)` | Return variant frequencies |
| `ergminer.get_activity_labels(log)` | Return list of unique activities |

### Filtering

| Function | Description |
|---|---|
| `ergminer.filter_start_activities(log, activities)` | Keep cases starting with given activities |
| `ergminer.filter_end_activities(log, activities)` | Keep cases ending with given activities |
| `ergminer.filter_case_size(log, min_size, max_size)` | Keep cases within a size range |
| `ergminer.filter_variants(log, variants)` | Keep cases matching given variants |

### Discovery

| Function | Description |
|---|---|
| `ergminer.discover_erg(log, ...)` | Mine an ERG from an event log |

**`discover_erg` key parameters:**
- `case_id_col`, `activity_col`, `timestamp_col`, `resource_col` — column names
- `delay_config` — `DelayConfig` instance for tuning delay analysis
- `remove_incomplete_traces` — drop partial traces (default: `True`)
- `min_events_per_trace` — minimum trace length to keep (default: `2`)
- `erg_name` — name for the resulting ERG
- `verbose` — print progress

### Writing

| Function | Description |
|---|---|
| `ergminer.write_ergml(erg, file_path, sim_time)` | Export to ERGML XML format |
| `ergminer.write_erg_json(erg, file_path)` | Export to JSON format |
| `ergminer.write_dot(erg, file_path)` | Export to GraphViz DOT format |

### Simulation

| Function | Description |
|---|---|
| `ergminer.play_out(erg, n, sim_time, seed, ...)` | Simulate ERG for `n` runs; returns combined DataFrame |

### Conformance

| Function | Description |
|---|---|
| `ergminer.conformance_erg(log, erg, ...)` | Run conformance checking; returns `ConformanceResult` |

**`ConformanceResult` attributes:**
- `overall_score` — mean score across all checks (0–1)
- `group_scores` — `{group_name: {'score': float, 'pass': bool, 'checks': int}}`
- `checker_results` — per-check detailed results dict
- `summary_df` — DataFrame summary of all checks
- `print_report()` — print a formatted report to stdout

### Visualisation

| Function | Description |
|---|---|
| `ergminer.save_vis_erg(erg, filepath, ...)` | Save ERG visualisation to PNG or SVG |
| `ergminer.view_erg(erg, ...)` | Open ERG in an interactive window |

### Data Classes

**`DelayConfig`** — tune delay analysis:
- `immediate_threshold` (float, default `0.01`) — delays below this (seconds) are instantaneous
- `immediate_fraction_min` (float, default `0.02`) — min fraction of near-zero delays to classify as immediate
- `resource_blocked_fraction_min` (float, default `0.5`) — min fraction blocked to confirm immediacy
- `min_sample_size` (int, default `10`) — min observations before fitting a distribution
- `min_p_value` (float, default `0.05`) — min KS p-value to accept a fit
- `ks_stat_tolerance` (float, default `0.05`) — tolerance for preferring simpler distributions

**`ERG`** — the mined graph object:
- `get_statistics()` — returns node/arc/state-variable counts
- `print_summary(verbose)` — prints human-readable summary
- `generate_graphviz_dot()` — returns DOT string

## Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

## License

This project is licensed under the GNU General Public License v3.0 or later (GPLv3+).

## Author

Zach Eyde (zeyde@asu.edu)

## Citation

If you use this software in your research, please cite:

```
Eyde, Z. (2026). ERG Miner: Event Relationship Graph Mining from Event Logs.
GitHub repository: https://github.com/zeyde-eng/ERG_miner
```

## Acknowledgments

This package builds upon:
- PM4PY for process mining algorithms
- scikit-learn for pattern recognition
- scipy for statistical distribution fitting
- networkx for graph analysis
- graphviz / matplotlib for visualisation
