catbench 1.0.0

CatBench: Benchmark Framework of Machine Learning Interatomic Potentials for Adsorption Energy Predictions in Heterogeneous Catalysis

CatBench

CatBench Framework for Benchmarking Machine Learning Interatomic Potentials in Adsorption Energy Predictions for Heterogeneous Catalysis

CatBench provides a unified framework for evaluating MLIP performance across diverse catalytic systems, offering automated data processing, calculation workflows, and comprehensive analysis tools for adsorption energies, surface energies, bulk formation energies, and equation of state properties.

If you want to use MLIPs in your catalysis research, CatBench enables you to establish quantitative reliability through systematic benchmarking against DFT references.

Quick Navigation

• Installation
• Overview
• Adsorption Energy Benchmarking
• Relative Energy Benchmarking
  • Surface Energy
  • Bulk Formation Energy
• Equation of State (EOS) Benchmarking
• Configuration Options
• Citation

Installation

# Basic installation (core features only)
pip install catbench

# With D3 dispersion correction support (requires CUDA)
pip install catbench[d3]

# Development installation
git clone https://github.com/JinukMoon/CatBench.git
cd CatBench
pip install -e .

# Development with D3 support
pip install -e .[d3]

Installation Options

• Basic: Core benchmarking features without dispersion correction
• [d3]: Includes PyTorch for GPU-accelerated D3 dispersion correction

Note: D3 dispersion correction requires CUDA toolkit for GPU acceleration. CPU-only mode is not currently supported.

Overview

CatBench follows a three-step workflow for comprehensive MLIP evaluation:

1. Data Processing: Automated download from CatHub or processing of user VASP calculations
2. Calculation: MLIP-based adsorption and reaction energy calculations
3. Analysis: Statistical evaluation, anomaly detection, and visualization

Adsorption Energy Benchmarking

Data Preparation

Option A: CatHub Database

Download and preprocess catalysis reaction data directly from CatHub:

from catbench.adsorption import cathub_preprocessing

# Single benchmark dataset
cathub_preprocessing("MamunHighT2019")

# Multiple datasets with adsorbate name integration
cathub_preprocessing(
    ["MamunHighT2019", "AraComputational2022"],
    adsorbate_integration={'HO': 'OH', 'O2H': 'OOH'}  # Unify naming conventions
)

Option B: User VASP Data

⚠️ Important: The VASP preprocessing functions will DELETE all files except CONTCAR and OSZICAR to save disk space. Always work with a copy of your original VASP data!

# STRONGLY RECOMMENDED: Copy your original data first
cp -r original_vasp_data/ your_dataset_name/

Organize your VASP calculation folders following this hierarchy (folder name is customizable):

your_dataset_name/  # You can use any name for this folder
├── gas/
│   ├── H2gas/            # Complete VASP calculation folder
│   │   ├── INCAR
│   │   ├── POSCAR
│   │   ├── POTCAR
│   │   ├── KPOINTS
│   │   ├── CONTCAR      # Required
│   │   ├── OSZICAR      # Required
│   │   ├── OUTCAR
│   │   ├── vasprun.xml
│   │   └── ...          # All other VASP files
│   └── H2Ogas/
│       ├── CONTCAR
│       ├── OSZICAR
│       └── ...
├── system1/  # e.g., material_1
│   ├── slab/
│   │   ├── CONTCAR
│   │   ├── OSZICAR
│   │   └── ...
│   ├── H/
│   │   ├── 1/
│   │   │   ├── CONTCAR
│   │   │   ├── OSZICAR
│   │   │   └── ...
│   │   └── 2/
│   │       ├── CONTCAR
│   │       ├── OSZICAR
│   │       └── ...
│   └── OH/
│       ├── 1/
│       │   ├── CONTCAR
│       │   ├── OSZICAR
│       │   └── ...
│       └── 2/
│           ├── CONTCAR
│           ├── OSZICAR
│           └── ...
└── system2/  # e.g., material_2
    └── ...

Process the data with coefficient settings:

⚠️ Critical: Required Keywords

• "slab" and "adslab" are mandatory fixed keywords - do NOT change these names
• Gas phase references must end with "gas" suffix (e.g., "H2gas", "COgas", "H2Ogas")
• These naming conventions are hard-coded in CatBench and changing them will cause errors

from catbench.adsorption import vasp_preprocessing

# Define reaction stoichiometry
coeff_setting = {
    "H": {
        "slab": -1,      # E(slab) - REQUIRED: must be exactly "slab"
        "adslab": 1,     # E(H*) - REQUIRED: must be exactly "adslab"
        "H2gas": -1/2,   # -1/2 E(H2) - REQUIRED: must end with "gas"
    },
    "OH": {
        "slab": -1,      # REQUIRED: must be exactly "slab"
        "adslab": 1,     # REQUIRED: must be exactly "adslab"
        "H2gas": +1/2,   # REQUIRED: must end with "gas"
        "H2Ogas": -1,    # REQUIRED: must end with "gas"
    },
}

# Process and prepare data (use your actual folder name here)
vasp_preprocessing("your_dataset_name", coeff_setting)  # Replace "your_dataset_name" with your actual folder name

# Output: Creates raw_data/{your_folder_name}_adsorption.json with all processed data

After processing:

• All VASP files except CONTCAR and OSZICAR are deleted (saves disk space)
• Data is stored in raw_data/{dataset_name}_adsorption.json
• Original folder structure is preserved but cleaned

Calculation

Basic Calculation

from catbench.adsorption import AdsorptionCalculation
from your_mlip import YourCalculator

# Initialize calculators for reproducibility testing
calc_num = 3  # Number of independent calculations
calculators = []
for i in range(calc_num):
    calc = YourCalculator(...)  # Your MLIP with desired settings
    calculators.append(calc)

# Configure and run (only required parameters shown)
config = {
    "mlip_name": "YourMLIP",
    "benchmark": "dataset_name",
    # "rate": None,  # IMPORTANT: Use None to preserve VASP's original fixing constraints
    # "save_files": False,  # Set to False to save disk space by skipping trajectory/log files
    # For all available configuration options, see the Configuration Options section below
}

adsorption_calc = AdsorptionCalculation(calculators, **config)
adsorption_calc.run()

With D3 Dispersion Correction

from catbench.adsorption import AdsorptionCalculation
from catbench.dispersion import DispersionCorrection
from your_mlip import YourCalculator

# Setup D3 correction (using default PBE parameters)
d3_corr = DispersionCorrection()

# Apply D3 to calculators
calc_num = 3
calculators = []
for i in range(calc_num):
    calc = YourCalculator(...)  # Your MLIP with desired settings
    calc_d3 = d3_corr.apply(calc)  # Combine MLIP with D3
    calculators.append(calc_d3)

# Run calculation
config = {
    "mlip_name": "YourMLIP_D3",
    "benchmark": "dataset_name",
    # "rate": None,  # IMPORTANT: Use None to preserve VASP's original fixed atoms
    # "save_files": False,  # Set to False to save disk space by skipping trajectory/log files
    # For all available configuration options, see the Configuration Options section below
}
adsorption_calc = AdsorptionCalculation(calculators, **config)
adsorption_calc.run()

OC20 Mode (Direct Adsorption Energy Prediction)

For MLIPs trained on OC20 dataset that directly predict adsorption energies:

from catbench.adsorption import AdsorptionCalculation
from your_oc20_mlip import OC20Calculator  # Your OC20-trained MLIP

# OC20-trained models (directly predict adsorption energies)
calc_num = 3
calculators = []
for i in range(calc_num):
    oc20_calculator = OC20Calculator(...)  # Your OC20 MLIP with desired settings
    calculators.append(oc20_calculator)

# Run in OC20 mode
config = {
    "mlip_name": "OC20_MLIP",
    "benchmark": "dataset_name",
    # "rate": None,  # IMPORTANT: Use None to preserve VASP's original fixed atoms
    # "save_files": False,  # Set to False to save disk space by skipping trajectory/log files
    # For all available configuration options, see the Configuration Options section below
}
adsorption_calc = AdsorptionCalculation(calculators, mode="oc20", **config)
adsorption_calc.run()

Analysis

from catbench.adsorption import AdsorptionAnalysis

# Configure and run analysis
config = {
    #"mlip_list": ["MLIP_A", "MLIP_B", ...],  # If not set, auto-detects all MLIPs in result folder
    #"font_setting": ["~/fonts/your_font_file.ttf", "sans-serif"],  # Custom font path
    # ... add any other options from Configuration Options section
}

analysis = AdsorptionAnalysis(**config)
analysis.analysis()

This generates:

• Parity plots: Visual comparison of MLIP vs DFT energies
• Excel report: Comprehensive metrics including MAE, RMSE, anomaly statistics
• Anomaly detection: Automatic identification of problematic calculations

All configuration options are documented in the Configuration Options section below.

Threshold Sensitivity Analysis

Evaluate how different threshold values affect anomaly detection:

from catbench.adsorption import AdsorptionAnalysis

analysis = AdsorptionAnalysis()

# Run both threshold sensitivity analyses automatically (default)
analysis.threshold_sensitivity_analysis()

# Or specify a specific mode if needed
analysis.threshold_sensitivity_analysis(mode="disp_thrs")  # Only displacement threshold
analysis.threshold_sensitivity_analysis(mode="bond_length_change_threshold")  # Only bond length threshold

This generates stacked area charts showing how anomaly detection rates change with different threshold values, helping you optimize threshold parameters for your specific system. By default, both displacement and bond length threshold analyses are performed automatically.

Output Files

Results are automatically saved to organized directories with comprehensive analysis reports, parity plots, and Excel summaries containing detailed performance metrics.

1. Parity Plot Analysis

CatBench generates comprehensive parity plots for visual assessment of MLIP performance:

Mono Plot All reactions combined in a single parity plot	Multi Plot Separate parity plots displayed by adsorbate type

2. Comprehensive Excel Analysis

The {current_directory}_Benchmarking_Analysis.xlsx file provides detailed performance metrics across multiple sheets:

Main Performance Comparison

MLIP-to-MLIP performance overview with key metrics:

MLIP	Normal (%)	Anomaly (%)	MAE_total (eV)	MAE_normal (eV)	ADwT (%)	AMDwT (%)	Time/step (ms)
MLIP_A	77.25	14.39	1.118	0.316	77.98	84.71	125.3
MLIP_B	74.22	16.84	0.667	0.512	69.66	80.80	89.7
MLIP_C	80.18	13.51	0.917	0.241	78.97	86.79	156.8
MLIP_D	73.20	16.26	0.738	0.413	71.27	81.03	203.4
MLIP_E	78.45	12.87	0.892	0.298	76.15	83.92	142.1
...	...	...	...	...	...	...	...

Direct performance comparison across all benchmarked MLIPs with comprehensive metrics including ADwT (Accuracy within Threshold) and AMDwT (Anomaly-free Mean Deviation within Threshold)

Anomaly Analysis Sheet

Detailed breakdown of calculation anomalies by category:

MLIP	Normal	Migration	Energy Anom.	Unphys. Relax	Reprod. Fail
MLIP_A	34,869	3,774	590	3,845	2,052
MLIP_B	33,503	4,035	834	5,221	1,537
MLIP_C	36,178	2,847	1,334	3,671	1,100
MLIP_D	33,025	4,759	956	5,372	1,018
...	...	...	...	...	...

Identifies systematic issues and reliability patterns across MLIPs

Individual MLIP Sheets

Adsorbate-specific performance for each MLIP (example from MLIP_A sheet):

Adsorbate	Normal	Anomaly	MAE_total (eV)	MAE_normal (eV)	ADwT (%)	AMDwT (%)
H	1,247	89	0.891	0.234	89.3	93.4
OH	1,156	124	1.045	0.298	82.7	87.1
O	1,089	156	1.234	0.387	78.5	82.9
CO	978	203	1.567	0.445	74.2	78.6
NH3	892	167	1.789	0.512	71.8	76.3
...	...	...	...	...	...	...

Each MLIP has its own sheet revealing adsorbate-specific strengths and weaknesses

3. Threshold Sensitivity Analysis

CatBench provides automated threshold sensitivity analysis to optimize anomaly detection parameters:

Displacement Threshold Analysis Impact of displacement threshold on anomaly detection rates	Bond Length Threshold Analysis Impact of bond length change threshold on anomaly detection rates

Relative Energy Benchmarking

CatBench supports two main types of relative energy calculations: surface energy and bulk formation energy.

Surface Energy

Data Preparation

Warning: Preprocessing functions will DELETE all VASP files except CONTCAR and OSZICAR to save disk space. Always work with copies of your original data.

your_surface_data/  # You can use any name for this folder
├── Material_1/
│   ├── bulk/
│   │   ├── CONTCAR      # Required
│   │   ├── OSZICAR      # Required
│   │   └── ...          # Other VASP files are preserved
│   └── slab/
│       ├── CONTCAR      # Required
│       ├── OSZICAR      # Required
│       └── ...          # Other VASP files are preserved
├── Material_2/
│   ├── bulk/
│   │   ├── CONTCAR
│   │   └── OSZICAR
│   └── slab/
│       ├── CONTCAR
│       └── OSZICAR
├── Material_3/
│   ├── bulk/
│   │   ├── CONTCAR
│   │   └── OSZICAR
│   └── slab/
│       ├── CONTCAR
│       └── OSZICAR
└── Material_4/
    ├── bulk/
    │   ├── CONTCAR
    │   └── OSZICAR
    └── slab/
        ├── CONTCAR
        └── OSZICAR

Each material folder must contain:

• bulk/: Bulk phase calculation
• slab/: Surface slab calculation

from catbench.relative.surface_energy.data import surface_energy_vasp_preprocessing

# Process surface energy data (use your actual folder name here)
surface_energy_vasp_preprocessing("your_surface_data")  # Deletes extra VASP files
# Output: Creates raw_data/{your_surface_data}_surface_energy.json

Calculation

from catbench.relative import SurfaceEnergyCalculation
from your_mlip import YourCalculator

calc = YourCalculator(...)  # Your MLIP with desired settings

surface_calc = SurfaceEnergyCalculation(
    calculator=calc,
    benchmark="surface_benchmark",
    mlip_name="YourMLIP"
)
surface_calc.run()

Analysis

from catbench.relative import RelativeEnergyAnalysis

# Configure and run analysis
config = {
    "task_type": "surface",  # Required: "surface", "bulk_formation", or "custom"
    #"mlip_list": ["MLIP_A", "MLIP_B", ...],  # If not set, auto-detects all MLIPs in result folder
    #"font_setting": ["~/fonts/your_font_file.ttf", "sans-serif"],  # Custom font path
    # ... add any other options from Configuration Options section
}

analysis = RelativeEnergyAnalysis(**config)
analysis.analysis()

Output Files

Results include comprehensive Excel reports with performance metrics and publication-ready parity plots for visual comparison.

Surface Energy Analysis Examples

Surface energy parity plot showing MLIP performance against DFT references for various metal surfaces

The Excel report provides comprehensive surface energy analysis:

MLIP	MAE (J/m²)	RMSE (J/m²)	Max Error (J/m²)	Num_surfaces
MLIP_A	0.185	0.255	1.251	1915
MLIP_B	0.483	0.567	2.110	1915
MLIP_C	0.127	0.182	1.305	1915
MLIP_D	0.261	0.333	1.326	1915
MLIP_E	0.122	0.179	1.358	1915
MLIP_F	0.138	0.194	1.245	1915
MLIP_G	0.119	0.173	1.287	1915
...	...	...	...	...

Bulk Formation Energy

Data Preparation

Warning: Preprocessing functions will DELETE all VASP files except CONTCAR and OSZICAR to save disk space. Always work with copies of your original data.

your_formation_data/  # You can use any name for this folder
├── bulk_compounds/
│   ├── Compound_1/
│   │   ├── INCAR
│   │   ├── POSCAR
│   │   ├── POTCAR
│   │   ├── KPOINTS
│   │   ├── CONTCAR      # Required
│   │   ├── OSZICAR      # Required
│   │   ├── OUTCAR
│   │   ├── vasprun.xml
│   │   └── ...          # All other VASP files
│   └── Compound_2/
│       ├── CONTCAR
│       ├── OSZICAR
│       └── ...
└── elements/
    ├── Element_A/
    │   ├── CONTCAR
    │   ├── OSZICAR
    │   └── ...
    ├── Element_B/
    │   ├── CONTCAR
    │   ├── OSZICAR
    │   └── ...
    └── Element_C/
        ├── CONTCAR
        ├── OSZICAR
        └── ...

from catbench.relative.bulk_formation.data import bulk_formation_vasp_preprocessing

# Define formation reaction stoichiometry
coeff_setting = {
    "Compound_1": {
        "bulk": 1,         # Compound_1
        "Element_A": -1,   # -Element_A
        "Element_C": -1/2, # -1/2 Element_C2
    },
    "Compound_2": {
        "bulk": 1,         # Compound_2
        "Element_B": -2,   # -2Element_B
        "Element_C": -3/2, # -3/2 Element_C2
    },
}

bulk_formation_vasp_preprocessing("your_formation_data", coeff_setting)  # Deletes extra VASP files
# Output: Creates raw_data/{your_folder_name}_bulk.json

Calculation

from catbench.relative import BulkFormationCalculation
from your_mlip import YourCalculator

calc = YourCalculator(...)  # Your MLIP with desired settings

formation_calc = BulkFormationCalculation(
    calculator=calc,
    benchmark="formation_benchmark",
    mlip_name="YourMLIP"
)
formation_calc.run()

Analysis

from catbench.relative import RelativeEnergyAnalysis

# Configure and run analysis
config = {
    "task_type": "bulk_formation",  # Required: "surface", "bulk_formation", or "custom"
    #"mlip_list": ["MLIP_A", "MLIP_B", ...],  # If not set, auto-detects all MLIPs in result folder
    #"font_setting": ["~/fonts/your_font_file.ttf", "sans-serif"],  # Custom font path
    # ... add any other options from Configuration Options section
}

analysis = RelativeEnergyAnalysis(**config)
analysis.analysis()

Output Files

Results include detailed Excel reports with formation energy metrics and comparative parity plots across different MLIPs.

Equation of State (EOS) Benchmarking

Data Preparation

Note: Unlike adsorption preprocessing, EOS preprocessing does NOT delete any files.

EOS data requires multiple volume points for each material:

your_eos_data/  # You can use any name for this folder
├── Material_1/
│   ├── 0/                 # Volume point 0 (smallest)
│   │   ├── INCAR
│   │   ├── POSCAR
│   │   ├── POTCAR
│   │   ├── KPOINTS
│   │   ├── CONTCAR      # Required
│   │   ├── OSZICAR      # Required
│   │   ├── OUTCAR
│   │   ├── vasprun.xml
│   │   └── ...          # All other VASP files
│   ├── 1/
│   │   ├── CONTCAR
│   │   ├── OSZICAR
│   │   └── ...
│   ├── ...
│   └── 10/                # Volume point 10 (largest)
│       ├── CONTCAR
│       ├── OSZICAR
│       └── ...
├── Material_2/
│   ├── 0/
│   │   ├── CONTCAR
│   │   ├── OSZICAR
│   │   └── ...
│   ├── 1/
│   │   ├── CONTCAR
│   │   ├── OSZICAR
│   │   └── ...
│   └── ...
└── Material_3/
    ├── 0/
    │   ├── CONTCAR
    │   ├── OSZICAR
    │   └── ...
    └── ...

Each material folder contains subdirectories (0, 1, 2, ..., 10) representing different volume points for EOS fitting.

from catbench.eos import eos_vasp_preprocessing

# Process EOS data (use your actual folder name here)
eos_vasp_preprocessing("your_eos_data")  # Deletes extra VASP files
# Output: Creates raw_data/{your_eos_data}_eos.json

Calculation

from catbench.eos import EOSCalculation
from your_mlip import YourCalculator

calc = YourCalculator(...)  # Your MLIP with desired settings

eos_calc = EOSCalculation(
    calculator=calc,
    mlip_name="YourMLIP",
    benchmark="eos_benchmark"
)
eos_calc.run()

Analysis

from catbench.eos import EOSAnalysis

# Configure and run analysis
config = {
    #"mlip_list": ["MLIP_A", "MLIP_B", ...],  # If not set, auto-detects all MLIPs in result folder
    #"font_setting": ["~/fonts/your_font_file.ttf", "sans-serif"],  # Custom font path
    # ... add any other options from Configuration Options section
}

eos_analysis = EOSAnalysis(**config)
eos_analysis.analysis()

Output Files

Comprehensive analysis results including individual material EOS curves and multi-MLIP comparison reports with Birch-Murnaghan equation fitting parameters.

EOS Analysis Examples

EOS curve comparison showing MLIP vs DFT results fitted with Birch-Murnaghan equation

The Excel report includes comprehensive EOS analysis with Birch-Murnaghan equation fitting:

MLIP	RMSE (eV)	MAE (eV)	VASP B0 (GPa)	MLIP B0 (GPa)	B0 Error (GPa)	VASP V0 (ų)	MLIP V0 (ų)	V0 Error (ų)
MLIP_A	0.634	0.462	80.53	102.59	22.06	475.37	469.42	5.95
MLIP_B	0.411	0.318	80.53	72.29	8.24	475.37	478.51	3.13
MLIP_C	0.444	0.350	80.53	88.02	7.49	475.37	470.70	4.67
MLIP_F	0.343	0.229	80.53	89.10	8.57	475.37	474.96	0.42
MLIP_G	0.762	0.447	80.53	97.94	17.41	475.37	472.02	3.36
...	...	...	...	...	...	...	...	...

Analysis includes bulk modulus (B0), equilibrium volume (V0), and derivative (B0') from Birch-Murnaghan EOS fitting

Configuration Options

AdsorptionCalculation

Parameter	Description	Type	Default
mlip_name	Name identifier for the MLIP	str	Required
benchmark	Dataset name or "multiple_tag" for combined	str	Required
mode	Calculation mode: "basic" or "oc20"	str	"basic"
f_crit_relax	Force convergence criterion (eV/Å)	float	0.05
n_crit_relax	Maximum optimization steps	int	999
rate	Fraction of atoms to fix (0: no atoms fixed, None: preserve original constraints)	float	0.5
damping	Optimization damping factor	float	1.0
optimizer	ASE optimizer: "LBFGS", "LBFGSLineSearch", "BFGS", "BFGSLineSearch", "GPMin", "MDMin", "FIRE"	str	"LBFGS"
save_step	Save interval for updating result.json file	int	50
save_files	Save trajectory, log, and gas files (False: only result.json)	bool	True
chemical_bond_cutoff	Cutoff distance for bond change calculation (Å)	float	6.0

AdsorptionAnalysis

Parameter	Description	Type	Default
calculating_path	Path to results directory	str	"./result"
mlip_list	MLIPs to analyze	list[str]	Auto-detect
target_adsorbates	Specific adsorbates to analyze	list[str]	All
exclude_adsorbates	Adsorbates to exclude	list[str]	None
benchmarking_name	Output file prefix	str	Current dir
disp_thrs	Displacement threshold (Å)	float	0.5
energy_thrs	Energy anomaly threshold (eV)	float	2.0
reproduction_thrs	Reproducibility threshold (eV)	float	0.2
bond_length_change_threshold	Bond length change threshold for anomaly detection (fraction)	float	0.2
energy_cutoff	Max reference energy to include (eV)	float	None
Plot Customization
figsize	Figure size (width, height) in inches	tuple	(9, 8)
dpi	Plot resolution (dots per inch)	int	300
time_unit	Time display unit: "s", "ms", "µs"	str	"ms"
plot_enabled	Generate plots	bool	True
mlip_name_map	Dictionary for MLIP display names	dict[str, str]	{}
Plot Appearance
mark_size	Marker size in plots	int	100
linewidths	Line width in plots	float	1.5
Plot Axes
min	Minimum value for plot axes	float	None
max	Maximum value for plot axes	float	None
tick_bins	Number of tick bins for both axes (None = auto)	int	6
tick_decimal_places	Decimal places for tick labels (None = auto)	int	1
tick_labelsize	Font size for tick labels	int	25
Font Sizes
xlabel_fontsize	Font size for x-axis labels	int	40
ylabel_fontsize	Font size for y-axis labels	int	40
mae_text_fontsize	Font size for MAE text	int	30
legend_fontsize	Legend font size	int	25
comparison_legend_fontsize	Comparison plot legend font size	int	15
threshold_xlabel_fontsize	X-axis label font size for threshold plots	int	40
threshold_ylabel_fontsize	Y-axis label font size for threshold plots	int	40
Display Options
legend_off	Hide legends in plots	bool	False
mae_text_off	Hide MAE text in plots	bool	False
error_bar_display	Show error bars in plots	bool	False
xlabel_off	Hide x-axis labels	bool	False
ylabel_off	Hide y-axis labels	bool	False
grid	Show grid on plots	bool	False
specific_color	Color for single MLIP plots	str	"#2077B5"
Advanced
font_setting	Custom font settings [family, path]	list[str]	False

DispersionCorrection

Parameter	Description	Type	Default
damping_type	Damping function: "damp_bj", "damp_zero"	str	"damp_bj"
functional_name	DFT functional for parameters	str	"pbe"
vdw_cutoff	van der Waals cutoff (au²)	int	9000
cn_cutoff	Coordination number cutoff (au²)	int	1600

SurfaceEnergyCalculation / BulkFormationCalculation

Parameter	Description	Type	Default
calculator	ASE calculator instance	ASE Calculator	Required
mlip_name	MLIP identifier	Required
benchmark	Dataset name	Required
f_crit_relax	Force convergence (eV/Å)	0.05
n_crit_relax	Max steps	999

RelativeEnergyAnalysis

Parameter	Description	Type	Default
calculating_path	Path to results directory	str	"./result"
plot_path	Path for plot output	str	"./plot"
benchmark	Dataset name	str	Current dir name
task_type	Analysis type: "surface", "bulk_formation", "custom"	str	Required
mlip_list	MLIPs to analyze	list[str]	Auto-detect
figsize	Plot dimensions	tuple[int, int]	(9, 8)
dpi	Plot resolution	int	300
mark_size	Marker size in plots	int	100
linewidths	Line width in plots	float	1.5
specific_color	Color for plots	str	"#2077B5"
min	Minimum value for plot axes	float	None
max	Maximum value for plot axes	float	None
grid	Show grid on plots	bool	False
font_setting	Custom font settings	list[str]	False

EOSCalculation

Parameter	Description	Type	Default
calculator	ASE calculator instance	ASE Calculator	Required
mlip_name	MLIP identifier	Required
benchmark	Dataset name	Required

EOSAnalysis

Parameter	Description	Type	Default
calculating_path	Path to results directory	str	"./result"
plot_path	Path for plot output	str	"./plot"
benchmark	Dataset name	str	Current dir name
mlip_list	MLIPs to analyze	list[str]	Auto-detect
figsize	Plot dimensions	tuple[int, int]	(9, 8)
dpi	Plot resolution	int	300
mark_size	Marker size in plots	int	100
x_tick_bins	Number of x-axis tick bins	int	5
y_tick_bins	Number of y-axis tick bins	int	5
tick_decimal_places	Decimal places for tick labels	int	1
tick_labelsize	Font size for tick labels	int	25
xlabel_fontsize	Font size for x-axis labels	int	40
ylabel_fontsize	Font size for y-axis labels	int	40
legend_fontsize	Legend font size	int	25
comparison_legend_fontsize	Comparison plot legend font size	int	15
grid	Show grid on plots	bool	False
font_setting	Custom font settings	list[str]	False

Citation

If you use CatBench in your research, please cite:

@article{catbench2025,
  title={CatBench Framework for Benchmarking Machine Learning Interatomic Potentials in Adsorption Energy Predictions for Heterogeneous Catalysis},
  author={Moon, Jinuk and Jeon, Uchan and Choung, Seokhyun and Han, Jeong Woo},
  journal={Cell Reports Physical Science},
  year={2025}
}

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contact

Jinuk Moon - [e-mail]
Jeong Woo Han - [e-mail]
Seoul National University

---

For bug reports, feature requests, and contributions, visit our GitHub repository.