docling-metrics-chemistry

Chemistry evaluation metrics (SMILES and Markush structures) for the Docling metrics framework.

Installation

pip install docling-metrics-chemistry

Package Structure

docling_metrics_chemistry/
├── __init__.py            # Public API exports
├── smiles_metric.py       # BaseMetric implementation (SmilesMetric)
├── molecule_scores.py     # Score computation (Tanimoto, InChI, Markush)
├── smiles_utils.py        # Molecule parsing, canonicalization, wildcard replacement
└── cxsmiles_parser.py     # CXSMILES section parsing (M-sections, Sg-sections)

Key Classes

SmilesMetric

The main metric class implementing the BaseMetric interface. Provides three methods:

• evaluate_sample(sample) — evaluate a single predicted/ground-truth SMILES pair
• aggregate(results) — compute summary statistics from multiple sample results
• evaluate_dataset(samples) — evaluate an entire dataset (calls both of the above)

SmilesInputSample

Input model for a single evaluation sample.

SmilesSampleResult

Per-sample result with all computed scores.

SmilesAggregateResult

Aggregated statistics across a dataset.

Metrics

• Tanimoto similarity: RDKit fingerprint-based molecular similarity (0–1)
• InChI equality: International Chemical Identifier comparison
• String equality: Exact canonical SMILES string match
• Validity: Whether a SMILES string parses into a valid molecule
• Markush evaluation: R-group, M-section, Sg-section accuracy for CXSMILES

Usage

Simple molecule evaluation

from docling_metrics_chemistry import SmilesMetric, SmilesInputSample

metric = SmilesMetric()

# Evaluate a single pair
sample = SmilesInputSample(
    id="sample_1",
    predicted_smiles="CCO",
    gt_smiles="CCO",
)
result = metric.evaluate_sample(sample)
print(result.tanimoto)        # 1.0
print(result.inchi_equality)  # True
print(result.valid)           # True

Dataset evaluation

samples = [
    SmilesInputSample(id="1", predicted_smiles="CCO", gt_smiles="CCO"),
    SmilesInputSample(id="2", predicted_smiles="c1ccccc1", gt_smiles="C1=CC=CC=C1"),
    SmilesInputSample(id="3", predicted_smiles="INVALID", gt_smiles="CCO"),
]

aggregate = metric.evaluate_dataset(samples)
print(aggregate.mean_tanimoto)        # Mean Tanimoto across all samples
print(aggregate.validity_rate)        # Fraction of valid predictions
print(aggregate.inchi_equality_rate)  # Fraction with matching InChI

Markush structure evaluation

# Evaluate CXSMILES with R-groups, M-sections, and Sg-sections
sample = SmilesInputSample(
    id="markush_1",
    predicted_smiles="*C(=O)O.*Cl |m:4:10.11.12.9|",
    gt_smiles="*C(=O)O.*Cl |m:4:10.11.12.9|",
    is_markush=True,
)
result = metric.evaluate_sample(sample)
print(result.r)               # R-group label accuracy (0-1 or None)
print(result.m)               # M-section accuracy (0-1 or None)
print(result.sg)              # Sg-section accuracy (0-1 or None)
print(result.cxsmi_equality)  # Overall CXSMILES match