foldmetrics: unified confidence metrics & interface contacts for structure-prediction models

Testing
Package
Meta

🧬 What is it?

foldmetrics ingests the raw output folders of the mainstream structure predictors — AlphaFold2 / AlphaFold-Multimer, ColabFold, AlphaFold3, Boltz, Chai-1, Protenix — and computes a consistent set of quality metrics for monomers and complexes (protein, nucleic acid and small-molecule ligands), for a single model or whole batches:

Beyond scores it extracts confident interface contacts (with ready-to-open PyMOL/ChimeraX sessions) and renders publication-oriented figures.

📦 Installation

pip install foldmetrics

With DockQ support (reference-based scoring):

pip install "foldmetrics[dockq]"

Development install:

git clone https://github.com/ChiaChunL/foldmetrics.git 
cd foldmetrics
pip install -e ".[dev]"

⚡ Quickstart

CLI (foldmetrics, short alias fmx):

Every command below runs as-is from a clone, against the bundled real examples — swap examples/data for your own prediction folders or files:

# score everything (tools are auto-detected and can be mixed)
fmx score examples/data -o metrics.tsv --interfaces interfaces.tsv --plot plots/

# one metric only — every metric name is also a subcommand
fmx ipsae examples/data
fmx score examples/data --metrics ipsae,pdockq2,lis

# campaign view: aggregate seeds/samples per target and tool
fmx score preds/ --by-target summary_by_target.tsv

# confident interface contacts: table + figure + PyMOL/ChimeraX sessions
fmx contacts examples/data/af3_server -o contacts.tsv --plot plots/

# DockQ: the AlphaFold2 model against the AlphaFold-Server model as reference
fmx dockq examples/data/af2_multimer --ref examples/data/af3_server/fold_barnase_barstar_s318_model_0.cif

# what would be scored?
fmx detect examples/data

Python:

import foldmetrics as fmx

df = fmx.evaluate("examples/data")  # one row per model
print(df[["model", "tool", "iptm", "ipsae", "pdockq2"]].round(3))
#                                model       tool   iptm  ipsae  pdockq2
#           model_1_multimer_v3_pred_0 alphafold2  0.937  0.897    0.952
#                    mpro_nirmatrelvir alphafold3  0.970  0.836      NaN
#    fold_barnase_barstar_s318_model_0 alphafold3  0.930  0.890    0.944
#              barnase_barstar_model_0      boltz  0.959  0.935    0.939

dfi = fmx.evaluate_interfaces("examples/data")  # one row per chain pair
agg = fmx.aggregate_by_target(df)  # one row per target+tool over all models
pred = fmx.load_predictions("examples/data")[0]
contacts = fmx.find_contacts(pred, dist_cutoff=8.0, pae_cutoff=12.0)

More recipes live in examples/ — real example predictions are included there, so every command runs as-is straight after cloning.

🛠️ Command-line reference

paths accepts anything: a directory (scanned recursively; different tools can be mixed freely), one specific model file, or several paths at once.

fmx <command> --help prints the complete option list for any command. Shell tab-completion is available via pip install "foldmetrics[completion]" followed by activate-global-python-argcomplete.

Screening many seeds and samples

Large campaigns produce many models per complex. --by-target collapses them into one row per target and tool — model count, mean/std/max ipTM and ipSAE, best pDockQ2, and the name of the best model (by ranking_score, else ipSAE, else ipTM):

         target       tool  n_models                         best_model  iptm_mean  iptm_std  ipsae_mean  ipsae_max
barnase_barstar alphafold3        16 barnase_barstar_seed-1030_sample-2      0.930     0.000       0.888      0.893

The standard deviations show how stable the prediction is across seeds — a low mean with high spread is a very different situation from a consistently low one. The same table is available in Python as fmx.aggregate_by_target(df).

🖼️ Visualization

--plot DIR (on score and every metric subcommand) or the plot subcommand renders figures that adapt automatically to the shape of the batch:

The structure panel renders via headless PyMOL when installed (--renderer pymol|trace overrides; FOLDMETRICS_PYMOL sets the path). plot also takes --format png|pdf|svg and --dpi.

Single model (AlphaFold3, SARS-CoV-2 Mpro + nirmatrelvir):

Targets × methods comparison (real batch: 10 complexes × 4 tools):

Batch overview (one target, AlphaFold2 + AlphaFold3 models):

🤝 Confident interface contacts

A contact is an inter-chain residue (or ligand-atom) pair within --dist-cutoff 8 Å whose PAE is below --pae-cutoff 12 Å in both directions (negative disables the PAE filter).

fmx contacts examples/data/af3_server -o contacts.tsv --plot plots/

-o writes the contact table; --plot DIR adds, per model:

• *_contacts.png — the figure below
• *_contacts.pse / *_contacts.cxs — PyMOL / ChimeraX sessions: double-click to open the styled interface scene, with if_A / if_B / hotspots / interface selections ready (--no-sessions skips)
• *_contacts.pml / *_contacts.cxc — the same scene as plain scripts, always written

🎯 DockQ against a reference

When an experimental (or otherwise trusted) structure exists, fmx dockq computes the actual interface accuracy via the official DockQ implementation (install with pip install "foldmetrics[dockq]"):

fmx dockq preds/ --ref 1brs.pdb -o dockq.tsv
fmx dockq preds/ --ref native.cif --mapping A:A,B:D   # explicit chain pairing
fmx dockq preds/ --ref homodimer.cif --best-mapping   # search all assignments
fmx dockq preds/ --ref complex.cif --small-molecule   # score ligand poses too

Reports DockQ, fnat, iRMSD, LRMSD, the CAPRI-style class and the chain mapping used, per interface. Chains are matched by name when both structures share names, otherwise by order — mmCIF label vs auth chain ids differ between tools, so check the mapping column. Override explicitly with --mapping MODEL:REF,..., or let --best-mapping try every assignment and keep the best total DockQ (recommended for homo-multimers; refused above 5 chains). Additional switches: --no-align (skip sequence alignment when residue numbering already matches), --low-memory (huge complexes), --capri-peptide (protein–peptide criteria).

📊 How to read the scores

pDockQ2 has two established readings

Zhu 2023 fixes pDockQ2's fitted sigmoid and its PAE term, but not which atom defines a contact, which atom's pLDDT to read, or whether the partner chain contributes to the pLDDT average. Implementations diverge:

The gap is ~0.005 on real AlphaFold3 output, because CA and CB contacts describe different interfaces (33 vs 53 pairs on barnase–barstar). It only matters for engines with per-atom pLDDT: pDockQ2 was fitted on AlphaFold2, where pLDDT is constant within a residue and all readings coincide. Pass variant= to foldmetrics.metrics.pdockq2_asym to choose; pdockq2_ab/pdockq2_ba are the per-chain values the paper defines, and the aggregate pdockq2 is their maximum, which is our convention — the paper defines no interface-level aggregate.

Single scores can mislead — pDockQ ignores PAE and can stay deceptively high on confidently-folded but wrongly-docked chains, which pDockQ2/ipSAE expose. Read them together (that is rather the point of this package).

🧰 Supported tools and files

Native per-tool extras (e.g. Boltz complex_iplddt/ligand_iptm, AF3 chain_pair_pae_min, Chai clash flags) are preserved on Prediction.extras and chain-pair ipTM is surfaced as iptm_native in the interface table.

✅ Validation

• Numerical parity with the ipsae.py reference implementation (Dunbrack Lab) verified digit-for-digit on real AlphaFold3 server output: ipSAE (both directions and d0chn variant), pDockQ, pDockQ2 and LIS all match to 6 decimal places at the default cutoffs (10/10).
• Batch-tested on 720+ real predictions across AlphaFold2-Multimer, AlphaFold3 (server + local), Boltz-2, Chai-1, ColabFold and Protenix — including protein–small-molecule complexes, homodimers, monomers and negative controls — with zero parse errors; known binders score ipSAE 0.9+, decoy pairs < 0.1, monomers report NA.
• Cross-implementation agreement: on real ColabFold 1.6 output our ipSAE/pDockQ/pDockQ2 reproduce ColabFold's own embedded values to ~1e-5 (bounded only by the 2-decimal PAE rounding in its JSON) — a CI regression test enforces this parity on every commit.

📋 What each metric needs

The structure file is always required (it defines chains and tokens); the table shows which additional inputs each metric consumes. When an input is missing the metric is NA and a note lands in the warnings column — nothing crashes.

📁 Outputs and paths

• Summary table → stdout; -o FILE writes it. The extension picks the format: .tsv (default), .csv, .json; missing values are NA.
• --interfaces FILE → the per chain-pair table (same formats).
• --plot DIR → figures as described under Visualization; contacts --plot adds <model>_contacts.png + <model>_contacts.pml. Model names are sanitized ([^\w.-] → _) for use as filenames.
• plot -o DIR defaults to ./foldmetrics_plots/.
• Exit codes: 0 success, 1 nothing recognized/found, 2 bad arguments or missing optional dependency.

💡 Conventions worth knowing

• Tokens. Standard residues are one token; ligands and modified residues are one token per heavy atom (AF3-style), so token-level PAE matrices line up across tools. pLDDT is stored on the 0–100 scale everywhere (Boltz 0–1 values are rescaled).
• Complex-level interface metrics are the best interface. For >2 chains, ipsae/pdockq/pdockq2/lis in the summary table are the maximum over chain pairs; use --interfaces for the full breakdown.
• Ligand interfaces. pdockq/pdockq2/iplddt are defined for polymer–polymer interfaces only. For chain pairs involving a ligand chain, ipsae/lis are computed over ligand atom tokens (experimental) and marked ipsae_mode = "tokens" in the interface table.
• Missing data degrades gracefully. No PAE → PAE-based metrics are NaN and a note lands in the warnings column; nothing crashes.
• Directionality. PAE is asymmetric, so pdockq2/ipsae/lis have two directional values; the interface table reports both (*_ab, *_ba) plus the aggregate used everywhere else (max for ipSAE/pDockQ2, mean for LIS, matching the reference implementations).

📚 References

• Jumper J et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021). — pLDDT / PAE and their confidence bands
• Bryant P, Pozzati G, Elofsson A. Improved prediction of protein-protein interactions using AlphaFold2. Nat Commun 13, 1265 (2022). — pDockQ
• Zhu W, Shenoy A, Kundrotas P, Elofsson A. Evaluation of AlphaFold-Multimer prediction on multi-chain protein complexes. Bioinformatics 39, btad424 (2023). — pDockQ2
• Dunbrack RL. ipSAE: scoring pairwise interactions in AlphaFold models. bioRxiv (2025). — ipSAE
• Kim AR et al. Enhanced protein-protein interaction discovery via AlphaFold-Multimer. bioRxiv (2024). — LIS
• Basu S, Wallner B. DockQ: A quality measure for protein-protein docking models. PLoS ONE 11, e0161879 (2016); Mirabello C, Wallner B. DockQ v2. Bioinformatics (2024). — DockQ

📄 License

BSD 3-Clause