tree-accuracy: score a phylogenetic tree against a reference taxonomy

tree-accuracy measures how well a phylogenetic tree agrees with a known taxonomy, using the F-measure metric from tax2tree (McDonald et al, 2012, see below). For each named taxon (e.g. a genus), it finds the best-matching clade in the tree and scores it by the harmonic mean of precision and recall of that clade's leaf set against the taxon's true leaf set. The overall score is the mean F-measure over all taxa, pooled across every level of the lineage.

A perfectly congruent tree — every named taxon forms an exact clade — scores 1.0. A tree whose topology bears no relation to the taxonomy scores much lower.

Installation

pip install tree-accuracy

Requires Python < 3.13: the ete3 dependency imports the standard-library cgi module, which was removed in 3.13, so import ete3 (and therefore import tree_accuracy) fails outright on 3.13+ until upstream fixes this.

Command-line usage

tree_fscore taxmap.tsv treefile

• taxmap.tsv: a two-column, whitespace-separated file. The first column is an accession (a leaf name in treefile); the rest of the line is a lineage string of taxonomy labels separated by colons and/or semicolons, coarsest first, e.g.:

  AB012345.1.1200	Eukaryota;Amorphea;Obazoa;Opisthokonta;Nucletmycea;Fungi;Dikarya;Ascomycota;Saccharomycotina;Saccharomycetes;Saccharomycetales;Debaryomycetaceae;Scheffersomyces;Scheffersomyces stipitis
  

• treefile: a Newick tree whose leaf names match the accessions in taxmap.tsv.

The command prints a single number — the pooled mean F-measure — to stdout.

See examples/ for two small worked datasets (a perfectly congruent tree and a scrambled one) and examples/README.md for details.

Python API

import ete3
from tree_accuracy import fscore
from tree_accuracy.taxonomy import load_taxmap

lineages = load_taxmap("taxmap.tsv")   # accession -> [label_at_depth_0, ...]
tree = ete3.Tree("treefile.nwk")
scores = fscore(tree, lineages)
print(scores["pooled"])       # overall accuracy
print(scores["depth0"])       # mean F-measure at the coarsest lineage level

fscore returns a dict mapping each lineage depth to the mean F-measure of taxa found at that depth, plus 'pooled' for the mean over all taxa at all depths combined. Taxon labels shared by fewer than min_count leaves (default 2) are excluded from scoring, since a label unique to a single leaf trivially scores F=1 there and would otherwise inflate the mean.

See examples/python_example.py for a complete, runnable script that scores both example datasets this way and prints the full per-depth breakdown, not just the pooled number the CLI prints.

Reference

The F-measure scoring method is from tax2tree:

McDonald D, Price MN, Goodrich J, Nawrocki EP, DeSantis TZ, Probst A, Andersen GL, Knight R, Hugenholtz P. "An improved Greengenes taxonomy with explicit ranks for ecological and evolutionary analyses of bacteria and archaea." The ISME Journal 6, 610–618 (2012). https://doi.org/10.1038/ismej.2011.139

This package reimplements the metric directly against ete3 rather than depending on tax2tree itself (which is built around skbio TreeNode objects and its own rank-prefixed consensus string format), so it works with any colon/semicolon-delimited lineage string (SILVA, PFAM clan/family hierarchies, ...) without first converting it into that format.