Skip to main content

Smatch (semantic match) tool

Project description

Smatch (semantic match) tool

Version on PyPI Python Support

Smatch is an evaluation tool for AMR (Abstract Meaning Representation). It computes the Smatch score (defined below) of two AMR graphs in terms of their matching triples (edges) by finding a variable (node) mapping that maximizes the count, M, of matching triples, then:

  • M is the number of matching triples
  • T is the total number of triples in the first AMR
  • G is the total number of triples in the second AMR
  • Precision is defined as P = M/T
  • Recall is defined as R = M/G
  • The Smatch score is the F-score: F = 2 * (P*R)/(P+R)

For more information, see Cai and Knight, 2013.

Requirements, Installation, and Usage

This Smatch implementation is tested for Python 3.5 or higher. It is released on PyPI so you can install it with pip:

$ pip install smatch

You can also clone this repository and run the script directly as it does not need to be installed to be used.

To use the script, run it with at least the -f option, which takes two filename arguments:

$ -f test.amr gold.amr

Note that the order of these arguments does not matter for the Smatch score as the F-score is symmetric, but swapping the arguments will swap the precision and recall. The files contain AMRs separated by a blank line, with comment lines starting with # (see test_input1.txt for an example).

For other options, try --help.


    title = "{S}match: an Evaluation Metric for Semantic Feature Structures",
    author = "Cai, Shu and Knight, Kevin",
    booktitle = "Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
    month = aug,
    year = "2013",
    address = "Sofia, Bulgaria",
    publisher = "Association for Computational Linguistics",
    url = "",
    pages = "748--752",

Recommendations for Reproducible Research

You can help make your research reproducible by including the following information in your writing:

  • The software version (e.g., repository URL and version number)
  • The number of restarts (-r) used, even if unchanged from the default
  • The order of the arguments to -f (if reporting precision and recall)
  • Any other options or preprocessing steps


The code was mostly developed during 2012 and 2013, and has undergone many fixes and updates. Note that the versions distributed for SemEval-2016 were numbered 2.0–2.0.2, but these predate this repository and the 1.0 series on PyPI. For more details, see the Changelog.

Related Projects

Here are some notable forks of Smatch:

And here are other evaluation metrics for AMR:

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for smatch, version 1.0.4
Filename, size File type Python version Upload date Hashes
Filename, size smatch-1.0.4.tar.gz (26.6 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page