idiomatch

An implementation of SpaCy(3.0)'s Matcher specifically designed for identifying English idioms.

Quick Start

Install the library via uv (or whatever package manager you prefer)

uv add idiomatch 

from idiomatch import Idiomatcher

def main():
    sent = "The floodgates will remain opened for a host of new lawsuits."  # a usecase of *open the floodgates*
    idiomatcher = Idiomatcher.from_pretrained()  # this will take approx 50 seconds.
    doc = idiomatcher.nlp(sent)  # use and only use the nlp model that comes with the matcher 
    print(idiomatcher(doc))  # identify the idiom in the sentence



if __name__ == '__main__':
    main()

adding patterns into idiom_matcher...: 100%|██████████| 2756/2756 [00:52<00:00, 52.83it/s]
[{'idiom': 'open the floodgates', 'span': 'The floodgates will remain opened', 'meta': (13612509636477658373, 0, 5)}]

Supported Idioms

List of supported idioms can be found in idiomatch/resources/idioms.txt. Total of 2758 idioms are available for matching. These "target idioms" were extracted from a vocabulary of 5000 most frequently used English idioms, which had been made available for open use courtesy of IBM's SLIDE project.

Adding Idioms Yourself

If you have idioms that are not included in the list of supported idioms, you can add them to Idiomatcher yourself with the add_idioms member method:

from idiomatch import Idiomatcher

def main():
    idiomatcher = Idiomatcher.from_pretrained()  # instantiate 
    # As for a placeholder for openslot, use either: someone / something / someone's / one's 
    idioms = ["have blood on one's hands", "on one's hands"]
    idiomatcher.add_idioms(idioms)  # this will train idiomatcher to identify the given idioms
    sent = "The leaders of this war have the blood of many thousands of people on their hands."
    doc = idiomatcher.nlp(sent)
    print(idiomatcher(doc))


if __name__ == '__main__':
    main()

100%|██████████| 2/2 [00:00<00:00, 145.62it/s]
adding patterns into idiom_matcher...: 100%|██████████| 2/2 [00:00<00:00, 196.40it/s]
[{'idiom': "have blood on one's hands", 'span': 'have the blood of many thousands of people on their hands', 'meta': (5930902300252675198, 5, 16)}]

Supported Variations

English idioms extensively vary in forms, at least in six different ways. Idiomatcher can gracefully handle all the cases, as exemplified below:

How Does it Work?

The idiom-matching patterns, which are the foundations of Idiomatcher's flexibility, are heavily inspired by Hughs et al.'s briliant work (2021) on Flexible Retrieval of Idiomatic Expressions from a Large Text Corpus.