Data-structure for keyword searches
The KeywordTree is a data structure designed to speed up the process of searching for words within a large group of text. Given a large set of keywords, a data structure is constructed to help speed up the process of searching. Much like any other tree-based structure, a branch is constructed of the longest consecutive substring similarly shared between any of its leaves. When searching, this allows us to “prune” any branches that do not match any of the text we’re searching in, drastically speeding up the process. The purpose is to spend a 1-time cost constructing the tree, then utilizing its structure to boost searching performance.
This library has the ability to define individual keyword attributes to use or not use regular expressions, and to preserve capitalization or ignore it. Since the biggest cost is in build-time, you can also export and import a prebuilt tree. You can add or remove keywords from an existing tree, and limit how much memory the tree can consume.
Please see my project’s page for installation details, examples, and benchmarking results.