flashtext 2.0

Extract/Replaces keywords in sentences.

This module can be used to replace keywords in sentences or extract keywords from sentences.

Installation

$ pip install flashtext

Usage

Extract keywords

>>> from flashtext.keyword import KeywordProcessor
>>> keyword_processor = KeywordProcessor()
>>> keyword_processor.add_keyword('Big Apple', 'New York')
>>> keyword_processor.add_keyword('Bay Area')
>>> keywords_found = keyword_processor.extract_keywords('I love Big Apple and Bay Area.')
>>> keywords_found
>>> ['New York', 'Bay Area']

Replace keywords

>>> keyword_processor.add_keyword('New Delhi', 'NCR region')
>>> new_sentence = keyword_processor.replace_keywords('I love Big Apple and new delhi.')
>>> new_sentence
>>> 'I love New York and NCR region.'

Case Sensitive example

>>> from flashtext.keyword import KeywordProcessor
>>> keyword_processor = KeywordProcessor(case_sensitive=True)
>>> keyword_processor.add_keyword('Big Apple', 'New York')
>>> keyword_processor.add_keyword('Bay Area')
>>> keywords_found = keyword_processor.extract_keywords('I love big Apple and Bay Area.')
>>> keywords_found
>>> ['Bay Area']

No clean name for Keywords

>>> from flashtext.keyword import KeywordProcessor
>>> keyword_processor = KeywordProcessor()
>>> keyword_processor.add_keyword('Big Apple')
>>> keyword_processor.add_keyword('Bay Area')
>>> keywords_found = keyword_processor.extract_keywords('I love big Apple and Bay Area.')
>>> keywords_found
>>> ['Big Apple', 'Bay Area']

API doc

Documentation can be found at FlashText Read the Docs.

Test

$ git clone https://github.com/vi3k6i5/flashtext
$ cd flashtext
$ pip install pytest
$ python setup.py test

Why not Regex?

It’s a custom algorithm based on Aho-Corasick algorithm and Trie Dictionary.

To do the same with regex it will take a lot of time:

Docs count	# Keywords	:	Regex	flashtext
1.5 million	2K	:	16 hours	Not measured
2.5 million	10K	:	15 days	15 mins

The idea for this library came from the following StackOverflow question.

Contribute

• Issue Tracker: https://github.com/vi3k6i5/flashtext/issues

• Source Code: https://github.com/vi3k6i5/flashtext/

License

The project is licensed under the MIT license.