Search for the most relevant documents containing words from a query
Search for the most relevant documents containing words from the query.
query = ['A', 'B'] documents = [ ['N', 'A', 'M'], # matching features: 'A' ['C', 'B', 'A'], # matching features: 'A', 'B' ['X', 'Y'] # no matching features ]
The search with return
['C', 'B', 'A'] and
['N', 'A', 'M'] in that
It's not necessarily about text. Words are just any
str instances. Documents
are unordered collections of these
str. We search for documents considering
frequency, rarity and match accuracy.
pip3 install git+https://github.com/rtmigo/skifts_py#egg=skifts
Use for full-text search
Finding documents that contain words from the query.
from skifts import SkiFts # three documents, one per row documents = [ ["wait", "mister", "postman"], ["please", "mister", "postman", "look", "and", "see"], ["oh", "yes", "wait", "a", "minute", "mister", "postman"] ] fts = SkiFts(documents) # find and print the most relevant documents: for doc_index in fts.search(['postman', 'wait']): print(documents[doc_index])
Words inside the
documents list are considered ready-made feature identifiers.
If your text needs preprocessing or stemming, this should be done separately.
The ranking takes into account the frequency of words in the document and the rarity of words in the corpus. The word order in the document and the distance between words do not matter.
The search uses the scikit-learn library, which ranks documents using tf-idf and cosine similarity.
The gifts package implements the same search, but in pure Python with no binary dependencies.
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.