Archived

This project has been archived by its maintainers, and is no longer receiving any updates.

trie-search

Trie-search is a package for text pattern search using marisa-trie.

Installation

$ pip install trie-search

Usage

Create trie dictionary

Before using this package, you need to create trie dictionary, or prepare a list of patterns.

The following example simply creates trie dictionary of marisa_trie.Trie from list of article titles in English version of Wikipedia, and saves it to ./example/triedict.

$ cd ./example
$ curl https://dumps.wikimedia.org/enwiki/latest/enwiki-latest-all-titles-in-ns0.gz | gzcat | python create_triedict.py

NOTICE : This script will consume more than 2GB memory.

trie_search.TrieSearch

Create an instance, and load dictionary:

>>> import trie_search
>>> trie = trie_search.TrieSearch(filepath='./example/triedict')

If you have list or tuple object of patterns, you can create an instance as follows:

>>> patterns = ['pattern1', 'pattern2', 'pattern3']
>>> trie = trie_search.TrieSearch(patterns)

TrieSearch.search_all_patterns

Search all patterns in an input text:

>>> text = ('in computer science , a trie , also called digital tree and '
...         'sometimes radix tree or prefix tree ( as they can be searched '
...         'by prefixes ) , is a kind of search tree - an ordered tree data '
...         'structure that is used to store a dynamic set or associative array '
...         'where the keys are usually strings .')
>>> for pattern, start_idx in trie.search_all_patterns(text):
...     print pattern, start_idx
...
in 0
computer 3
computer science 3
science 12
, 20
a 22
trie 24
, 29
also 31
called 36
digital 43
... skipped ...
array 246
where 252
where the 252
the 258
the keys 258
keys 262
are 267
usually 271
strings 279

The text is the 1st sentence of https://en.wikipedia.org/wiki/Trie. For normalization, remove capitalizations and add single white space before/after symbols.
search_all_patterns returns an iterator. Each searched pattern is represented as a tuple (pattern_string, pattern_start_index). The results are sorted by the start index. If you want to get the result as a list object, use list function as follow:
```
>>> patterns = list(trie.search_all_patterns(text))
```

TrieSearch.search_longest_patterns

Search longest patterns in an input text:

>>> for pattern, start_idx in sorted(trie.search_longest_patterns(text), key=lambda x: x[1]):
...     print pattern, start_idx
...
in 0
computer science 3
, 20
a 22
trie 24
, 29
also 31
called 36
digital tree 43
and 56
sometimes 60
radix tree 70
or 81
prefix tree 84
( 96
as 98
they 101
can 106
be 110
by 122
prefixes 125
) 134
, 136
is a 138
kind 143
of 148
search tree 151
- 163
an 165
ordered tree data structure 168
that 196
is 201
used to 204
store 212
a 218
dynamic set 220
or 232
associative array 235
where the 253
the keys 259
are 268
usually 272
strings 280

search_all_patterns also returns an iterator. The result sorted by the length of patterns. In the above example, the result is re-sorted by the start index.

trie_search.RecordTrieSearch

trie_search.RecordTrieSearch is a sub class of marisa_trie.RecordTrie, which maps unicode keys to data tuples.

The functions, search_all_patterns and search_longest_patterns, are also implemented in trie_search.RecordTrieSearch.

Metadata

Release files for trie-search 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for trie-search 0.3.0
File	Size	Uploaded	Provenance	Actions
trie-search-0.3.0.tar.gz	4.7 kB	Jan 16, 2018	Without provenance	Details for trie-search-0.3.0.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for trie-search 0.3.0
File	Interpreter	ABI	Platform	Actions Reset
trie_search-0.3.0-py3.5.egg 6.9 kB Jan 16, 2018 Without provenance	Legacy Egg format	-	-	Details for trie_search-0.3.0-py3.5.egg
trie_search-0.3.0-py3-none-any.whl 6.6 kB Jan 16, 2018 Without provenance	Python 3	none	any	Details for trie_search-0.3.0-py3-none-any.whl
trie_search-0.3.0-py2.7.egg 6.7 kB Jan 16, 2018 Without provenance	Legacy Egg format	-	-	Details for trie_search-0.3.0-py2.7.egg
trie_search-0.3.0-py2-none-any.whl 6.6 kB Jan 16, 2018 Without provenance	Python 2	none	any	Details for trie_search-0.3.0-py2-none-any.whl

Total release size: 31.4 kB

Release files / trie-search-0.3.0.tar.gz

Download URL	trie-search-0.3.0.tar.gz
Size	4.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`a507c4696951fc7745f91a09c92367262c92d648ed05ee071ab8bc579d7deb10`
BLAKE2b-256 checksum How to use checksums	`5ee2f14c28bda657b9ef623d2ab4bc905a6c4adadffeda945708c9fb719d601c`
Upload date	Jan 16, 2018
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / trie_search-0.3.0-py3.5.egg

Download URL	trie_search-0.3.0-py3.5.egg
Size	6.9 kB
Tags	Egg
SHA-256 checksum How to use checksums	`eeeec95c0f320129382e71bc1da02e9945198608f1996b08249665ec09e512a6`
BLAKE2b-256 checksum How to use checksums	`3b81a874fedaf96290365132d515269465c2b049930b8bf965fb3438184df497`
Upload date	Jan 16, 2018
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / trie_search-0.3.0-py3-none-any.whl

Download URL	trie_search-0.3.0-py3-none-any.whl
Size	6.6 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`3e0d66ea7057e65a0aa0bf23da1bb50f03b41c94b4752e471884f1c82b33e885`
BLAKE2b-256 checksum How to use checksums	`934cd0b45b593b262ef188c187e27fb726b5ec34e06585abd685a57a09c5d576`
Upload date	Jan 16, 2018
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / trie_search-0.3.0-py2.7.egg

Download URL	trie_search-0.3.0-py2.7.egg
Size	6.7 kB
Tags	Egg
SHA-256 checksum How to use checksums	`37435dcb1bfd69776d58be3be34fe0e04d39cb9757b0f48f329112ebe64c9ef7`
BLAKE2b-256 checksum How to use checksums	`6ee69896cb19402dc2490a3e5505d5419bf89de99703bd16386c108944ab82c0`
Upload date	Jan 16, 2018
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / trie_search-0.3.0-py2-none-any.whl

Download URL	trie_search-0.3.0-py2-none-any.whl
Size	6.6 kB
Tags	Python 2
SHA-256 checksum How to use checksums	`e46f01ce6c2f2c610b822df3074a669a50b3967042dcfe7a91c91ed33b15e502`
BLAKE2b-256 checksum How to use checksums	`ea4274d14b4911e623eb82687a39e53d6a8ab05fdf730d18da18d908576cc30c`
Upload date	Jan 16, 2018
Uploaded using Trusted Publishing? What is trusted publishing?	No

Security for trie-search 0.3.0

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

Archived

0.3.0 This release

Jan 16, 2018 5 release files No provenance

Archived

trie-search 0.3.0