Rapidly search your text for large list of cities and countries
If you have thousands of passages of text and you want to search them for hundreds or thousands of search strings, the process can become very cumbersome very soon. This is where ahocorasick search becomes very relevant and blazingly fast. You can read more about the algorithm here.
This code combines the very efficient implementation of ahocorasick in python with a pre-bundled list of cities, states and countries to tag locations in text.
- Python 3.6 or higher
- pyahocorasick 1.4.0 or higher
Using PIP via PyPi
pip install ahogeotagger
Right now the usage is pretty simple. You import and init the tagger with the number of cities you want to search your text for. The cities are in order of population (Tokyo, New york, Mexico City etc.).
The data for these cities has been prepopulated from the free version of simplemaps database which you can find here.
from ahogeotagger import tagger tagger.init(num_cities = 10000)
Optionally, if you dont want to use the built-in database of cities, you can provide your own list of cities. The list needs to be a list of tuples with each tuple's values in the following order:
The types for
lng are floats and all the rest are strings.
tagger.init(num_cities = 500, cities = [a,b,c])
where a,b,c are tuples described above.
To search whether text contains locations, supply any plain text to the search function like this:
results = tagger.search('New york and London are are competing for tech talent') print(results)
This produces the following list of tuples as a result
[(0, 7, 'New York', 'New York', 'United States', 'US', 'USA', 19354922, 40.6943, -73.9249), (13, 18, 'London', 'London, City of', 'United Kingdom', 'GB', 'GBR', 8567000, 51.5, -0.1167)]
Each tuple always contains values in this order
start_index (for the match in the source string),
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size ahogeotagger-0.1.5.tar.gz (449.1 kB)||File type Source||Python version None||Upload date||Hashes View|