moscow-toponyms 0.1.0

Moscow toponym extractor for Russian texts

Moscow Toponym Extractor

This module contains an extractor of Moscow toponyms from Russian texts using such Python libraries as SpaCy, Natasha, and PyMorphy2.

Returned attributes for extracted Moscow toponym:

• toponym - toponym in an inflected form (e.g., Кремле)
• lemmatized_toponym - toponym in the base form (e.g., Кремль)
• start_char - start character index (e.g., 79)
• stop_char - end character index (e.g., 85)

Installation

1. Install the package using pip:

pip install moscow-toponyms

2. Download ru_core_news_sm

pip install https://github.com/explosion/spacy-models/releases/download/ru_core_news_sm-3.1.0/ru_core_news_sm-3.1.0.tar.gz

Quick start

>>> from moscow_toponyms import QuickExtract
>>> text = "Однажды весною, в час небывало жаркого заката, в Москве, на Патриарших прудах, появились два гражданина."
>>> toponyms = QuickExtract(text)
>>> toponyms.extract()
[{'toponym': 'Патриарших прудах',
  'lemmatized_toponym': 'Патриаршие пруды',
  'start_char': 60,
  'stop_char': 77}]

Usage

>>> from moscow_toponyms import ExtractMosToponyms
>>> text = "Однажды весною, в час небывало жаркого заката, в Москве, на Патриарших прудах, появились два гражданина."
>>> extract_toponyms = ExtractMosToponyms(text)

Using SpaCy extract toponyms and their position in a text, lemmatize extracted toponyms using PyMorphy2:

>>> spacy_extracted = extract_toponyms.spacy_extract()
>>> print(spacy_extracted)
({51: 'смоленский площадь'}, {0: 'саша панкратов'})
>>> spacy_dict = spacy_extracted[0]
>>> spacy_names = spacy_extracted[1]

Using Natasha extract toponyms and their position in a text:

>>> natasha_extractor = extract_toponyms.natasha_extract()
>>> print(natasha_extractor)
({51: ['Смоленской площади', 'Смоленская площадь', 69]}, {0: 'Саша Панкратов'})
>>> natasha_dict = natasha_extractor[0]
>>> natasha_names = natasha_extractor[1]

Add the extracted names to the existing black list for cleaner output:

>>> black_list = extract_toponyms.merging_blacklists(spacy_names, natasha_names)

Filter all extracted toponyms and return only Moscow toponyms in inflected and base forms, their start and end character indices

>>> final_results = extract_toponyms.inner_merging_filtering(black_list, spacy_dict, natasha_dict)
>>> print(final_results)
[{'toponym': 'Смоленской площади', 'lemmatized_toponym': 'Смоленская площадь', 'start_char': 51, 'stop_char': 69}]