ner-kit 0.0.1

Named Entity Recognition Toolkit

Named Entity Recognition Toolkit

Provide a toolkit for rapidly extracting useful entities from text using various Python packages, including Stanza.

Features

We try to bring the complicated use of existing NLP toolkits down to earth by keeping APIs as simple as possible with best practice.

Installation

pip install ner-kit

Examples

Example 1: Word segmention

from nerkit.StanzaApi import StanzaWrapper
if __name__=="__main__":
    sw=StanzaWrapper()
    sw.download(lang="en")
    text='This is a test sentence for stanza. This is another sentence.'
    result1=sw.tokenize(text)
    sw.print_result(result1)

Example 2: Chinese word segmentation

from nerkit.StanzaApi import StanzaWrapper
if __name__=="__main__":
    sw=StanzaWrapper()
    sw.download(lang="zh")
    text='我在北京吃苹果！'
    result1=sw.tokenize(text,lang='zh')
    sw.print_result(result1)

Example 3: Multi-Word Token (MWT) Expansion

from nerkit.StanzaApi import StanzaWrapper
if __name__=="__main__":
    sw=StanzaWrapper()
    sw.download(lang="fr")
    text='Nous avons atteint la fin du sentier.'
    result1=sw.mwt_expand(text,lang='fr')
    sw.print_result(result1)

Example 4: POS tagging

from nerkit.StanzaApi import StanzaWrapper
if __name__=="__main__":
    sw=StanzaWrapper()
    sw.download(lang='en')
    text='I like apple'
    result1=sw.tag(text)
    sw.print_result(result1)
    sw.download_chinese_model()
    text='我喜欢苹果'
    result2=sw.tag_chinese(text,lang='zh')
    sw.print_result(result2)

Example 5: Named Entity Recognition

from nerkit.StanzaApi import StanzaWrapper

if __name__=="__main__":
    sw=StanzaWrapper()

    sw.download(lang='en')
    sw.download_chinese_model()

    text_en = 'I like Beijing!'
    result1 = sw.ner(text_en)
    sw.print_result(result1)

    text='我喜欢北京！'
    result2=sw.ner_chinese(text)
    sw.print_result(result2)

Example 6: Sentiment Analysis

from nerkit.StanzaApi import StanzaWrapper

if __name__=="__main__":
    sw=StanzaWrapper()
    text_en = 'I like Beijing!'
    result1 = sw.sentiment(text_en)
    sw.print_result(result1)

    text_zh='我讨厌苹果！'
    result2=sw.sentiment_chinese(text_zh)
    sw.print_result(result2)

Example 7: Language detection from text

from nerkit.StanzaApi import StanzaWrapper
if __name__=="__main__":
    sw=StanzaWrapper()
    list_text = ['I like Beijing!','我喜欢北京！', "Bonjour le monde!"]
    result1 = sw.lang(list_text)
    sw.print_result(result1)

Example 8: Language detection from text with a user-defined processing function

from nerkit.StanzaApi import StanzaWrapper
if __name__=="__main__":
    sw=StanzaWrapper()
    list_text = ['I like Beijing!','我喜欢北京！', "Bonjour le monde!"]
    def process(model):# do your own business
        doc=model["doc"]
        print(f"{doc.sentences[0].dependencies_string()}")
    result1 = sw.lang_multi(list_text,func_process=process,download_lang='en,zh,fr')
    print(result1)
    sw.print_result(result1)

Example 9: Stanza's NER (Legacy use for Java-based Stanford CoreNLP)

from nerkit.StanzaApi import *
# First, set environment variable CORENLP_HOME to the CoreNLP folder
corenlp_root_path=r"stanford-corenlp-4.3.2"
text="我喜欢游览广东孙中山故居景点！"
list_token=get_entity_list(text,corenlp_root_path=corenlp_root_path,language="chinese")
for token in list_token:
    print(f"{token['value']}\t{token['pos']}\t{token['ner']}")

Example 10: Stanford CoreNLP (Not official version)

import os
from nerkit.StanfordCoreNLP import get_entity_list
text="我喜欢游览广东孙中山故居景点！"
current_path = os.path.dirname(os.path.realpath(__file__))
res=get_entity_list(text,resource_path=f"{current_path}/stanfordcorenlp/stanford-corenlp-latest/stanford-corenlp-4.3.2")
print(res)
for w,tag in res:
    if tag in ['PERSON','ORGANIZATION','LOCATION']:
        print(w,tag)

Credits & References

• Stanza
• Stanford CoreNLP

License

The ner-kit project is provided by Donghua Chen.