Estnltk — open source tools for Estonian natural language processing
Project description
EstNLTK -- Open source tools for Estonian natural language processing
EstNLTK provides common natural language processing functionality such as paragraph, sentence and word tokenization, morphological analysis, named entity recognition, etc. for the Estonian language.
The project is funded by EKT (Eesti Keeletehnoloogia Riiklik Programm).
This package contains EstNLTK's basic linguistic analysis, system and database tools:
Text
class with the Estonian NLP pipeline;- tokenization tools: word, sentence and paragraph tokenization; clause segmentation;
- morphology tools: morphological analysis and disambiguation, spelling correction, morphological synthesis and syllabification, HFST based analyser and GT converter;
- information extraction tools: named entity recognizer, temporal expression tagger, verb chain detector, address tagger, adjective phrase tagger, phrase tagger and grammar based fact extraction;
- syntactic analysis tools: preprocessing for syntactic analysis, VislCG3 and Maltparser based syntactic parsers;
- Estonian Wordnet;
- web taggers -- such as bert embeddings web tagger, stanza syntax web tagger and stanza ensemble syntax web tagger;
- corpus importing tools -- tools for importing data from large Estonian corpora, such as the Reference Corpus or the National Corpus of Estonia;
- system taggers -- regex tagger, disambiguator, atomizer, merge tagger etc;
- Postgres database tools;
Version 1.7
Installation
EstNLTK is available for osx, windows-64, and linux-64, and for python versions 3.7 to 3.9. You can install the latest version via PyPI:
pip install estnltk==1.7.0rc0
Alternatively, you can install EstNLTK via Anaconda. Installation steps with conda:
- create a conda environment with python 3.8, for instance:
conda create -n py38 python=3.8
- activate the environment, for instance:
conda activate py38
- install EstNLTK with the command:
conda install -c estnltk -c conda-forge estnltk=1.7.0rc0
Note: for using some of the tools in estnltk, you also need to have Java installed in your system. We recommend using Oracle Java http://www.oracle.com/technetwork/java/javase/downloads/index.html, although alternatives such as OpenJDK (http://openjdk.java.net/) should also work.
Using on Google Colab
You can install EstNLTK on Google Colab environment via command:
!pip install estnltk==1.7.0rc0
Documentation
EstNLTK's tutorials come in the form of jupyter notebooks. However, updating tutorials is work-in-progress. Most of the tutorials of version 1.6 should also work for version 1.7, although imports may need fixing:
Additional educational materials on EstNLTK are available on web pages of an NLP course taught at the University of Tartu:
- https://github.com/d009/EstNLP (in Estonian)
Note: if you have trouble viewing jupyter notebooks in github (you get an error message Sorry, something went wrong. Reload? at loading a notebook), then try to open notebooks with the help of https://nbviewer.jupyter.org
Source
The source of the package development can be found at branch devel_1.6_split. The main repository is https://github.com/estnltk/estnltk
Development changelog.
Citation
In case you use EstNLTK in your work, please cite us as follows:
@InProceedings{laur-EtAl:2020:LREC,
author = {Laur, Sven and Orasmaa, Siim and Särg, Dage and Tammo, Paul},
title = {EstNLTK 1.6: Remastered Estonian NLP Pipeline},
booktitle = {Proceedings of The 12th Language Resources and Evaluation Conference},
month = {May},
year = {2020},
address = {Marseille, France},
publisher = {European Language Resources Association},
pages = {7154--7162},
url = {https://www.aclweb.org/anthology/2020.lrec-1.884}
}
If you use EstNLTK v1.4.1 (or older), please cite:
@InProceedings{ORASMAA16.332,
author = {Siim Orasmaa and Timo Petmanson and Alexander Tkachenko and Sven Laur and Heiki-Jaan Kaalep},
title = {EstNLTK - NLP Toolkit for Estonian},
booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016)},
year = {2016},
month = {may},
date = {23-28},
location = {Portorož, Slovenia},
editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Marko Grobelnik and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis},
publisher = {European Language Resources Association (ELRA)},
address = {Paris, France},
isbn = {978-2-9517408-9-1},
language = {english}
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for estnltk-1.7.0rc0-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f3fd3da166f404529e3260c3e85b1c345ee18462392c1b2d995fed27e2187b87 |
|
MD5 | 4a98edd549ec1939be7137ad9565af2a |
|
BLAKE2b-256 | f55017aa246580cd18a5f225e10486aa8557cebace55dd87253daf27fe1cb86e |
Hashes for estnltk-1.7.0rc0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 72343573cec2786f653974e2cb1c0ad98c5f71b0d2c48dd6b958f2393b5e1ba5 |
|
MD5 | 9bf6d99e6ed1a92314e78a40bf75ec16 |
|
BLAKE2b-256 | 844dc657af95b8014b450f1a74b3e7c7bf8561229b2697c2f11987cf440466a6 |
Hashes for estnltk-1.7.0rc0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 77394da2abd3709176a4f1d50f809fa42f3d00ae72dc9eef0868e0297bd1b004 |
|
MD5 | 8e65df15717bd4d390c7ab4a8e3495db |
|
BLAKE2b-256 | fa294bc426978a6388de82db3147dfd0af830482466448c0567b1470f4ae18c9 |
Hashes for estnltk-1.7.0rc0-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | be8953e179b0bbc8cbe42c0237cc59bd482e3219b67bcd1857676f88c73fab4e |
|
MD5 | 00e18b335ef6f45b5141b72173e85578 |
|
BLAKE2b-256 | 5573e65cf2099e62651ffdbdbf84d3375b50700a54bb5a026cea98224999a025 |
Hashes for estnltk-1.7.0rc0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 683246b309b100c775a3a8cb8e0b2f97830a05a5f11c8cf5507f8d877e7eeeae |
|
MD5 | d7ce25b73b797b9989352e99f05f97ad |
|
BLAKE2b-256 | fa4d01b0b269c3d59294f9c814937f2091bfe6f931134eb1f8d24b4675892669 |
Hashes for estnltk-1.7.0rc0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9e324755f6a7089beccb53be32764daeb9a4dcd47480357be28a2f3071fb22d9 |
|
MD5 | 04444fe112422ff0436dd16e1af8a0ce |
|
BLAKE2b-256 | 7228a4481eaecd29b9b9d0ccee6bfbc8c2f08a203ab68c13633fa3ca7e62b80d |
Hashes for estnltk-1.7.0rc0-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 605307d0ebc6d25aa419008d5606f51677ba3bf465dd9c643e9226f4f9f9bc27 |
|
MD5 | 7ea216dc10338bf1063983d25a26fda8 |
|
BLAKE2b-256 | 2d2bb69fc0aa86643da3ff81ab579454e83c9300298db1a454bd31f1884b24a8 |
Hashes for estnltk-1.7.0rc0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 20b78c59390ed412fe6b89eeb6bfa7c9ead9ce20869c9bb8ec5fa602547a9435 |
|
MD5 | f7c30ab9ed7c8e846062d51e64452460 |
|
BLAKE2b-256 | 0b414c1563371d6136db5189fe10807d7cc80bcf88a0e15e99bf9bd6b072679e |
Hashes for estnltk-1.7.0rc0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b20b8c01f68a9ab7eefcc8f633ef09622f37afabb3597dd5c98681b10250cef5 |
|
MD5 | 95040bc9f10bff9118327009ab07c418 |
|
BLAKE2b-256 | 347fe1abf76bf74bf45c644e20c279e6a81f5381193be1586eb0bbba7dcaf9b9 |