Skip to main content

Natural language processing tools for the Dhivehi language.

Project description


Natural language processing tools for the Dhivehi language.

Contains the following modules:

Tokenizer - Tokenize text into separate sentences or words (tokens).

Stopwords - Remove stopwords from text and return the resulting tokens.

Stemmer - Remove suffixes from words to return their root form.

Language Models - Create language models to predict future additions. Language models will give probability based on selected ngram. An ngram is contiguous sequence of n tokens from the given input text.

Dictionary- Get definitions definitions of Dhivehi words and the word list. Definitions obtained from

Corpus - Collections of various Dhivehi texts.

Trigram Similarity - Trigram similarity divides words or phrases into sequences of three consecutive letters, placed in a set where the order doesn't matter and duplicates are removed. Used to find string matches even if certain characters are different or out of order, based on similarity value.

Tagger - Tag words in text according to specified rules or patterns. For example, tagging words based on which part of speech it belongs to.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dhivehi_nlp-1.0.1.tar.gz (5.1 MB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page