Skip to main content

A language processing tool for Sinhalese (සිංහල)

Project description

A language processing tool for Sinhalese (සිංහල).

Update 2020.11.01: Fixed pypi package. Use 'pip install sinling' to install sinling directly from repository.

Update 2020.08.16: Add pypi package @ https://pypi.org/project/sinling/.

Update 2020.08.16: Integrated Part of speech tagger and stemmer tool.

Update 2019.07.21: This tool no longer requires java to run sinhala tokenizer. All java code is ported to Python implementation for convenience.

Binder PyPI version

Installation

Run the following command in your virtualenv to install this package.

pip install sinling

How to use

Sinhala Tokenizer

from sinling import SinhalaTokenizer

tokenizer = SinhalaTokenizer()

sentence = '...'  # your sentence

tokenizer.tokenize(sentence)

Sinhala Stemmer (Experimental)

from sinling import SinhalaStemmer

stemmer = SinhalaStemmer()

word = '...'  # your sentence

stemmer.stem(word)

Please cite sinhala-stemmer if you are using this implementation.

Part-of-Speech Tagger

from sinling import SinhalaTokenizer, POSTagger

tokenizer = SinhalaTokenizer()

document = '...'  # may contain multiple sentences

tokenized_sentences = [tokenizer.tokenize(f'{ss}.') for ss in tokenizer.split_sentences(document)]

tagger = POSTagger()

pos_tags = tagger.predict(tokenized_sentences)

Word Joiner (Morphological Joiner)

from sinling import preprocess, word_joiner

w1 = preprocess('මුනි')
w2 = preprocess('උතුමා')
results = word_joiner.join(w1, w2)
# Returns a list of possible results after applying join rules ['මුනිතුමා', ...]

Word Splitter (Morphological Splitter) / corpus based - experimental

from sinling import word_splitter

word = '...'
results = word_splitter.split(word)
# Returns a dict containing debug information, base word and affix

Visit here to see some sample splits.

Contributions

  • Contact wayasas.13@cse.mrt.ac.lk if you would like to contribute to this project.

License

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sinling-0.3.6.tar.gz (19.6 MB view details)

Uploaded Source

Built Distribution

sinling-0.3.6-py3-none-any.whl (20.0 MB view details)

Uploaded Python 3

File details

Details for the file sinling-0.3.6.tar.gz.

File metadata

  • Download URL: sinling-0.3.6.tar.gz
  • Upload date:
  • Size: 19.6 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.24.0 setuptools/49.2.1 requests-toolbelt/0.9.1 tqdm/4.51.0 CPython/3.9.0

File hashes

Hashes for sinling-0.3.6.tar.gz
Algorithm Hash digest
SHA256 a0c9cbd49823aab972b5ad059bb02bd315eff6dea480fc8025324b648919af93
MD5 6584b3ed1312b1e31da7d65aacebad4b
BLAKE2b-256 fc42b512f34da021decfed3b04fc21b806f96b71d6b76e5e211d492dccd38fe6

See more details on using hashes here.

File details

Details for the file sinling-0.3.6-py3-none-any.whl.

File metadata

  • Download URL: sinling-0.3.6-py3-none-any.whl
  • Upload date:
  • Size: 20.0 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.24.0 setuptools/49.2.1 requests-toolbelt/0.9.1 tqdm/4.51.0 CPython/3.9.0

File hashes

Hashes for sinling-0.3.6-py3-none-any.whl
Algorithm Hash digest
SHA256 eb3a58ede6531edd9865c8ae4f39ab74cb044bdc896848a0c088c836a91bb3cc
MD5 b2ab74213d99462634f0bbba226b9379
BLAKE2b-256 23bf43a39e626dfd56002d74c646cc4ab88b396c17e889bad03e4a27fbc10265

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page