Skip to main content

A tokenizer focused on Spanish language.

Project description

IAR Tokenizer

The IAR (Iván Arias Rodríguez) Tokenizer is a tokenizer developed mainly for Spanish. It is able to divide a text in paragraphs, those in sentences, and each sentence in a list of tokens.

More information to be added in the future...

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

iar_tokenizer-1.0.5.tar.gz (11.2 kB view hashes)

Uploaded Source

Built Distribution

iar_tokenizer-1.0.5-py3-none-any.whl (12.1 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page