Skip to main content

Python bindings for whatlang using pyo3

Project description

WhatLang ⚡

Python Binding for WhatLang, a blazing fast language detection library

PyPI version License PyPI - Downloads Build Status

Showcase

WhatLang is a Python library for detecting the language of a text. It is based on the WhatLang Rust library.

Installation

pip install whatlang-pyo3

Usage

Detect

>>> from whatlang import detect
>>> info = detect("This is written in English")
"Language: eng - Script: Latin - Confidence: 0.11450955767632877 - Is reliable: false"
>>> info.lang
"eng"
>>> info.script
"Latin"
>>> info.confidence
0.11450955767632877
>>> info.is_reliable
False
>>> info.to_iso()
"Language: en - Script: Latin - Confidence: 0.11450955767632877 - Is reliable: false"

Detect language

You can also detect the language of a text without the script and the confidence.

>>> from whatlang import detect_lang
>>> detect_lang("This is written in English")
"eng"
>>> detect_lang("Ceci est écrit en français")
"fra"

Detect script

You can also detect the script of a text without the language and the confidence.

>>> from whatlang import detect_script
>>> detect_script("This is written in English")
Name: Latin - Languages: spa, eng, por, ind, fra, deu, jav, vie, ita, tur, pol, ron, hrv, nld, uzb, hun, aze, ces, zul, swe, aka, sna, afr, fin, slk, tgl, tuk, dan, nob, cat, lit, slv, epo, lav, est, lat
>>> detect_script.name
"Latin"
>>> detect_script.langs
['spa', 'eng', 'por', 'ind', 'fra', 'deu', 'jav', 'vie', 'ita', 'tur', 'pol', 'ron', 'hrv', 'nld', 'uzb', 'hun', 'aze', 'ces', 'zul', 'swe', 'aka', 'sna', 'afr', 'fin', 'slk', 'tgl', 'tuk', 'dan', 'nob', 'cat', 'lit', 'slv', 'epo', 'lav', 'est', 'lat']

Batch detection

You can also detect the language of a list of texts, and take advantage of the parallelism with the n_jobs parameter.

>>> from whatlang import batch_detect
>>> batch_detect(["This is written in English", "Ceci est écrit en français"], n_jobs=-1)
[LanguageInfo(lang='eng', script='Latin', confidence=0.11450955767632877, is_reliable=False), LanguageInfo(lang='fra', script='Latin', confidence=0.11450955767632877, is_reliable=False)]

The performance of the batch detection is much better than the detection of a single text, it can be up to 5 times faster.

Contributing

Contributions are welcome! Please open an issue or a pull request.

Install maturin

We use maturin for the development of this library.

pip install maturin

Build

maturin build --release

License

This project is licensed under the MIT License - see the LICENSE file for details

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

whatlang_pyo3-0.5.0.tar.gz (1.3 MB view hashes)

Uploaded Source

Built Distributions

whatlang_pyo3-0.5.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.5.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.5.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.5.0-cp311-none-win_amd64.whl (352.5 kB view hashes)

Uploaded CPython 3.11 Windows x86-64

whatlang_pyo3-0.5.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.5.0-cp311-cp311-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (1.0 MB view hashes)

Uploaded CPython 3.11 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

whatlang_pyo3-0.5.0-cp310-none-win_amd64.whl (352.5 kB view hashes)

Uploaded CPython 3.10 Windows x86-64

whatlang_pyo3-0.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.5.0-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (1.0 MB view hashes)

Uploaded CPython 3.10 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

whatlang_pyo3-0.5.0-cp39-none-win_amd64.whl (352.7 kB view hashes)

Uploaded CPython 3.9 Windows x86-64

whatlang_pyo3-0.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.5.0-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (1.0 MB view hashes)

Uploaded CPython 3.9 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

whatlang_pyo3-0.5.0-cp38-none-win_amd64.whl (352.3 kB view hashes)

Uploaded CPython 3.8 Windows x86-64

whatlang_pyo3-0.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.5.0-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (1.0 MB view hashes)

Uploaded CPython 3.8 macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

whatlang_pyo3-0.5.0-cp37-none-win_amd64.whl (352.2 kB view hashes)

Uploaded CPython 3.7 Windows x86-64

whatlang_pyo3-0.5.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view hashes)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

whatlang_pyo3-0.5.0-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl (1.0 MB view hashes)

Uploaded CPython 3.7m macOS 10.9+ universal2 (ARM64, x86-64) macOS 10.9+ x86-64 macOS 11.0+ ARM64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page