pycld3 0.8

CLD3 Python bindings

pycld3

Python bindings to the Compact Language Detector v3 (CLD3).

This package contains Python bindings (via Cython) to Google's CLD3 library.

Installation

Install via Pip:

python -m pip install pycld3

Building from Source

To build this extension, you will need:

• Cython
• Protobuf, including the protoc Protobuf compiler available as an executable

Building the extension does not require the Chromium repository.

Usage

cld3 exports two module-level functions, get_language() and get_frequent_languages():

>>> import cld3

>>> cld3.get_language("影響包含對氣候的變化以及自然資源的枯竭程度")
LanguagePrediction(language='zh', probability=0.999969482421875, is_reliable=True, proportion=1.0)

>>> cld3.get_language("This is a test")
LanguagePrediction(language='en', probability=0.9999980926513672, is_reliable=True, proportion=1.0)

>>> for lang in cld3.get_frequent_languages(
...     "This piece of text is in English. Този текст е на Български.",
...     num_langs=3
... ):
...     print(lang)
...
LanguagePrediction(language='bg', probability=0.9173890948295593, is_reliable=True, proportion=0.5853658318519592)
LanguagePrediction(language='en', probability=0.9999790191650391, is_reliable=True, proportion=0.4146341383457184)
LanguagePrediction(language='und', probability=0.0, is_reliable=False, proportion=0.0)

FAQ

cld3 incorrectly detects my input, how can I fix this?

In some cases, you cannot. Language detection algorithms in general may perform poorly with very short inputs. Rarely should you trust the output of something like detect("hi"). Keep this limitation in mind regardless of what library you are using.

How do I fix an error telling me "The Protobuf compiler, protoc, could not be found"?

The Protobuf compiler, protoc, is required for building this package. (However, if you are installing from PyPI with pip, then the .h and .cc files generated with protoc will already be included.)

Below are some quick install commands, but please consult the official protobuf repository for information on installing Protobuf.

Ubuntu Linux:

sudo apt-get update
sudo apt-get install protobuf-compiler

Mac OSX:

brew update && brew install protobuf

Authors

This repository contains a fork of google/cld3 at commit 06f695f. The license for google/cld3 can be found at LICENSES/CLD3_LICENSE.

This repository is a combination of changes introduced by various forks of google/cld3 by the following people:

• Johannes Baiter (@jbaiter)
• Elizabeth Myers (@Elizafox)
• Witold Bołt (@houp)
• Alfredo Luque (@iamthebot)
• WISESIGHT (@ThothMedia)
• RNogales (@RNogales94)
• Brad Solomon (@bsolomon1124)