Installing

Should be as simple as

$ pip install cld2-cffi

$ pip install --upgrade 'git+https://github.com/GregBowyer/cld2-cffi.git'

Usage

import cld2

isReliable, textBytesFound, details = cld2.detect("This is my sample text")
print('  reliable: %s' % (isReliable != 0))
print('  textBytes: %s' % textBytesFound)
print('  details: %s' % str(details))

# The output looks like so:
#  reliable: True
#  textBytes: 24
#  details: (('ENGLISH', 'en', 95, 1736.0), ('Unknown', 'un', 0, 0.0), ('Unknown', 'un', 0, 0.0))

Documentation

First, you must get your content (plain text or HTML) encoded into UTF8 bytes. Then, detect like this:

isReliable, textBytesFound, details = cld2.detect(bytes)

isReliable

is True if the top language is much better than 2nd best language.

textBytesFound

tells you how many actual bytes CLD analyzed (after removing HTML tags, collapsing areas of too-many-spaces, etc.).

details

has an entry per top 3 languages that matched, that includes the percent confidence of the match as well as a separate normalized score.

The module exports these global constants:

cld2.ENCODINGS

list of the encoding names CLD recognizes (if you provide hintEncoding, it must be one of these names).

cld2.LANGUAGES

list of languages and their codes (if you provide hintLanguageCode, it must be one of the codes from these codes).

cld2.EXTERNAL_LANGUAGES

list of external languages and their codes. Note that external languages cannot be hinted, but may be matched if you pass includeExtendedLanguages=True (the default).

cld2.DETECTED_LANGUAGES

list of all detectable languages, as best I can determine (this was reverse engineered from a unit test, ie it contains a language X if that language was tested and passes for at least one example text).

Authors

Please see AUTHORS.

Reporting bugs

Please see BUG_REPORTS.

Contribute

Please see CONTRIBUTING.

Licence

Please see LICENSE.