cchardet 1.1.3

Universal encoding detector. This library is faster than chardet.

cChardet

cChardet is high speed universal character encoding detector. - binding to charsetdetect.

Support codecs

• Big5

• EUC-JP

• EUC-KR

• GB18030

• HZ-GB-2312

• IBM855

• IBM866

• ISO-2022-CN

• ISO-2022-JP

• ISO-2022-KR

• ISO-8859-2

• ISO-8859-5

• ISO-8859-7

• ISO-8859-8

• KOI8-R

• Shift_JIS

• TIS-620

• UTF-8

• UTF-16BE

• UTF-16LE

• UTF-32BE

• UTF-32LE

• WINDOWS-1250

• WINDOWS-1251

• WINDOWS-1252

• WINDOWS-1253

• WINDOWS-1255

• EUC-TW

• X-ISO-10646-UCS-4-2143

• X-ISO-10646-UCS-4-3412

• x-mac-cyrillic

Requirements

• Cython

Example

# -*- coding: utf-8 -*-
import cchardet as chardet
with open(r"src/tests/testdata/wikipediaJa_One_Thousand_and_One_Nights_SJIS.txt", "rb") as f:
    msg = f.read()
    result = chardet.detect(msg)
    print(result)

Benchmark

$ cd src/
$ pip install chardet
$ python tests/bench.py

Results

CPU: Intel(R) Core(TM) i3-4170 CPU @ 3.70GHz

RAM: DDR3 1600Mhz 16GB

Platform: Ubuntu 16.04 amd64

Python 2.7.12

	Request (call/s)
chardet	0.26
cchardet	1408.73

Python 3.5.2

	Request (call/s)
chardet	0.28
cchardet	1380.40

License

• The MIT License: src/cchardet

• Other Libraries License: Please, look at the src/ext directory.

Thanks

• uchardet-enhanced

• Cython

Contact

Issues

CHANGES

1.1.3 (2017-02-26)

• Support AArch64

1.1.2 (2017-01-08)

• Support Python 3.6

1.1.1 (2016-11-05)

• Use len() function (9e61cb9e96b138b0d18e5f9e013e144202ae4067)

• Remove detect function in _cchardet.pyx (25b581294fc0ae8f686ac9972c8549666766f695)

• Support manylinux1 wheel

1.1.0 (2016-10-17)

• Add Detector class

• Improve unit tests