xiaojingcc

English | 中文

编码损坏（乱码）的检测与修复。

安装

pip install xiaojingcc

用法

from xiaojingcc import decode_bytes, looks_mojibake, repair

looks_mojibake("测试")      # True
repair("测试")              # '测试'
repair("正常的中文")           # 原样返回

decode_bytes(b"²âÊÔ", ("utf-8", "gbk"))   # ('测试', 'gbk')

说明

典型场景是 UTF-8 字节被按 latin-1 解码后再存成 UTF-8，中文变成 测试 这样的串。repair 按 latin-1 还原字节再按 UTF-8 解码，失败时原样返回。decode_bytes 按候选编码顺序试，全部失败抛 ValueError——不猜一个替代字符糊过去。

许可证

MIT

English

Detect and repair encoding damage (mojibake).

Install

pip install xiaojingcc

Usage

from xiaojingcc import decode_bytes, looks_mojibake, repair

looks_mojibake("测试")      # True
repair("测试")              # '测试'
repair("正常的中文")           # 原样返回

decode_bytes(b"²âÊÔ", ("utf-8", "gbk"))   # ('测试', 'gbk')

Notes

The classic case is UTF-8 bytes decoded as latin-1 and saved again, turning text into 测试. repair re-encodes as latin-1 and decodes as UTF-8, returning the input unchanged when that fails. decode_bytes tries candidate encodings in order and raises ValueError when none works.

License

MIT