shirkhan
the python tool for shirkhan
currently using with project u-open-imla
Usage
安装
pip install shirkhan
更新
pip install --upgrade shirkhan
删除
pip uninstall shirkhan
from shirkhan import decode, encode, syllabify
encode("xxxx")
decode("yyy")
word = "شىرخان"
print(syllabify(word))
shirkhan 专用脚本库的更多功能示例：
分音节
思路：
1. 把单词向量化，按照元音，辅音 的0，1 值生成token 0100100
2. 从后往前分析 所以需要反转 retoken 0010010
3. 把retoken 以元音为分界分组 001 001 0
4. 按照分音节通用算法进行给retoken 植入分隔符
- 两个元音之间有1个辅音它属于前面的音节
- 两个元音之间有2个辅音它一个属于前面的，一个属于后面的
- 两个元音之间有3个辅音 第一个属于前面的，后两个属于后面的
- 两个元音之间有4个辅音 第一个属于前面的，其后的两个一组，最后一个属于后面的 【shirkhan 给自己出的规则，目前没有任何凭据这么做，而且是不对的】
- 两个元音之间有5个辅音 第一个属于前面的，其后的三个一组，最后一个属于后面的 【shirkhan 给自己出的规则，目前没有任何凭据这么做，而且是不对的】
5. 把嵌入分割符的retoken分割点坐标映射到原始内容上 i -> len(word)-i
6. 按照分割符切割
from shirkhan import syllabify
print(syllabify('شىرخان'))
# output ['شىر', 'خان']
元音辅音组合的向量
from shirkhan import SWord
target_word = "شىرخان"
print(SWord(target_word).tokenize())
# output 010010
组合向量分组
from shirkhan import SWord, do_group
target_word = "شىرخان"
token = SWord(target_word).tokenize()
retoken = token[::-1] # 反转
print(do_group(retoken))
# output [['0', '1'], ['0', '0', '1'], ['0']]
分音原始内容
from shirkhan import SWord, position_transform, embed_delimiter
target_word = "شىرخان"
token = SWord(target_word).tokenize()
retoken = token[::-1]
print(position_transform(target_word, embed_delimiter(retoken)))
# output شىرxخان
单词繁殖器
from shirkhan import similar_word_generator
word = "شىرخاننىڭكى"
print(word)
print(similar_word_generator(word))
# شىرخاننىڭكى
# ['شىرخان', 'شىرخاننىڭ', 'شىر', 'شىرخاننىڭكى']
更多使用案例请看 examples目录中的脚本
开发，贡献指南
- 本地安装项目，安装时指定可编辑方式安装，这样可以边使用边看效果，甚至调整逻辑时不需要频繁修改源码后重新安装
pip install --editable shirkhan_lib_path // 这个目录指定到有setup.py为止即可
1 maintainer
Metadata
1 maintainer
Release files for shirkhan 0.0.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| shirkhan-0.0.5.tar.gz | 8.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| shirkhan-0.0.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 17.9 kB
Release files / shirkhan-0.0.5.tar.gz
| Download URL | shirkhan-0.0.5.tar.gz |
|---|---|
| Size | 8.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
027efe27295ee7d1c56b2788166a2f82151851329816e1fa755cf1733173e777
|
|
BLAKE2b-256 checksum How to use checksums |
dd2009ee103fe56314dd9c3ab37b87cea48d256f9eba3dead029501f9a521f04
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/53.0.0 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.9.0
|
Release files / shirkhan-0.0.5-py3-none-any.whl
| Download URL | shirkhan-0.0.5-py3-none-any.whl |
|---|---|
| Size | 9.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0a58421660d2326beb79228c7e85a2c0c8f48841dfa5cdcc1328b078151234ed
|
|
BLAKE2b-256 checksum How to use checksums |
e7623f7e0bcca6dac18d6ea221f6d42b9d9863760e05ef933d927ab75452653f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/53.0.0 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.9.0
|
