shirkhan
the python tool library for shirkhan
currently using with project u-open-imla
Usage
安装
pip install shirkhan
更新
pip install --upgrade shirkhan
删除
pip uninstall shirkhan
from shirkhan import decode, encode, syllabify
encode("xxxx")
decode("yyy")
word = "شىرخان"
print(syllabify(word))
功能示例：
分音节
思路：
1. 把单词向量化，按照元音，辅音 的0，1 值生成token 0100100
2. 从后往前分析 所以需要反转 retoken 0010010
3. 把retoken 以元音为分界分组 001 001 0
4. 按照分音节通用算法进行给retoken 植入分隔符
- 两个元音之间有1个辅音它属于前面的音节
- 两个元音之间有2个辅音它一个属于前面的，一个属于后面的
- 两个元音之间有3个辅音 第一个属于前面的，后两个属于后面的
- 两个元音之间有4个辅音 第一个属于前面的，其后的两个一组，最后一个属于后面的 【shirkhan 给自己出的规则，目前没有任何凭据这么做，而且是不对的】
- 两个元音之间有5个辅音 第一个属于前面的，其后的三个一组，最后一个属于后面的 【shirkhan 给自己出的规则，目前没有任何凭据这么做，而且是不对的】
5. 把嵌入分割符的retoken分割点坐标映射到原始内容上 i -> len(word)-i
6. 按照分割符切割
from shirkhan import syllabify
print(syllabify('شىرخان'))
# output ['شىر', 'خان']
元音辅音组合的向量
from shirkhan import SWord
target_word = "شىرخان"
print(SWord(target_word).tokenize())
# output 010010
组合向量分组
from shirkhan import SWord
target_word = "شىرخاننىڭمۇ"
sw = SWord(target_word)
gtoken = sw.get_grouped_token()
gretoken = sw.get_grouped_retoken()
print(sw.tokenize())
print(gtoken)
print(gretoken)
# 01001001001
# [['0', '1'], ['0', '0', '1'], ['0', '0', '1'], ['0', '0', '1']]
# [['1'], ['0', '0', '1'], ['0', '0', '1'], ['0', '0', '1'], ['0']]
分音原始内容
from shirkhan import SWord
target_word = "شىرخاننىڭمۇ"
sw = SWord(target_word)
print(sw.get_positional_word())
print(sw.get_positional_token())
print(sw.get_positional_retoken())
# شىرxخانxنىڭxمۇ
# 010x010x010x01
# 10x010x010x010
单词生成字单词
# 第一种方式
from shirkhan import SWord
target_word = "شىرخاننىڭمۇ"
sw = SWord(target_word)
print(sw.get_similar_words())
# 第二种方式
from shirkhan import SWord
target_word = "شىرخاننىڭمۇ"
print(SWord(target_word))
# output:
# ['شىر', 'شىرخان', 'شىرخاننىڭ', 'شىرخاننىڭمۇ']
1 maintainer
Metadata
1 maintainer
Release files for shirkhan 0.0.28
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| shirkhan-0.0.28.tar.gz | 23.6 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| shirkhan-0.0.28-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 52.5 kB
Release files / shirkhan-0.0.28.tar.gz
| Download URL | shirkhan-0.0.28.tar.gz |
|---|---|
| Size | 23.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
701392023e028367e7bcb36f30ec15c2e7041fe9c21e02ee8b806d6bf8557f20
|
|
BLAKE2b-256 checksum How to use checksums |
362de81cc3a89a445c630d8ea51296d2738211c168abb9b67ae23a61213f9477
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.10.6
|
Release files / shirkhan-0.0.28-py3-none-any.whl
| Download URL | shirkhan-0.0.28-py3-none-any.whl |
|---|---|
| Size | 29.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
524146bce4f7f5e8df7c89ff043fc308dd39652ebdb8ae641ba68bc55d20e1f5
|
|
BLAKE2b-256 checksum How to use checksums |
adec2a1ca2d08c4807a213e5b45c1f1f8cc1ca41d8fbce32b35d5ea1b70b5e71
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.10.6
|
Security for shirkhan 0.0.28
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
