No project description provided
Project description
kuro2sudachi
kuro2sudachi lets you to convert kuromoji user dict to sudachi user dict.
Usage
$ pip install kuro2sudachi
# prepase riwirte.def
# https://github.com/WorksApplications/Sudachi/blob/develop/src/main/resources/rewrite.def
$ ls
rewiite.def
$ kuro2sudachi kuromoji_dict.txt -o sudachi_user_dict.txt
Custom pos convert dict
you can overwrite convert config with setting json file.
{
"固有名詞": {
"sudachi_pos": "名詞,固有名詞,一般,*,*,*",
"left_id": 4786,
"right_id": 4786,
"cost": 5000
},
"名詞": {
"sudachi_pos": "名詞,普通名詞,一般,*,*,*",
"left_id": 5146,
"right_id": 5146,
"cost": 5000
}
}
$ kuro2sudachi kuromoji_dict.txt -o sudachi_user_dict.txt -c convert_config.json
if you want to ignore unsupported pos error & invalid format, use --ignore
flag.
Dictionary type
You can specify the dictionary with the tokenize option -s (default: core).
$ pip install sudachidict_full
$ kuro2sudachi kuromoji_dict.txt -o sudachi_user_dict.txt -s full
Splitting Words
Currently, the CLI does not support word splitting. Therefore, the split representation of kuromoji is ignored.
中咽頭ガン,中咽頭 ガン,チュウイントウ ガン,カスタム名詞
↓
中咽頭ガン,4786,4786,7000,中咽頭ガン,名詞,固有名詞,一般,*,*,*,チュウイントウガン,中咽頭ガン,*,*,*,*,*
Develop
test kuro2sudachi
$ poetry install
$ poetry run pytest
exec kuro2sudachi command
$ poetry run kuro2sudachi tests/kuromoji_dict_test.txt -o sudachi_user_dict.txt
TODO
- split mode
- default rewrite
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
kuro2sudachi-0.2.8.tar.gz
(7.4 kB
view hashes)
Built Distribution
Close
Hashes for kuro2sudachi-0.2.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0e82c6b35a41e60527b828023c95d8e9fd6b56c1fba259a62c19d2e7593ab31d |
|
MD5 | c409415008d787a19e1fa947eafd2ac3 |
|
BLAKE2b-256 | 42a6fe687dac1ad5e2a1a392ec16f225c497829cf9169459ede0247544b9bca6 |