Ho-Cho
Install
Mac
# Install MeCab
brew install mecab mecab-ipadic
# Install mecab-ipadic-neologd
git clone --depth 1 https://github.com/neologd/mecab-ipadic-neologd.git \
&& cd mecab-ipadic-neologd \
&& bin/install-mecab-ipadic-neologd -n -a -y
&& cd ..
pip install ho-cho
Ubuntu
# Install MeCab and mecab-ipadic-neologd
apt-get update && apt-get install -y mecab libmecab-dev mecab-ipadic mecab-ipadic-utf8
# Install mecab-ipadic-neologd
git clone --depth 1 https://github.com/neologd/mecab-ipadic-neologd.git \
&& cd mecab-ipadic-neologd \
&& bin/install-mecab-ipadic-neologd -n -a -y
&& cd ..
pip install ho-cho
Window
coming soon ...
Usage
1. cleaning
テキストのクリーニング
2. tokenizing
単語の分割
hocho/tokenizer/impl/mecab_tokenizer.py
3. normalization
単語の正規化
- 文字種の統一
- 数字の置き換え
- 辞書を用いた単語の統一
4. stopwords
ストップワードの除去
- 辞書による方法
- 出現頻度による方法
- 有名なストップワードを用いた除去方法
Development
How to develop
git pull origin main
git checkout -b feature/xxxx
git add .
git commit -m "xxx"
git push origin feature/xxx
Run test
pytest -v tests
Set up
pip install -e .
Publish to TestPyPI
# Install dependencies
pip install setuptools wheel twine
# Build
python setup.py sdist bdist_wheel
# Publish to TestPyPI
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
Publish to PyPI
# Install dependencies
pip install setuptools wheel twine
# Build
python setup.py sdist bdist_wheel
# Publish to PyPI
twine upload dist/*
1 maintainer
Metadata
1 maintainer
Release files for ho-cho 0.0.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ho-cho-0.0.3.tar.gz | 4.8 kB | Without provenance | Details for ho-cho-0.0.3.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ho_cho-0.0.3-py3-none-any.whl | Python 3 | none | any | Details for ho_cho-0.0.3-py3-none-any.whl |
Total release size: 11.0 kB
Release files / ho-cho-0.0.3.tar.gz
| Download URL | ho-cho-0.0.3.tar.gz |
|---|---|
| Size | 4.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
270392ff2867e2fec458888e6297fac2e6e7e0e1c8b7b54d169f29cb08561a01
|
|
BLAKE2b-256 checksum How to use checksums |
e954494fb82b4ca03d7ff858f700c81599f9b8cbab13cbfc4dc077916a1a5e64
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.1
|
Release files / ho_cho-0.0.3-py3-none-any.whl
| Download URL | ho_cho-0.0.3-py3-none-any.whl |
|---|---|
| Size | 6.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
07625a46ad208052a675697e3031593d22f2cb367bc56a167c5832d534ed354e
|
|
BLAKE2b-256 checksum How to use checksums |
f6c525e4054480a5bb73077d9cc3b390720a4c26d73564c61b9c0cbe63fc14cb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.1
|
Security for ho-cho 0.0.3
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
