some useful code
Project description
This is a project for later lazy work! Only support for python3, ☹️, but maybe you can try in python2
Install
命令行直接安装
pip install poros
从代码库安装
git clone https://github.com/diqiuzhuanzhuan/poros.git
cd poros
python setup install
Some code is from other people, and some is from me.
bert_model
usage:
- create pretrain data
from poros.bert_model import create_pretraining_data
>>> create_pretraining_data.create_data(input_file="./test_data/sample_text.txt",
output_file="./test_data/output", vocab_file="./test_data/vocab.txt")
- pretrain bert model
from poros.bert_model import pretrain
>>> pretrain.run(input_file="./test_data/output", bert_config_file="./test_data/bert_config.json",
output_dir="./output")
- prepare a trained model, tell classifier model
- prepare train.csv and test.csv, its format is like this: "id, text1, label"
- init the model, the code is like below
from poros.bert_model.run_classifier import SimpleClassifierModel
>>> model = SimpleClassifierModel(
bert_config_file="./data/chinese_L-12_H-768_A-12/bert_config.json",
vocab_file="./data/chinese_L-12_H-768_A-12/vocab.txt",
output_dir="./output",
max_seq_length=512,
train_file="./data/train.csv",
dev_file="./data/dev.csv",
init_checkpoint="./data/chinese_L-12_H-768_A-12/bert_model.ckpt",
label_list=[0, 1, 2, 3]
)
poros_chars
Provide a list of small functions
usage:
- convert chinese words into arabic number:
from poros.poros_chars import chinese_to_arabic
>>> print(chinese_to_arabic.NumberAdapter.convert("四千三百万"))
43000000
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
poros-0.0.16-py3-none-any.whl
(52.3 kB
view hashes)
Close
Hashes for poros-0.0.16.macosx-10.7-x86_64.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3deffab37d68cb92ee35570ca128467d501e1d26aa4fa87e8087e88ce1e52066 |
|
MD5 | 495a3ab3e95c17df7b6ec0bad647b458 |
|
BLAKE2b-256 | 4a109759aad44b67b434916151d954af815cd69261d30cd976a333dcc1bd1482 |