NLU Trainer
nlu-trainer 是一个用于训练nlu模型的工具,它主要有以下特点:
- 完整的命令行应用,使用方便
- 端侧和云侧模型皆可训练
- 中英模型皆可训练
- ddp多卡训练
- fasttext模型量化和导出onnx模型功能
- 模型测试功能
- 模型发布功能
安装
- 安装pytorch
- 安装nlu-trainer
pip install nlu-trainer --upgrade -i http://192.168.130.5:5002/simple --trusted-host 192.168.130.5 --extra-index-url https://mirrors.aliyun.com/pypi/simple/
使用
项目初始化
nlu-trainer init --name schedule_cmn
当你执行完上述命令后,你会初始化一个名为schedule_cmn的项目,项目结构如下:
schedule_cmn
├── configs
│ ├── corpus.yaml
│ ├── dataset.yaml
│ ├── docs.yaml
│ ├── intentention.yaml
│ ├── ner.yaml
│ ├── domain.yaml
|-- corpus
│ ├── abnf
│ ├── |--- run.sh
│ ├── Generator.jar
│ ├── neg_extra.txt
│ ├── pos_extra.txt
数据准备
- 在
corpus/abnf/目录下面编写abnf语法文件,注意:语法文件的命名必须是*.abnf,且每个文件的名称必须是意图标签名称 - 命令行生成NLUDoc格式文档
nlu-trainer prepare docs --help
- 生成领域训练语料
nlu-trainer prepare corpus --help
- 生成数据集
nlu-trainer prepare dataset --help
模型训练
- 训练意图识别模型
# 查看帮助
nlu-trainer train intention --help
# 通过配置文件训练
nlu-trainer train intention --config configs/intention.yaml
- 训练槽位识别模型
# 查看帮助
nlu-trainer train ner --help
# 通过配置文件训练
nlu-trainer train ner --config configs/ner.yaml
- 训练领域识别模型
# 查看帮助
nlu-trainer train domain --help
# 通过配置文件训练
nlu-trainer train domain --config configs/domain.yaml
模型测试
- 添加领域测试用例:
corpus/pos_extra中添加正向测试用例, 在corpus/neg_extra中添加负向测试用例 - 添加nlu测试用例:
# 查看帮助
nlu-trainer test add --help
- 列出测试用例
# 查看帮助
nlu-trainer test list --help
- 删除测试用例
# 查看帮助
nlu-trainer test delete --help
- 测试云端模型
# 查看帮助
nlu-trainer test cloud --help
- 测试端侧模型
# 查看帮助
nlu-trainer test local --help
- 推送模型
# 查看帮助
nlu-trainer push cloud --help
nlu-trainer push local --help
Metadata
Release files for nlu-trainer 0.2.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| nlu_trainer-0.2.1.tar.gz | 367.6 kB | Without provenance | Details for nlu_trainer-0.2.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| nlu_trainer-0.2.1-py3-none-any.whl | Python 3 | none | any | Details for nlu_trainer-0.2.1-py3-none-any.whl |
Total release size: 739.2 kB
Release files / nlu_trainer-0.2.1.tar.gz
| Download URL | nlu_trainer-0.2.1.tar.gz |
|---|---|
| Size | 367.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e14ea932bde4466dcd154ab908d7eedcab1d3fdd71db2bf490bf40300191c1f8
|
|
BLAKE2b-256 checksum How to use checksums |
6bb3c53609a33db874ad1ad3a6b8b3e2c1858f97aac2d87d0fcaf4b324180d39
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.6.1 CPython/3.11.4 Linux/3.10.0-1062.el7.x86_64
|
Release files / nlu_trainer-0.2.1-py3-none-any.whl
| Download URL | nlu_trainer-0.2.1-py3-none-any.whl |
|---|---|
| Size | 371.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5551def3c5ac266560be689708893d876c4b69e89a29a1b276ec2fba7e7cfbb8
|
|
BLAKE2b-256 checksum How to use checksums |
8dd71ae54e61fde0ac054df44681f637eebdeb4283ca73263232b084933f32d8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.6.1 CPython/3.11.4 Linux/3.10.0-1062.el7.x86_64
|
Security for nlu-trainer 0.2.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.