Simple open-source Rasa command-line add-on that generates training model health reports for your projects.
Project description
Simple open-source Rasa command-line add-on that generates training model health reports for your projects.
🔍 About
Rasa is the most popular open-source framework for building chat and voice-based AI assistants. The rasa-model-report is a unofficial Rasa add-on to facilitate the work of developers and curators of Rasa chatbots. Rasa provides a lot of valuable data that can be "faceted" and extract different information about the training model. This information makes it possible to discover problems in the training model. The rasa-model-report does just that, it extracts this information to be displayed more clearly in a report. You can see this example.
rasa-model-report is an open-source project.
☕ Donate
You can collaborate with this project donating any amount. Thanks! ☕
📜 Changelog
Changelog can be found here. You can also follow the releases on Github and find planned enhancements for project in the Project Board.
📦 Installation
This module is distributed via Pypi and is required to use Python v3.8 or higher. To install the package, use the command:
pip install rasa-model-report
🚀 Execution
Before anything, is necessary to have the reports generated by the rasa test
command. To run the program, use the command:
rasa-model-report
This command must be used in the root of your Rasa project. Otherwise, you can use --path
parameter to pass the project path.
Step-by-step
This is the step-by-step guide for using rasa-model-report in your project.
- Go to the root folder of your Rasa project.
- Train model on your Rasa project using
rasa train
command. - Run Rasa end-to-end tests using
rasa test
command.- This command will generate some data in json, markdown and image files in
result/
directory. - This data is needed for rasa-model-report to generate the report.
- This command will generate some data in json, markdown and image files in
- (Optional) If you want to know model NLU rating for each sentence in your project, run your project's Rasa API through the command
rasa run --enable-api
.- When you run rasa-model-report, automatically it will request NLU rating for each sentence. The result will be in the NLU section of the report.
- If you don't want to use this option, just pass the parameter
--disable-nlu
or don't run Rasa API (if you don't run Rasa API, rasa-model-report will try to connect, after two tries it will skip this step).
- Run rasa-model-report in root project.
- If you haven't installed it, see how to install.
- The result will be in the
model_report.md
file generated in the project root folder.
Below, I created this video to show how to use the rasa-model-report. I used the Rasa sample project (from rasa-init
command). In this link is the generated report.
Furthermore, I used the tool in Sara - the Rasa Demo Bot. The result you can check here.
🦾 Rasa Version Support
Not every version of Rasa is supported. Check the table below:
Rasa version | Supported |
---|---|
3.X | ✅ |
2.X | ✅ |
1.X | ❌ |
0.X | ❌ |
⚙️ Options
There are parameters that can be used. Available options are below:
--actions-path TEXT Actions path. (default: actions/ inside Rasa project
path)
--disable-nlu Disable processing NLU sentences. NLU section will
not be generated in the report. Required Rasa API.
-e, --exclude LIST List of utter and actions that will be exclude in
the E2E test coverage. Use commas to separate items.
Example: utter_greet,utter_goodbye,action_listen
-h, --help Show this help message.
--model-link TEXT Model download link. It's only displayed in the
report to model download.
--no-images Generate model report without images.
--output-path TEXT Report output path. (default: ./)
-p, --path TEXT Rasa project path. (default: ./)
--precision INTEGER Score precision. Used to change precision of the
model report overview scores. Can vary between 0 and
5 (default: 2)
--project-name TEXT Rasa project name. It's only displayed in the
report. (default: My project)
--project-version TEXT Project version. It's only displayed in the report
for project documentation.
--rasa-api TEXT Rasa API URL. Is needed to create NLU section of
report. (default: http://localhost:5005)
--rasa-version TEXT Rasa version. It's only displayed in the report for
project documentation.
-v, --version Show installed rasa-model-report version.
Usage examples
Some usage examples with parameters:
- Without parameters is usually used at the root of the Rasa project.
rasa-model-report
- When you aren't at the root of the project, use
--path
parameter.rasa-model-report --path path/to/rasa/project
- Aren't at the root of the project and without NLU report.
rasa-model-report --path path/to/rasa/project --disable-nlu
- Aren't at the root of the project and change the report output directory.
rasa-model-report --path path/to/rasa/project --output-path path/to/place/report
- Aren't at the root of the project, the actions path isn't at the root project and change the report output directory.
rasa-model-report --path path/to/rasa/project --output-path path/to/place/report --actions-path path/to/actions/path
- If you want exclude some utters and actions from the E2E test coverage.
rasa-model-report --exclude utter_greet,action_help
💻 Development
The instructions for development and contributing are in the CONTRIBUTING.md file.
🐞 Bugs
Please file an issue for bugs, missing documentation, or unexpected behavior.
💬 Discussions
Please file an issue to suggest new features. Vote on feature requests. This helps maintainers prioritize what to work on.
❓ Questions
For questions related to using the add-on, please ask the community on Q&A.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for rasa-model-report-1.4.2b10.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | fceb96f27d49b34d111e1d31c7e439d8e65ac42c6fcb63d3c28e8ee2392e15a4 |
|
MD5 | a4762eb38a6bcac9c0674be302cdcb25 |
|
BLAKE2b-256 | 9aca35dfd6bc7e3ea59d0d078e3f6c20ac59007154327b8727cbd1a7a4216a8d |
Hashes for rasa_model_report-1.4.2b10-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | daad399abc69d8c898d4cedec27a290cc577232fd39c3a30b24400b61326fa21 |
|
MD5 | 21575fc0cf559547113dc091b9bde528 |
|
BLAKE2b-256 | af4ce6734ba44dd41e46b5f5db9ed78f566d7c0b0d9050b52a71ab1b1ca1dd76 |