Torchserve dashboard using Streamlit
Project description
Torchserve Dashboard
Torchserve Dashboard using Streamlit
Related blog post
Usage
Additional Requirement: torchserve (recommended:v0.5.0)
Simply run:
pip3 install torchserve-dashboard --user
# torchserve-dashboard [streamlit_options(optional)] -- [config_path(optional)] [model_store(optional)] [log_location(optional)] [metrics_location(optional)]
torchserve-dashboard
#OR change port
torchserve-dashboard --server.port 8105 -- --config_path ./torchserve.properties
#OR provide a custom configuration
torchserve-dashboard -- --config_path ./torchserve.properties --model_store ./model_store
:exclamation: Keep in mind that If you change any of the --config_path
,--model_store
,--metrics_location
,--log_location
options while there is a torchserver already running before starting torch-dashboard they won't come into effect until you stop&start torchserve. These options are used instead of their respective environment variables TS_CONFIG_FILE, METRICS_LOCATION, LOG_LOCATION
.
OR
git clone https://github.com/cceyda/torchserve-dashboard.git
streamlit run torchserve_dashboard/dash.py
#OR
streamlit run torchserve_dashboard/dash.py --server.port 8105 -- --config_path ./torchserve.properties
Example torchserve config:
inference_address=http://127.0.0.1:8443
management_address=http://127.0.0.1:8444
metrics_address=http://127.0.0.1:8445
grpc_inference_port=7070
grpc_management_port=7071
number_of_gpu=0
batch_size=1
model_store=./model_store
If the server doesn't start for some reason check if your ports are already in use!
Updates
[15-oct-2020] add scale workers tab
[16-feb-2021] (functionality) make logpath configurable,(functionality)remove model_name requirement,(UI)add cosmetic error messages
[10-may-2021] update config & make it optional. update streamlit. Auto create folders
[31-may-2021] Update to v0.4 (Add workflow API) Refactor out streamlit from api.py.
[30-nov-2021] Update to v0.5, adding support for encrypted model serving (not tested). Update streamlit to v1+
FAQs
-
Does torchserver keep running in the background?
The torchserver is spawned using
Popen
and keeps running in the background even if you stop the dashboard. -
What about environment variables?
These environment variables are passed to the torchserve command:
ENVIRON_WHITELIST=["LD_LIBRARY_PATH","LC_CTYPE","LC_ALL","PATH","JAVA_HOME","PYTHONPATH","TS_CONFIG_FILE","LOG_LOCATION","METRICS_LOCATION","AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "AWS_DEFAULT_REGION"]
-
How to change the logging format of torchserve?
You can set the location of your custom log4j config in your configuration file as in here
vmargs=-Dlog4j.configuration=file:///path/to/custom/log4j.properties
-
What is the meaning behind the weird versioning?
The minor follows the compatible torchserve version, patch version reflects the dashboard versioning
Help & Question & Feedback
Open an issue
TODOs
- Async?
- Better logging
- Remote only mode
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for torchserve_dashboard-0.5.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | dca704f59286d5eeb27f93fd0cddbba9b9c8023ca48a097445672a6ba1a1f2f4 |
|
MD5 | 9e39494153066ccaf9a6cba91ed0b10b |
|
BLAKE2b-256 | ea1ffa8e78e19f76a23bf3477922ef509ae39d8ca5d76280ba946f37905fc363 |
Hashes for torchserve_dashboard-0.5.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 012dd43b832ee15acfb190228206776005a1a0f79d3a2042a6f0951cf658900f |
|
MD5 | f9609947c922afc6e8f4bd72b07c457c |
|
BLAKE2b-256 | 1d49cdf48c0ef76ae38973fd3df9a92ad4b4e4a1fe2149f7f5f98572cc3aadc9 |