a tool that collects and visualizes TiFlash runtime infomation
Project description
Usage
# install from pypi
pip3 install flashprof
# collect tiflash logs from tiup cluster to current directory, which will also be parsed to json
flashprof collect --cluster $CLUSTER_NAME
# render all cluster runtime info, currently supports task DAG
flashprof render
# help
flashprof -h
flashprof <subcommand> -h
DAG
Currently only task runtime info is visualized as DAG. Tasks in a single query may span across several tiflash instances, and those with status != FINISHED
or error_message != ""
will be labelled with red border.
data layout
The collected/generated artifacts have the following layout
flashprof
└── cluster
├── cluster1_name
│ ├── log (collected from tiflash log dir)
│ │ ├── ip1.tiflash.log
│ │ └── ip2.tiflash.log
│ └── task_dag (parsed and combined task dag)
│ ├── json
│ │ ├── ip1.tiflash.log.task_dag.json
│ │ ├── ip2.tiflash.log.task_dag.json
│ │ └── cluster.task_dag.json
│ ├── png (rendered png files)
│ └── svg (rendered svg files)
└── cluster2_name
...
Development
# install a local dev version of python package, then we can call flashprof
# rerun this when code is changed
# it internally creates a symbolic link to the current source code
pip3 install -e .
# remove if you want
pip3 uninstall flashprof
Packaging
TL;DR
pip3 install build
python3 -m build
twine check dist/*
twine upload dist/*
# upload to test.pypi.org for package publish related test
# twine upload --repository testpypi dist/*
Instructions
Please refer to https://packaging.python.org/guides/distributing-packages-using-setuptools/ for detaild instructions.
Internals
collect
command collects tiflash logs according to the tiup configurations for the specified --cluster $CLUSTER_NAME
, and logs are named $IP.tiflash.log
in flashprof/cluster/$CLUSTER_NAME/log
.
parse
command parses all the tiflash logs collected above to the json format, which only contains task DAGs for now. The json files a then merged into a cluster.json
in flashprof/cluster/$CLUSTER_NAME/task_dag/json
.
render
command renders cluster.json
into dag graphs per query_tso
in flashprof/cluster/$CLUSTER_NAME/$FORMAT
.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flashprof-0.0.14-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 346ac9cfd9fd89e28b1a2902cc15c43ed54ffed4714d60a5672764ba06abebe7 |
|
MD5 | e23a26e61cf02fe5e490bfe4f89ae8ef |
|
BLAKE2b-256 | 7f10249d4a611c36558d87d50ed15c1346d7bd0a0e02f1ebdcb0c78a1e07da7b |