A profiling tool that can visualize python code in flame graph
Project description
CodeSnap
CodeSnap is a light-weighted deterministic profiling tool that can visualize python code running result in flame graph. The major data CodeSnap displays is FEE(function entry/exit), or equivalently, the call stack.
Unlike traditional flame graph, which is normally generated by sampling profiler, CodeSnap can display every function executed the the corresponding entry/exit time from the beginning of the program to the end, which is helpful for programmers to catch sporatic performance issues.
With CodeSnap, the programmer can intuitively understand what their code is doing and how long each function takes.
You can take a look at the demo result of an example program running recursive merge and quick sort algorithm.
A modified version of d3-flame-graph is used to display the data.
Requirements
CodeSnap requires python 3.5+. No other package is needed. For now, CodeSnap binary on pip only supports CPython + Linux. However, in theory the source code can build on Windows/MacOS.
Install
The prefered way to install CodeSnap is via pip
pip install codesnap
You can also download the source code and build it yourself.
Usage
There are a couple ways to use CodeSnap
Command Line
The easiest way to use CodeSnap it through command line. Assume you have a python script to profile and the normal way to run it is:
python3 my_script.py
You can simply use CodeSnap as
python3 -m codesnap my_script.py
which will generate a result.html
file in the directory you run this command. Open it in browser and there's your result.
If your script needs arguments like
python3 my_script.py arg1 arg2
Just feed it as it is to CodeSnap
python3 -m codesnap my_script.py arg1 arg2
Inline
Sometimes the command line may not work as you expected, or you do not want to profile the whole script. You can manually start/stop the profiling in your script as well.
First of all, you need to import CodeSnap
class from the package, and make an object of it.
from codesnap import CodeSnap
snap = CodeSnap()
If your code is executable by exec
function, you can simply call snap.run()
snap.run("import random;random.randrange(10)")
This will as well generate a result.html
file in your current directory. You can pass other file path to the function if you do not like the name result.html
snap.run("import random; random.randrange(10)", output_file = "better_name.html")
When you need a more delicate profiler, you can manually enable/disable the profile using start()
and stop()
function.
snap.start()
# Something happens here
snap.stop()
snap.save() # also takes output_file as an optional argument
With this method, you can only record the part that you are interested in
# Some code that I don't care
snap.start()
# Some code I do care
snap.stop()
# Some code that I want to skip
snap.start()
# Important code again
snap.stop()
snap.save()
It is higly recommended that start()
and stop()
function should be in the same frame(same level on call stack). Problem might happen if the condition is not met
Choose tracer
The default tracer for current version is a pure-python tracer, which works fine with any OS/Interpreter. However, the overhead it introduces is pretty large. In the worst case(a lot of FEEs), it might be 20-30x.
To deal with this problem, the c tracer is under development and is currently available for CPython on Linux. It might work on Windows, but I've never tested it.
To use c tracer, set the optional argument tracer
to "c"
when you initialize CodeSnap
object.
snap = CodeSnap(tracer="c")
Cleanup of c tracer
The interface for c trace is almost exactly the same as python tracer, except for the fact that c tracer does not support command line run now. However, to achieve lower overhead, some optimization is applied to c tracer so it will withhold the memory it allocates for future use to reduce the time it calls malloc()
. If you want the c trace to free all the memory it allocates while collecting trace, use
snap.cleanup()
Performance
Overhead is a big consideration when people choose profilers. CodeSnap now has a similar overhead as native cProfiler. It works slightly worse in the worst case(Pure FEE) and better in easier case because even though it collects some extra information than cProfiler, the structure is lighter.
Admittedly, CodeSnap is only focusing on FEE now, so cProfiler also gets other information that CodeSnap does not acquire.
An example run for test_performance with Python 3.8 / Ubuntu 18.04.4 on Github VM
fib (10336, 10336): 0.000852800 vs 0.013735200(16.11)[py] vs 0.001585900(1.86)[c] vs 0.001628400(1.91)[cProfile]
hanoi (8192, 8192): 0.000621400 vs 0.012924899(20.80)[py] vs 0.001801800(2.90)[c] vs 0.001292900(2.08)[cProfile]
qsort (10586, 10676): 0.003457500 vs 0.042572898(12.31)[py] vs 0.005594100(1.62)[c] vs 0.007573200(2.19)[cProfile]
slow_fib (1508, 1508): 0.033606299 vs 0.038840998(1.16)[py] vs 0.033270399(0.99)[c] vs 0.032577599(0.97)[cProfile]
Limitations
CodeSnap uses sys.setprofile()
for its profiler capabilities, so it will conflict with other profiling tools which also use this function. Be aware of it when using CodeSnap.
Bugs/Requirements
Please send bug reports and feature requirements through github issue tracker. CodeSnap is currently under development now and it's open to any constructive suggestions.
License
Copyright Tian Gao, 2020.
Distributed under the terms of the Apache 2.0 license.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
File details
Details for the file codesnap-0.0.4-cp38-cp38-manylinux2010_x86_64.whl
.
File metadata
- Download URL: codesnap-0.0.4-cp38-cp38-manylinux2010_x86_64.whl
- Upload date:
- Size: 54.0 kB
- Tags: CPython 3.8, manylinux: glibc 2.12+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 97115d7dbcd8577b30c01a62cca719e62a8c2f62168705478405d1d4035eb3d5 |
|
MD5 | 198bcac2bc5de2a8caf374e0971716f6 |
|
BLAKE2b-256 | e81970a423b9c484e5f06e723c5c38396744162c39e839b2ca376db55ed6db1b |
File details
Details for the file codesnap-0.0.4-cp38-cp38-manylinux1_x86_64.whl
.
File metadata
- Download URL: codesnap-0.0.4-cp38-cp38-manylinux1_x86_64.whl
- Upload date:
- Size: 54.0 kB
- Tags: CPython 3.8
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7b69052a913af02e9a79032146bd30cf9a11c70925cc85fbffebd6620eafba88 |
|
MD5 | a7c7b27e33d252989a10882d6336dd6a |
|
BLAKE2b-256 | 3b96dd9faf698f7cbc7db492d2b7c642ce61c379f4685aa4b1af64d08a3fa3c3 |
File details
Details for the file codesnap-0.0.4-cp37-cp37m-manylinux2010_x86_64.whl
.
File metadata
- Download URL: codesnap-0.0.4-cp37-cp37m-manylinux2010_x86_64.whl
- Upload date:
- Size: 53.9 kB
- Tags: CPython 3.7m, manylinux: glibc 2.12+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 58144aca732bd8520b74515644c05bcc4261bbcf564d0dda559938d825d396c1 |
|
MD5 | bf9f50c858fdf0499659612c3d36a9e0 |
|
BLAKE2b-256 | 9410a9e4916f61ed7adbc7c627d395b7358d311ceebf409150382474378d7ff2 |
File details
Details for the file codesnap-0.0.4-cp37-cp37m-manylinux1_x86_64.whl
.
File metadata
- Download URL: codesnap-0.0.4-cp37-cp37m-manylinux1_x86_64.whl
- Upload date:
- Size: 53.9 kB
- Tags: CPython 3.7m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e6e92d28e0ae470f1bebd0e3c07df622b37e052ea2f6c0bdce08cfc78665c4d8 |
|
MD5 | d7e12934280daf78311869ba5d15269b |
|
BLAKE2b-256 | 0925d70eae8a6fc3e0c32ab1b72b5f1eb37ab6df05518f37078d6046fe87ef4d |
File details
Details for the file codesnap-0.0.4-cp36-cp36m-manylinux2010_x86_64.whl
.
File metadata
- Download URL: codesnap-0.0.4-cp36-cp36m-manylinux2010_x86_64.whl
- Upload date:
- Size: 52.9 kB
- Tags: CPython 3.6m, manylinux: glibc 2.12+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0087c5e79e8a8384a1bb5bf825bf234b058a45736c5d1904190a445b52754beb |
|
MD5 | d733e46bfe171f183985abeda25a8680 |
|
BLAKE2b-256 | 2733a1820574cbafc70e947ac5eff08720f411ecbbb435af8ee5cd9f7435e394 |
File details
Details for the file codesnap-0.0.4-cp36-cp36m-manylinux1_x86_64.whl
.
File metadata
- Download URL: codesnap-0.0.4-cp36-cp36m-manylinux1_x86_64.whl
- Upload date:
- Size: 52.9 kB
- Tags: CPython 3.6m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | d2e0243f15cc5fc7f8ff7cc963001eac3d14137a0be9ce5dff7633e16a9d339e |
|
MD5 | e41c7cdb680a9748c584c4e2b2e2c24c |
|
BLAKE2b-256 | d70e2de2f1540164518ee974e337f62bfce8513dcf1291a4f0cff01960bcef9b |
File details
Details for the file codesnap-0.0.4-cp35-cp35m-manylinux2010_x86_64.whl
.
File metadata
- Download URL: codesnap-0.0.4-cp35-cp35m-manylinux2010_x86_64.whl
- Upload date:
- Size: 52.7 kB
- Tags: CPython 3.5m, manylinux: glibc 2.12+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e1d011efe6cc67c2683ee58712b663fa73f71cc41abb9fe0a4f2d6388b886e1d |
|
MD5 | 5c76bf4276f5b87eaa6e2fb9dce047cd |
|
BLAKE2b-256 | 5ba0495ee73cc6f5d99ca8cadfcccc7d608456ff97c0de17daeaf892c9444406 |
File details
Details for the file codesnap-0.0.4-cp35-cp35m-manylinux1_x86_64.whl
.
File metadata
- Download URL: codesnap-0.0.4-cp35-cp35m-manylinux1_x86_64.whl
- Upload date:
- Size: 52.7 kB
- Tags: CPython 3.5m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | bf128c2dab143a6665c0619ee1160b910ff7e0ae236c97457b590a73dbe1889e |
|
MD5 | bc422f849b083afe28438b30677beb1c |
|
BLAKE2b-256 | 31e7e69eaf63db1f8e30cc6d1db442cb3cddce56e1a38a67aca95ee8157cecce |