Skip to main content

A profiling tool that can visualize python code in flame graph

Project description

CodeSnap

build pypi

CodeSnap is a light-weighted deterministic profiling tool that can visualize python code running result in flame graph. The major data CodeSnap displays is FEE(function entry/exit), or equivalently, the call stack.

Unlike traditional flame graph, which is normally generated by sampling profiler, CodeSnap can display every function executed the the corresponding entry/exit time from the beginning of the program to the end, which is helpful for programmers to catch sporatic performance issues.

With CodeSnap, the programmer can intuitively understand what their code is doing and how long each function takes.

You can take a look at the demo result of an example program running recursive merge and quick sort algorithm.

A modified version of d3-flame-graph is used to display the data.

Requirements

CodeSnap requires python 3.5+. No other package is needed. For now, CodeSnap binary on pip only supports CPython + Linux. However, in theory the source code can build on Windows/MacOS.

Install

The prefered way to install CodeSnap is via pip

pip install codesnap

You can also download the source code and build it yourself.

Usage

There are a couple ways to use CodeSnap

Command Line

The easiest way to use CodeSnap it through command line. Assume you have a python script to profile and the normal way to run it is:

python3 my_script.py

You can simply use CodeSnap as

python3 -m codesnap my_script.py

which will generate a result.html file in the directory you run this command. Open it in browser and there's your result.

If your script needs arguments like

python3 my_script.py arg1 arg2

Just feed it as it is to CodeSnap

python3 -m codesnap my_script.py arg1 arg2

Inline

Sometimes the command line may not work as you expected, or you do not want to profile the whole script. You can manually start/stop the profiling in your script as well.

First of all, you need to import CodeSnap class from the package, and make an object of it.

from codesnap import CodeSnap

snap = CodeSnap()

If your code is executable by exec function, you can simply call snap.run()

snap.run("import random;random.randrange(10)")

This will as well generate a result.html file in your current directory. You can pass other file path to the function if you do not like the name result.html

snap.run("import random; random.randrange(10)", output_file = "better_name.html")

When you need a more delicate profiler, you can manually enable/disable the profile using start() and stop() function.

snap.start()
# Something happens here
snap.stop()
snap.save() # also takes output_file as an optional argument

With this method, you can only record the part that you are interested in

# Some code that I don't care
snap.start()
# Some code I do care
snap.stop()
# Some code that I want to skip
snap.start()
# Important code again
snap.stop()
snap.save()

It is higly recommended that start() and stop() function should be in the same frame(same level on call stack). Problem might happen if the condition is not met

Choose tracer

The default tracer for current version is a pure-python tracer, which works fine with any OS/Interpreter. However, the overhead it introduces is pretty large. In the worst case(a lot of FEEs), it might be 20-30x.

To deal with this problem, the c tracer is under development and is currently available for CPython on Linux. It might work on Windows, but I've never tested it.

To use c tracer, set the optional argument tracer to "c" when you initialize CodeSnap object.

snap = CodeSnap(tracer="c")

Cleanup of c tracer

The interface for c trace is almost exactly the same as python tracer, except for the fact that c tracer does not support command line run now. However, to achieve lower overhead, some optimization is applied to c tracer so it will withhold the memory it allocates for future use to reduce the time it calls malloc(). If you want the c trace to free all the memory it allocates while collecting trace, use

snap.cleanup()

Performance

Overhead is a big consideration when people choose profilers. CodeSnap now has a similar overhead as native cProfiler. It works slightly worse in the worst case(Pure FEE) and better in easier case because even though it collects some extra information than cProfiler, the structure is lighter.

Admittedly, CodeSnap is only focusing on FEE now, so cProfiler also gets other information that CodeSnap does not acquire.

An example run for test_performance with Python 3.8 / Ubuntu 18.04.4 on Github VM

fib       (10336, 10336): 0.000852800 vs 0.013735200(16.11)[py] vs 0.001585900(1.86)[c] vs 0.001628400(1.91)[cProfile]
hanoi     (8192, 8192): 0.000621400 vs 0.012924899(20.80)[py] vs 0.001801800(2.90)[c] vs 0.001292900(2.08)[cProfile]
qsort     (10586, 10676): 0.003457500 vs 0.042572898(12.31)[py] vs 0.005594100(1.62)[c] vs 0.007573200(2.19)[cProfile]
slow_fib  (1508, 1508): 0.033606299 vs 0.038840998(1.16)[py] vs 0.033270399(0.99)[c] vs 0.032577599(0.97)[cProfile]

Limitations

CodeSnap uses sys.setprofile() for its profiler capabilities, so it will conflict with other profiling tools which also use this function. Be aware of it when using CodeSnap.

Bugs/Requirements

Please send bug reports and feature requirements through github issue tracker. CodeSnap is currently under development now and it's open to any constructive suggestions.

License

Copyright Tian Gao, 2020.

Distributed under the terms of the Apache 2.0 license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

codesnap-0.0.4-cp38-cp38-manylinux2010_x86_64.whl (54.0 kB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64

codesnap-0.0.4-cp38-cp38-manylinux1_x86_64.whl (54.0 kB view details)

Uploaded CPython 3.8

codesnap-0.0.4-cp37-cp37m-manylinux2010_x86_64.whl (53.9 kB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64

codesnap-0.0.4-cp37-cp37m-manylinux1_x86_64.whl (53.9 kB view details)

Uploaded CPython 3.7m

codesnap-0.0.4-cp36-cp36m-manylinux2010_x86_64.whl (52.9 kB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64

codesnap-0.0.4-cp36-cp36m-manylinux1_x86_64.whl (52.9 kB view details)

Uploaded CPython 3.6m

codesnap-0.0.4-cp35-cp35m-manylinux2010_x86_64.whl (52.7 kB view details)

Uploaded CPython 3.5m manylinux: glibc 2.12+ x86-64

codesnap-0.0.4-cp35-cp35m-manylinux1_x86_64.whl (52.7 kB view details)

Uploaded CPython 3.5m

File details

Details for the file codesnap-0.0.4-cp38-cp38-manylinux2010_x86_64.whl.

File metadata

  • Download URL: codesnap-0.0.4-cp38-cp38-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 54.0 kB
  • Tags: CPython 3.8, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.0.4-cp38-cp38-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 97115d7dbcd8577b30c01a62cca719e62a8c2f62168705478405d1d4035eb3d5
MD5 198bcac2bc5de2a8caf374e0971716f6
BLAKE2b-256 e81970a423b9c484e5f06e723c5c38396744162c39e839b2ca376db55ed6db1b

See more details on using hashes here.

File details

Details for the file codesnap-0.0.4-cp38-cp38-manylinux1_x86_64.whl.

File metadata

  • Download URL: codesnap-0.0.4-cp38-cp38-manylinux1_x86_64.whl
  • Upload date:
  • Size: 54.0 kB
  • Tags: CPython 3.8
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.0.4-cp38-cp38-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 7b69052a913af02e9a79032146bd30cf9a11c70925cc85fbffebd6620eafba88
MD5 a7c7b27e33d252989a10882d6336dd6a
BLAKE2b-256 3b96dd9faf698f7cbc7db492d2b7c642ce61c379f4685aa4b1af64d08a3fa3c3

See more details on using hashes here.

File details

Details for the file codesnap-0.0.4-cp37-cp37m-manylinux2010_x86_64.whl.

File metadata

  • Download URL: codesnap-0.0.4-cp37-cp37m-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 53.9 kB
  • Tags: CPython 3.7m, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.0.4-cp37-cp37m-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 58144aca732bd8520b74515644c05bcc4261bbcf564d0dda559938d825d396c1
MD5 bf9f50c858fdf0499659612c3d36a9e0
BLAKE2b-256 9410a9e4916f61ed7adbc7c627d395b7358d311ceebf409150382474378d7ff2

See more details on using hashes here.

File details

Details for the file codesnap-0.0.4-cp37-cp37m-manylinux1_x86_64.whl.

File metadata

  • Download URL: codesnap-0.0.4-cp37-cp37m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 53.9 kB
  • Tags: CPython 3.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.0.4-cp37-cp37m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 e6e92d28e0ae470f1bebd0e3c07df622b37e052ea2f6c0bdce08cfc78665c4d8
MD5 d7e12934280daf78311869ba5d15269b
BLAKE2b-256 0925d70eae8a6fc3e0c32ab1b72b5f1eb37ab6df05518f37078d6046fe87ef4d

See more details on using hashes here.

File details

Details for the file codesnap-0.0.4-cp36-cp36m-manylinux2010_x86_64.whl.

File metadata

  • Download URL: codesnap-0.0.4-cp36-cp36m-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 52.9 kB
  • Tags: CPython 3.6m, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.0.4-cp36-cp36m-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 0087c5e79e8a8384a1bb5bf825bf234b058a45736c5d1904190a445b52754beb
MD5 d733e46bfe171f183985abeda25a8680
BLAKE2b-256 2733a1820574cbafc70e947ac5eff08720f411ecbbb435af8ee5cd9f7435e394

See more details on using hashes here.

File details

Details for the file codesnap-0.0.4-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

  • Download URL: codesnap-0.0.4-cp36-cp36m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 52.9 kB
  • Tags: CPython 3.6m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.0.4-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 d2e0243f15cc5fc7f8ff7cc963001eac3d14137a0be9ce5dff7633e16a9d339e
MD5 e41c7cdb680a9748c584c4e2b2e2c24c
BLAKE2b-256 d70e2de2f1540164518ee974e337f62bfce8513dcf1291a4f0cff01960bcef9b

See more details on using hashes here.

File details

Details for the file codesnap-0.0.4-cp35-cp35m-manylinux2010_x86_64.whl.

File metadata

  • Download URL: codesnap-0.0.4-cp35-cp35m-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 52.7 kB
  • Tags: CPython 3.5m, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.0.4-cp35-cp35m-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 e1d011efe6cc67c2683ee58712b663fa73f71cc41abb9fe0a4f2d6388b886e1d
MD5 5c76bf4276f5b87eaa6e2fb9dce047cd
BLAKE2b-256 5ba0495ee73cc6f5d99ca8cadfcccc7d608456ff97c0de17daeaf892c9444406

See more details on using hashes here.

File details

Details for the file codesnap-0.0.4-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

  • Download URL: codesnap-0.0.4-cp35-cp35m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 52.7 kB
  • Tags: CPython 3.5m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.0.4-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 bf128c2dab143a6665c0619ee1160b910ff7e0ae236c97457b590a73dbe1889e
MD5 bc422f849b083afe28438b30677beb1c
BLAKE2b-256 31e7e69eaf63db1f8e30cc6d1db442cb3cddce56e1a38a67aca95ee8157cecce

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page