Skip to main content

A debugging and profiling tool that can trace and visualize python code

Project description

CodeSnap

build pypi

CodeSnap is a deterministic debugging/profiling tool that can trace and visualize python code. The major data CodeSnap displays is FEE(function entry/exit), or equivalently, the call stack.

Unlike traditional flame graph, which is normally generated by sampling profiler, CodeSnap can display every function executed and the corresponding entry/exit time from the beginning of the program to the end, which is helpful for programmers to catch sporatic performance issues.

With CodeSnap, the programmer can intuitively understand what their code is doing and how long each function takes.

You can take a look at the demo result of an example program running recursive merge and quick sort algorithm.

trace viewer is used to display the stand alone html data.

CodeSnap also supports json output that complies with Chrome trace event format, which can be loaded using perfetto

Requirements

CodeSnap requires python 3.5+. No other package is needed. For now, CodeSnap binary on pip only supports CPython + Linux. However, in theory the source code can build on Windows/MacOS.

Install

The prefered way to install CodeSnap is via pip

pip install codesnap

You can also download the source code and build it yourself.

Usage

There are a couple ways to use CodeSnap

Command Line

The easiest way to use CodeSnap it through command line. Assume you have a python script to profile and the normal way to run it is:

python3 my_script.py

You can simply use CodeSnap as

python3 -m codesnap my_script.py

which will generate a result.html file in the directory you run this command. Open it in browser and there's your result.

If your script needs arguments like

python3 my_script.py arg1 arg2

Just feed it as it is to CodeSnap

python3 -m codesnap my_script.py arg1 arg2

You can also specify the tracer to be used in command line by passing --tracer argument. c tracer is the default value, you can use python tracer instead

python3 -m codesnap --tracer c my_script.py
python3 -m codesnap --tracer python my_script.py

You can specify the output file using -o or --output_file argument. The default output file is result.html. Two types of files are supported, html and json.

python3 -m codesnap -o other_name.html my_script.py
python3 -m codesnap -o other_name.json my_script.py

Inline

Sometimes the command line may not work as you expected, or you do not want to profile the whole script. You can manually start/stop the profiling in your script as well.

First of all, you need to import CodeSnap class from the package, and make an object of it.

from codesnap import CodeSnap

snap = CodeSnap()

If your code is executable by exec function, you can simply call snap.run()

snap.run("import random;random.randrange(10)")

This will as well generate a result.html file in your current directory. You can pass other file path to the function if you do not like the name result.html

snap.run("import random; random.randrange(10)", output_file = "better_name.html")

When you need a more delicate profiler, you can manually enable/disable the profile using start() and stop() function.

snap.start()
# Something happens here
snap.stop()
snap.save() # also takes output_file as an optional argument

With this method, you can only record the part that you are interested in

# Some code that I don't care
snap.start()
# Some code I do care
snap.stop()
# Some code that I want to skip
snap.start()
# Important code again
snap.stop()
snap.save()

It is higly recommended that start() and stop() function should be in the same frame(same level on call stack). Problem might happen if the condition is not met

Display Result

By default, CodeSnap will generate a stand alone HTML file which you can simply open with Chrome(maybe Firefox?). The front-end uses trace-viewer to show all the data.

However, you can generate json file as well, which complies to the chrome trace event format. You can load the json file on perfetto, which will replace the deprecated trace viewer in the future.

At the moment, perfetto did not support locally stand alone HTML file generation, so I'm not able to switch completely to it. The good news is that once you load the perfetto page, you can use it even when you are offline.

Choose Tracer

The default tracer for current version is c tracer, which introduce a relatively small overhead(worst case 2-3x) but only works for CPython on Linux. However, if there's other reason that you would prefer a pure-python tracer, you can use python tracer using tracer argument when you initialize CodeSnap object.

snap = CodeSnap(tracer="python")

Cleanup of c Tracer

The interface for c trace is almost exactly the same as python tracer, except for the fact that c tracer does not support command line run now. However, to achieve lower overhead, some optimization is applied to c tracer so it will withhold the memory it allocates for future use to reduce the time it calls malloc(). If you want the c trace to free all the memory it allocates while collecting trace, use

snap.cleanup()

Performance

Overhead is a big consideration when people choose profilers. CodeSnap now has a similar overhead as native cProfiler. It works slightly worse in the worst case(Pure FEE) and better in easier case because even though it collects some extra information than cProfiler, the structure is lighter.

Admittedly, CodeSnap is only focusing on FEE now, so cProfiler also gets other information that CodeSnap does not acquire.

An example run for test_performance with Python 3.8 / Ubuntu 18.04.4 on Github VM

fib       (10336, 10336): 0.000852800 vs 0.013735200(16.11)[py] vs 0.001585900(1.86)[c] vs 0.001628400(1.91)[cProfile]
hanoi     (8192, 8192): 0.000621400 vs 0.012924899(20.80)[py] vs 0.001801800(2.90)[c] vs 0.001292900(2.08)[cProfile]
qsort     (10586, 10676): 0.003457500 vs 0.042572898(12.31)[py] vs 0.005594100(1.62)[c] vs 0.007573200(2.19)[cProfile]
slow_fib  (1508, 1508): 0.033606299 vs 0.038840998(1.16)[py] vs 0.033270399(0.99)[c] vs 0.032577599(0.97)[cProfile]

Limitations

CodeSnap uses sys.setprofile() for its profiler capabilities, so it will conflict with other profiling tools which also use this function. Be aware of it when using CodeSnap.

Bugs/Requirements

Please send bug reports and feature requirements through github issue tracker. CodeSnap is currently under development now and it's open to any constructive suggestions.

License

Copyright Tian Gao, 2020.

Distributed under the terms of the Apache 2.0 license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

codesnap-0.1.0-cp38-cp38-manylinux2010_x86_64.whl (670.9 kB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64

codesnap-0.1.0-cp38-cp38-manylinux1_x86_64.whl (670.9 kB view details)

Uploaded CPython 3.8

codesnap-0.1.0-cp37-cp37m-manylinux2010_x86_64.whl (670.8 kB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64

codesnap-0.1.0-cp37-cp37m-manylinux1_x86_64.whl (670.8 kB view details)

Uploaded CPython 3.7m

codesnap-0.1.0-cp36-cp36m-manylinux2010_x86_64.whl (669.9 kB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64

codesnap-0.1.0-cp36-cp36m-manylinux1_x86_64.whl (669.8 kB view details)

Uploaded CPython 3.6m

codesnap-0.1.0-cp35-cp35m-manylinux2010_x86_64.whl (669.6 kB view details)

Uploaded CPython 3.5m manylinux: glibc 2.12+ x86-64

codesnap-0.1.0-cp35-cp35m-manylinux1_x86_64.whl (669.6 kB view details)

Uploaded CPython 3.5m

File details

Details for the file codesnap-0.1.0-cp38-cp38-manylinux2010_x86_64.whl.

File metadata

  • Download URL: codesnap-0.1.0-cp38-cp38-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 670.9 kB
  • Tags: CPython 3.8, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.1.0-cp38-cp38-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 831edb6cdb059ab8cd006085e834c9e16400d2d2cfb4b1bce24cf75c18594d86
MD5 a9a37839b43060cf4779339dc9a229fd
BLAKE2b-256 d5e4a7f937f5302e1ec70c405b144bc823fcf1e6523157cd34485e63b2e632a6

See more details on using hashes here.

File details

Details for the file codesnap-0.1.0-cp38-cp38-manylinux1_x86_64.whl.

File metadata

  • Download URL: codesnap-0.1.0-cp38-cp38-manylinux1_x86_64.whl
  • Upload date:
  • Size: 670.9 kB
  • Tags: CPython 3.8
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.1.0-cp38-cp38-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 874bed55bba193c845967a96f02fd91480a28214370627d12681e3afb00c6936
MD5 121ef9eaef1d88b4ffdbcac938a8f617
BLAKE2b-256 3c5175bc4fb0568123b6d5ac52a20c49211f469e323349a03f18bed327e9963e

See more details on using hashes here.

File details

Details for the file codesnap-0.1.0-cp37-cp37m-manylinux2010_x86_64.whl.

File metadata

  • Download URL: codesnap-0.1.0-cp37-cp37m-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 670.8 kB
  • Tags: CPython 3.7m, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.1.0-cp37-cp37m-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 22bc4dfe29daccc390ae23397e7133d4301e9a7fa721d84a985761813178bc2d
MD5 e37c59f1bff32db6b91412d9f49201a5
BLAKE2b-256 03f72c740451951377f8b8f6b81cbbe5beecd71c801cc4006d3acff8afe791de

See more details on using hashes here.

File details

Details for the file codesnap-0.1.0-cp37-cp37m-manylinux1_x86_64.whl.

File metadata

  • Download URL: codesnap-0.1.0-cp37-cp37m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 670.8 kB
  • Tags: CPython 3.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.1.0-cp37-cp37m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 70fcecae98890f79e64da3433d7835de14eeb8bcb64b74b392b6aa89cc4cdd44
MD5 aac2ed972d42db707db0ed77a74dc2cf
BLAKE2b-256 cb6cbe4aaada92c1f504b734d0c0bda8b75bd05d294f53b3986f6e50197f3b08

See more details on using hashes here.

File details

Details for the file codesnap-0.1.0-cp36-cp36m-manylinux2010_x86_64.whl.

File metadata

  • Download URL: codesnap-0.1.0-cp36-cp36m-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 669.9 kB
  • Tags: CPython 3.6m, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.1.0-cp36-cp36m-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 4202370ef731e85f6bf966ad07dfbc441669b9c0309fbe4af1823d2f064d7e67
MD5 a4187f8adda9e42aae3e6e7a9f507d6b
BLAKE2b-256 03ba32be5b2340e99951c8ec3df25eb295d2c29a19d9305eccc52e6de38b9b70

See more details on using hashes here.

File details

Details for the file codesnap-0.1.0-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

  • Download URL: codesnap-0.1.0-cp36-cp36m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 669.8 kB
  • Tags: CPython 3.6m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.1.0-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 f6c031ada84a1e1a8c231e46b508a795cd51905c78030c269091e5b39ba1a6ef
MD5 c7893a8a04e30312276f48a11fb05590
BLAKE2b-256 b5785fa16d30793f2aa28cbb596bfcbe71a210257a1ef0d69a9e099f16fd4c63

See more details on using hashes here.

File details

Details for the file codesnap-0.1.0-cp35-cp35m-manylinux2010_x86_64.whl.

File metadata

  • Download URL: codesnap-0.1.0-cp35-cp35m-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 669.6 kB
  • Tags: CPython 3.5m, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.1.0-cp35-cp35m-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 b50b7e12af2aeb0896303fa3b986c7919f46096cde7cb2c22709d0c39af8ba7f
MD5 feb957552d9a9d5922ab3e70c9462927
BLAKE2b-256 7eb5daee5edfed7b379e7fafebe50f6fbdb9e7289948284aea208cb7b2a78f2a

See more details on using hashes here.

File details

Details for the file codesnap-0.1.0-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

  • Download URL: codesnap-0.1.0-cp35-cp35m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 669.6 kB
  • Tags: CPython 3.5m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.1.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.8.5

File hashes

Hashes for codesnap-0.1.0-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 1d09c56595d572cb1b4b45a00c49fd739ac807dba87a01b0508a9a303b68fd5a
MD5 38ab96273dda9d72d6f3a69996acf496
BLAKE2b-256 443f0b6b50eec531dd24e3ffe973bd75c21a7d295a2be6c7b24840402abde78b

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page