Skip to main content

This plugin implements a profiler for buildbot master.

  • It uses statistical profiling loosely based on plop https://github.com/bdarnell/plop

  • Suitable for prod as statistical profiling is very low overhead

  • Profiles all threads including main thread and db threads

  • Removes noise samples when the threads are actually in the main loop

  • In-browser UI based on nvd3 and d3-flame-graph

  • Show cpu and memory percent over time

  • flame graph can be restricted to a subset of the trace

  • Detailed caller/callee are displayed when clicking on a function

Usage

installation:

pip install buildbot_profiler

then in master.cfg:

c['www']['plugins']['profiler'] = True

Alternatively, you can install it in service mode, without the UI

c['services'] = [util.ProfilerService()]

ProfilerService takes following arguments:

ProfilerService(frequency=100, gatherperiod=30 * 60, mode='virtual', basepath=None, wantBuilds=100)
  • frequency: the profiling frequency in HZ. Not that if there is no activity during a profiling timer, no data will be recorded.

  • gatherperiod: the period during which the profiler works on one file. By default a new json file is created every 30min with the profile of the last 30min. Note that if there is no activity the gather period might be longer.

  • mode: profiling timer to use. Can be:

    • prof: use SIG_PROF timer,

    • virtual: use SIGVTALRM timer,

    • real: use SIGALRM timer,

  • basepath: the base path where to store the json files. (defaults to (master's basedir)/prof_)

  • wantBuilds: if this is > 0 then the last N builds are stored along side the profile data, for debug purpose.

Standalone Viewer

A standalone viewer is provided for offline browse of user submitted profiles.

bbprofiler

Then you can open your browser on http://localhost:8080

Screenshot

https://raw.githubusercontent.com/tardyp/buildbot_profiler/master/screenshot.png

ChangeLog

  • 1.3.1 Upgrade build to webpack, Fix argument parsing for record duration and sample rate

  • 1.2.3 Fixes for python2

  • 1.2.2 Fixes for python3

  • 1.2.1 Fix issues with load calculation

  • 1.2.0 Fold the flames so that you can explore larger traces.

    Fix the service so that the trace is reset between run.

  • 1.1.0 add a service to use the profiler without bother the users with a profiler UI.

  • 1.0.0 initial version

Metadata

Release files for buildbot-profiler 1.3.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for buildbot-profiler 1.3.1
File Interpreter ABI Platform
buildbot_profiler-1.3.1-py2.py3-none-any.whl Python 3, Python 2 none any Details

Release files / buildbot_profiler-1.3.1-py2.py3-none-any.whl

Download URL buildbot_profiler-1.3.1-py2.py3-none-any.whl
Size 517.5 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
2eb819dd54b76323e2c3e21b3b0fc9d2292174630dd3d73d14c500a634748548
BLAKE2b-256 checksum
How to use checksums
2dad309945597fc0cd4d2aec34c240d0f9fb0f05653940978d0d1f0e78a05275
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.3.0 pkginfo/1.6.1 requests/2.24.0 setuptools/50.3.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.8.6

Release history Release notifications | RSS feed

This release

1.3.1 This release

1 release file

1.3.0

1 release file

1.2.2

2 release files

1.2.1

2 release files

1.2.0

2 release files

1.1.1

2 release files

1.1.0

2 release files

1.0.0

1 release file

0.0.2

1 release file

0.0.1

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page