This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!
Project Description

Introduction

This product dumps stack traces of long running requests of a Zope2 instance to a log file. If a request takes more than a configured timeout, it’s stack trace will be dumped periodically to a log file.

It was authored by Leonardo Rochael Almeida, and made possible with developer time generously donated by Nexedi, and with design input from Sébastien Robin and Julien Muchembled.

Installation

Buildout Installation

Add “Products.LongRequestLogger” to the list of eggs of the part that defines your Zope instance.

Configuration

Add (or change) a “<product-config LongRequestLogger>” section of your zope.conf to something like this:

<product-config LongRequestLogger>
    logfile $INSTANCE/log/longrequest.log0.log
    timeout 4
    interval 2
</product-config>

The following variables are recognised:

  • “logfile”: This is a mandatory variable. Its absence means the LongRequestLogger monkey-patch to the publication machinery will not be applied. It should point to a file where you want the long requests to be logged.
  • “timeout”: The amount of seconds after which long requests start being logged. Accepts floating point values and defaults to 2.
  • “interval”: The frequency at which long requests will have their stack trace logged once they have exceeded their ‘timeout’ above. Defaults to 1 and accepts floating point values.

Interpreting results

It’s important to keep in mind a few important facts about the behaviour of Zope2 applications and threads while looking at the results:

  1. Each thread only handles one request at a time.
  2. Slow requests will usually have tracebacks with a common top part and a variable bottom part. The key to the cause of the slowdown in a request will be in the limit of both.

If you’re in a pinch and don’t want to parse the file to rank the slowest URLs for investigation, pick up a time in seconds that’s a multiple of your interval plus the timeout and grep for it. For the default settings, of time-out and interval, you will find log entries for 4 then 6 then 8 seconds, so you can do a grep like:

$ grep -n "Running for 8" longrequest.log

And decide with URLs show up more. Then you can open the log file, go to the line number reported and navigate the tracebacks by searching up and down the file for the same thread id (the number after “Thread” in the reported lines). Then analise the difference between the tracebacks of a single request to get a hint on what this particular request is doing and why it is slowing down.

By doing this for a number of similar requests you will be able to come up with optimisations or a caching strategy.

Changelog

2.0.0 (2015-11-04)

  • Configuration is now done with a “product-config” section in zope.conf, instead of environment variables.
  • Log queries executed by ZMySQLDA.
  • Consolidate stack trace output to a single line if it’s the same as the previous stack trace.
  • Remove the seemly unused mechanism for changing the behaviour at runtime by changing environment variables, like redirecting logging to a different filename, stopping the logging or changing the timeouts. Log rotation still works normally.
  • Stop creating and ending one extra thread per request. Instead, a single monitoring thread is launched at startup.
  • Drop compatibility with Python < 2.6.

1.1.0 (2012-09-10)

  • Some refactoring for code readability.
  • Use a os.pipe() pair and select.select() instead of threading.Condition to signal when the monitor should stop tracing the original thread. This avoids a performance bottleneck in some VMWare installations, which seem not to have good performance for locks in certain conditions.
  • Integrate the logging mechanism with Zope’s signal handling and ZConfig’s rotating file handler so that USR2 signals will cause the long request log to get reopened analogous to the access and event log.

1.0.0 (2010-10-28)

  • Initial release
Release History

Release History

2.0.0

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
Products.LongRequestLogger-2.0.0.tar.gz (9.5 kB) Copy SHA256 Checksum SHA256 Source Nov 4, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting