pyloot 0.0.5

Multiprocessing compatible memory leak debugger inspired by dozer/dowser

PyLoot

PyLoot is a memory leak detector based on Dozer and vprof with added support for process based server workloads.

This project is in active development and may contain bugs or otherwise work in ways not expected or intended.

Installation

$ pip install pyloot

Usage

from pyloot import PyLoot
loot = PyLoot()


"""
Collect objects still in `gc.get_objects` after a call to `gc.collect`.
"""
loot.collect_objects()

"""
Collecting data in a background thread every 30 seconds.

If gevent is detected, gevent.threadpool.spawn is used.
Otherwise, threading.Thread is used.
"""
loot.start()

"""
Stop running the collector background thread.

NOTE: This does not do a "final" collection.
To ensure objects were collected in a short lived execution, call collect_objects().

:param blocking: When true, wait until the thread has died
"""
loot.stop()

"""
Return a WSGI compatible application serving the PyLoot remote backend and
and the website.
:return: ::class::`PyLootServer`
"""
loot.get_wsgi()

Running embedded

Starlette/FastApi/ASGI

from pyloot import PyLoot
from fastapi import FastAPI
from starlette.applications import Starlette
from starlette.middleware.wsgi import WSGIMiddleware

app = FastAPI()  # or Starlette()

pyloot = PyLoot()
app.on_event("startup")(pyloot.start)
app.mount("/_pyloot", WSGIMiddleware(pyloot.get_wsgi()))

Flask/WSGI

from pyloot import PyLoot
from flask import Flask
from werkzeug.middleware.dispatcher import DispatcherMiddleware

app = Flask(__name__)

pyloot = PyLoot()
app.on_before_first_request(pyloot.start)

app = DispatcherMiddleware(app, {
    '/_pyloot': pyloot.get_wsgi()
})

Running in remote mode (multi-process servers)

# Embedded code
from pyloot import PyLoot
...
pyloot = PyLoot(host="127.0.0.1", port=8000)
...

# Start the remote server
$ pyloot --help
usage: pyloot [-h HOST] [-p PORT] [--help]

optional arguments:
-h HOST, --host HOST  Host to listen on. (Default: 0.0.0.0)
-p PORT, --port PORT  Port to listen on. (Default: 8000)
--help                show this help message and exit

Bypass the multiprocessing check

If pyloot detects it is running in a multiprocessing environment with an inmemory backend it will refuse to serve the webpages/requests.

The WSGIMiddleware of starlette sets environ["wsgi.multiprocess"]=True regardless of the server. This can by bypassed with a wrapper use with caution:

pyloot = PyLoot()

def pyloot_wrapper(wsgi_environ, start_response):
    pyloot_environ = wsgi_environ.copy()
    pyloot_environ["wsgi.multiprocess"] = False
    wsgi = pyloot.get_wsgi()
    return wsgi(pyloot_environ, start_response)

app.mount("/_pyloot", WSGIMiddleware(pyloot_wrapper))

Screenshots

View history of object counts by object group:

Modify history page size

Search history page

View objects by group

Modify objects fetch size

Modify objects page size

View an object, its attributes, repr, children, and parents