Skip to main content

pymemtrace provides tools for tracking and understanding Python memory usage at different levels, at different granularities and with different runtime costs.

Full documentation: https://pymemtrace.readthedocs.io

pymemtrace Tools

The tools provided by pymemtrace are:

process_tree

A command line tool that shows the total memory usage of a process and its child processes at regular time intervals. It can log this data to a JSON file for later analysis. See some process_tree examples

process

A very lightweight way of logging the total memory usage of a single process at regular time intervals. It can plot memory over time with plotting programs such as gnuplot. See some process examples

cPyMemTrace

This module, written in C, provides real time logging of Python and C actions:

  • pymemtrace.cPyMemTrace.Profile is a memory tracer written in C that can report total memory usage for every function call/return for both C and Python sections. This is more suitable for logging C code, for example Python’s C extensions.

  • pymemtrace.cPyMemTrace.Trace is a memory tracer written in C that can report total memory usage for every function call/return/line for Python sections. This is more suitable for logging pure Python code.

  • pymemtrace.cPyMemTrace.ReferenceTracing can report every object allocation and de-allocation with Reference Tracing. This is quite invasive but the API allows this to filter out most of the noise or target specific types of interest. (Python 3.13+ only).

See some cPyMemTrace examples and a technical note on cPyMemTrace.

DTrace

There are a number of D scripts that can trace the low level malloc() and free() system calls and report how much memory was allocated and by whom. See some DTrace examples and a technical note on DTrace.

trace_malloc

A convenience wrapper around the Python standard library tracemalloc module. This can report Python memory usage by module and line compensating for the cost of tracemalloc. This can take memory snapshots before and after code blocks and show the change on memory caused by that code. See some trace_malloc examples

debug_malloc_stats

Awrapper around the sys._debugmallocstats function that can take snapshots of memory before and after code execution and report the significant differences of the Python small object allocator. See some debug_malloc_stats examples

Tool Characteristics

Each tool can be characterised by:

  • Memory Granularity: In how much detail is a memory change is observed. An example of coarse memory granularity is measuring the Resident Set Size (RSS) which is normally in chunks of 4096 bytes. An example of fine memory granularity is recording every malloc() and free().

  • Execution Granularity: In how much code detail is the memory change observed. An example of coarse execution granularity is measuring the memory usage every second. An example of fine execution granularity is recording the memory usage every Python line.

  • Memory Cost: How much extra memory the tool needs.

  • Execution Cost: How much the execution time is increased.

Clearly there are trade-offs between these depending on the problem you are trying to solve.

Firstly granularity:

Tool Granularity

Tool

Memory Granularity

Execution Granularity

process_tree

RSS.

Regular time intervals.

process

RSS.

Regular time intervals.

cPyMemTrace.Profile

RSS.

Per Python line, Python function and return. C function call and return.

cPyMemTrace.Trace

RSS.

Per Python line, Python function and return. Python Opcode and exception.

cPyMemTrace.ReferenceTracing

RSS.

Every object allocation/de-allocation.

DTrace

Every malloc() and free().

Per function call and return.

trace_malloc

Every Python object.

Per Python line, per function call.

debug_malloc_stats

Python memory pool.

Snapshots the CPython memory pool either side of a block of code.

Secondly cost:

Tool Cost

Tool

Memory Cost

Execution Cost

process_tree

Near zero.

Near zero.

process

Near zero.

Near zero.

cPyMemTrace.Profile

Near zero.

10x to 40x.

cPyMemTrace.Trace

Near zero.

20x to 60x.

cPyMemTrace.ReferenceTracing

Near zero.

2x to 80x.

DTrace

Minimal.

90x to 100x.

trace_malloc

Significant but compensated.

900x for small objects, 6x for large objects.

debug_malloc_stats

Minimal.

+2000x for small objects, 12x for large objects.

Installation

To install pymemtrace, run this command in your terminal:

$ pip install pymemtrace

Licence

Python memory tracing.

Credits

Phil Smith (AHL) with whom a casual lunch time chat lead to the creation of an earlier, but quite different implementation, of cPyMemTrace in pure Python.

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

History

0.6.3 (2026-07-27)

  • Replace __FILE_NAME__ with __FILE__ for GCC compatibility. Fixes issue #7

  • Supported Python versions are: 3.9, 3.10, 3.11, 3.12, 3.13, 3.14, 3.15

  • Development Status :: 5 - Production/Stable

0.6.2 (2026-07-27)

  • Fixes issue #6

  • Initial support and design for adding thread safety. Not yet implemented.

  • Remove support for Python 3.8 (it will probably still work though).

  • Supported Python versions are: 3.9, 3.10, 3.11, 3.12, 3.13, 3.14, 3.15

  • Development Status :: 5 - Production/Stable

0.6.1 (2026-06-24)

  • Fixes: #5 (macros setting for strdup()).

  • Some debug improvements.

  • Added magic numbers to opaque data structures.

  • Add gnuplot output of live object count.

  • Documentation improvements.

  • Supported Python versions are: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13, 3.14, 3.15

  • Development Status :: 5 - Production/Stable

0.6.0 (2026-05-19)

  • Add a hash table of live type counts to Reference Tracing. This can be accessed from Python as a dictionary.

  • Fix Reference tracing example for build_all.sh.

  • Add module level functions to write to the Profile/Trace/Reference Trace log files.

  • Add some asserts and clean up test_cpyreftraceexample.py.

  • Add Reference Tracing minimal example in C, a Python module and tests.

  • Add Doxygen documentation.

  • Reorganise debug code.

  • Add basic example of Reference Tracing in C to debug code.

  • Supported Python versions are: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13, 3.14, 3.15

  • Development Status :: 5 - Production/Stable

0.5.0 (2026-05-12)

  • Add two console entry points in the distro: pymemtrace_ref_trace_analyse and pymemtrace_dtrace_log_analyse

  • Make option names for ref_trace_analyse.py more intuitive.

  • Fix issues with initialising the datatime capsule.

  • Minor change to the Reference Tracing log file format.

  • ref_trace_analyse.py can now merge log files.

  • Add the ability to get reference counts from arbitrary addresses.

  • Added documentation.

  • Supported Python versions are: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13, 3.14, 3.15

  • Development Status :: 5 - Production/Stable

0.4.1 (2026-05-03)

  • Add example of using Reference Tracing to detect memory leaks.

0.4.0 (2026-04-18)

  • cPyMemTrace:
    • User filtering of Reference Tracing events to include/exclude specific events.

    • Reference Tracing runs the Garbage Collector on exit to make the log more accurate.

    • Much more reliable processing of Reference Tracing events.

    • Fix the issue with Reference Tracing where handling type “frame” and “code” was causing pytest and CPython runtime assert failures.

    • Write profile/trace/reference tracing context switches to the appropriate log file.

    • Add cPyMemTrace.ReferenceTracingSimple as an example. Includes documentation and test code.

    • Add suspend() and resume() methods for Reference Tracing.

    • Document pytest issues with Reference Tracing (now historical information).

  • General documentation improvements, now around 140 PDF A4 pages.

  • Add Doxygen documentation of the C source.

  • Supported Python versions are: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13, 3.14, 3.15

  • Development Status :: 5 - Production/Stable

0.3.1 (2026-03-23)

  • pymemtrace:
    • Add decorator for pymemtrace.process.

    • pymemtrace.process can now summarise JSON in the log to stdout.

  • cPyMemTrace:
    • Add decorators for Python functions for Profile, Trace and ReferenceTracing.

    • Fix a SIGSEGV when bad keyword arguments were passed to cPyMemTrace.ReferenceTracing.

0.3.0 (2026-03-19)

  • Add process-tree.py for logging a process and its children.

  • cPyMemTrace:
    • Add Reference Tracing (Python 3.13+) that can record every object allocation or de-allocation.

    • Add an option to log to a specific file.

    • Add an API write_message_to_log() to inject text into the log file.

    • Better structure of the log file format.

    • Define the log file format.

    • Add debug exploration code with debug_cPyMemtrace().

    • Fix stacking pop() issue with trace/profile functions with linked list of tTraceFileWrapperLinkedList.

  • Add support for Python 3.14

  • Remove support for Python 3.7

  • Supported Python versions are: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13, 3.14

  • Development Status :: 5 - Production/Stable

0.2.0 (2024-11-17)

  • cPyMemTrace:
    • Add P/T, stack depth and python version to log file name, example: "20241107_195847_62264_P_0_PY3.13.0b3.log"

    • Add stacking of trace/profile functions with linked list of tTraceFileWrapperLinkedList.

    • Add an option to log to a specific file.

    • Add an API write_to_log() to inject text into the log file.

    • Add an optional message to the log file in cPyMemTrace.

    • Add Python API to get log file being written to by cPyMemTrace.

    • Bug fixes in cPyMemTrace.c

    • Safety fix for file path name lengths.

    • Fix for log files where '#' was being concatenated.

0.1.7 (2024-09-12)

  • Minor fix for a single test.

0.1.6 (2024-09-11)

  • Add support for Python versions 3.12, 3.13. Now supports Python versions 3.7, 3.8, 3.9, 3.10, 3.11, 3.12, 3.13.

0.1.5 (2023-06-21)

  • Add support for Python versions 3.10, 3.11. Now supports Python versions 3.7, 3.8, 3.9, 3.10, 3.11.

0.1.4 (2022-03-19)

  • Fix Linux build.

0.1.3 (2022-03-17)

  • Fix some tests.

0.1.2 (2022-03-17)

  • Fix source distribution that had missing headers.

0.1.1 (2020-11-17)

  • Add cPyMemTrace the C level profiler.

  • Add DTrace scripts for low level tracing.

  • Add debug_malloc_stats the wrapper around sys._debugmallocstats.

  • Add process.py from the TotalDepth project.

  • Add redirect_stdout for debug_malloc_stats.

  • Add trace_malloc, a wrapper around the tracemalloc module.

  • Includes extensive documentation and performance measurement.

  • First release on PyPI.

0.1.0 (2017-12-04)

  • Initial idea and implementation, never released.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pymemtrace-0.6.3.tar.gz (8.5 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

pymemtrace-0.6.3-cp315-cp315-macosx_10_15_universal2.whl (183.9 kB view details)

Uploaded CPython 3.15macOS 10.15+ universal2 (ARM64, x86-64)

pymemtrace-0.6.3-cp314-cp314-macosx_10_15_universal2.whl (183.8 kB view details)

Uploaded CPython 3.14macOS 10.15+ universal2 (ARM64, x86-64)

pymemtrace-0.6.3-cp313-cp313-macosx_10_13_universal2.whl (183.3 kB view details)

Uploaded CPython 3.13macOS 10.13+ universal2 (ARM64, x86-64)

pymemtrace-0.6.3-cp312-cp312-macosx_10_13_universal2.whl (159.9 kB view details)

Uploaded CPython 3.12macOS 10.13+ universal2 (ARM64, x86-64)

pymemtrace-0.6.3-cp311-cp311-macosx_14_0_x86_64.whl (140.4 kB view details)

Uploaded CPython 3.11macOS 14.0+ x86-64

pymemtrace-0.6.3-cp310-cp310-macosx_14_0_x86_64.whl (140.3 kB view details)

Uploaded CPython 3.10macOS 14.0+ x86-64

pymemtrace-0.6.3-cp39-cp39-macosx_14_0_x86_64.whl (140.3 kB view details)

Uploaded CPython 3.9macOS 14.0+ x86-64

File details

Details for the file pymemtrace-0.6.3.tar.gz.

File metadata

  • Download URL: pymemtrace-0.6.3.tar.gz
  • Upload date:
  • Size: 8.5 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.13

File hashes

Hashes for pymemtrace-0.6.3.tar.gz
Algorithm Hash digest
SHA256 02a18593b02150de3210522c94c3d393c5c6954f701f6bf3a3ebe3831a3d0b23
MD5 b4dccb7947c966f7967983c59ce89b1a
BLAKE2b-256 08721d72c6646a98d0c72b4ce1502ea1d44aedc40e4137553fb169dabbbbd435

See more details on using hashes here.

File details

Details for the file pymemtrace-0.6.3-cp315-cp315-macosx_10_15_universal2.whl.

File metadata

File hashes

Hashes for pymemtrace-0.6.3-cp315-cp315-macosx_10_15_universal2.whl
Algorithm Hash digest
SHA256 349ffa1b91caad974e6afa42d0493c3ff328bb012be1515a5c2663084360e19e
MD5 3f111592282654cb7ee916de0a82e100
BLAKE2b-256 c88409caa7fc4f322ba12a86c6f561dfadc2a02191ef2185c0f7e18686e0585f

See more details on using hashes here.

File details

Details for the file pymemtrace-0.6.3-cp314-cp314-macosx_10_15_universal2.whl.

File metadata

File hashes

Hashes for pymemtrace-0.6.3-cp314-cp314-macosx_10_15_universal2.whl
Algorithm Hash digest
SHA256 0b7bf38ce0817fe91e1ae77ea470bf74d0319105cc5c91dc943c814f6a359041
MD5 8ecef91b9acceeefd245ddc66b2b6c30
BLAKE2b-256 3aea32458893ab5e3ed65cc704890bdffb6985455203cde92c481a5473592989

See more details on using hashes here.

File details

Details for the file pymemtrace-0.6.3-cp313-cp313-macosx_10_13_universal2.whl.

File metadata

File hashes

Hashes for pymemtrace-0.6.3-cp313-cp313-macosx_10_13_universal2.whl
Algorithm Hash digest
SHA256 d9ccbdcc43f26698c161a4f170e91a3919871c9c826112a3b300ea293976fc5b
MD5 02d65f95b64b14f6d68db50749256385
BLAKE2b-256 a6acd3ed22f7b8a9d20c08fcdf61963890f510f6645118844a8b18162722e1a9

See more details on using hashes here.

File details

Details for the file pymemtrace-0.6.3-cp312-cp312-macosx_10_13_universal2.whl.

File metadata

File hashes

Hashes for pymemtrace-0.6.3-cp312-cp312-macosx_10_13_universal2.whl
Algorithm Hash digest
SHA256 31f3d6c91fd61e2376b95b75235fb7f70217d4928b5591ef494c16b02d25449b
MD5 bb306e8f48960540008b36fcbfc0a82c
BLAKE2b-256 0de10ff51d0647bcf0ef0a4db6ba884012d50e1985fa7d280523856949b80c82

See more details on using hashes here.

File details

Details for the file pymemtrace-0.6.3-cp311-cp311-macosx_14_0_x86_64.whl.

File metadata

File hashes

Hashes for pymemtrace-0.6.3-cp311-cp311-macosx_14_0_x86_64.whl
Algorithm Hash digest
SHA256 6b8b2f9e20888a1b00d00b6d243da10fe23cc2f2d823611ef3108c64e8a0c3e8
MD5 a6bc5f4ae835fbbc4227dfbacbfcdd08
BLAKE2b-256 50051a0f2c0a6da95c13dcc9da720989a951f0392094994c9ff8aa6e3d20398b

See more details on using hashes here.

File details

Details for the file pymemtrace-0.6.3-cp310-cp310-macosx_14_0_x86_64.whl.

File metadata

File hashes

Hashes for pymemtrace-0.6.3-cp310-cp310-macosx_14_0_x86_64.whl
Algorithm Hash digest
SHA256 a79162782103dd193bb2f2528b105ce06611bb4d770186630f04db785afa20d5
MD5 9c06451fe7b3c655cb5a82a2cfe78ac8
BLAKE2b-256 15e1e2f5e7524d2887540310f97b7eddcd89b027ba143fd3865c4e1232d371b2

See more details on using hashes here.

File details

Details for the file pymemtrace-0.6.3-cp39-cp39-macosx_14_0_x86_64.whl.

File metadata

File hashes

Hashes for pymemtrace-0.6.3-cp39-cp39-macosx_14_0_x86_64.whl
Algorithm Hash digest
SHA256 0f98b461b28592cd96cfbe938798665304ff4882fe40f5ae5906cae995d92ef6
MD5 ad9d2d2a992cfa670bebd4f6a807675d
BLAKE2b-256 dd16c01a15df374d66026007e546dc387f8eb4465e492482b871118683a4d1d2

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page