Skip to main content

Memory Time Tracker

Pypi project Pypi total project downloads

Memory time tracker is a simple python tool to track the memory and time requirements of software across both very brief (milliseconds) and large (days) time and memory requirements through adaptative log resolution.

The reports are CSV files with two columns, the first one with the time delta and the second one with the required RAM.

Tracking upwards to crash

This package handles gracefully also use cases where the tracked software dies because of OOM or generally crashes by adding a 0,0 as the last line of the CSV document it produces when the execution finishes nominally while adding a -1,-1 when the execution finishes with a detectable exception. When there are crashes not detectable through exceptions, such as machine freezes because of OOM, kernel panics or other things, neither 0,0``nor ``1,1 are (inevitably) written at the end of the CSV.

To help distinguish the different possible completion statuses, we have prepared three methods:

  • has_completed_successfully to detect whether the execution has been completed without hiccups.

  • has_crashed_gracefully to detect crashes that raised “normal” exceptions.

  • has_crashed_ungracefully to detect crashes that did not raise “normal” exceptions, such as OOM and core dumps.

See more below in the Examples section.

Requirements

Please do note that this package makes use of proc/meminfo, so it is strictly compatible only with Linux systems.

Like most tracker systems, this one works best if there is a limited amount of noise in the system. Do not run other software while running the benchmark, or your results may be skewed.

Installing package

As usual, to install this package from Pypi, just run:

pip install memory_time_tracker

Usage example

You can use this package to track the execution of a given method as follows:

from memory_time_tracker import Tracker, has_completed_successfully, has_crashed_gracefully, has_crashed_ungracefully, plot_reports
from time import sleep
import numpy as np

def example_function():
    """Small example of function that takes 2 seconds."""
    arrays = []
    for _ in range(10):
        arrays.append(np.zeros((10000, 10000)))
        sleep(0.2)

def example_function_which_crashes():
    """Small example of function that takes 2 seconds and crashes."""
    arrays = []
    for _ in range(20):
        arrays.append(np.zeros((10000, 5000)))
        sleep(0.1)
    raise ValueError("Argh! I'm crashig!")

# The path where we will store the log
path1 = "/tmp/tracker_example.csv"
# The path where we will store the log with the crash
path2 = "/tmp/tracker_example_with_crash.csv"

# Create the tracker context
with Tracker(path1):
    example_function()

# Wait between tracking to allow for memory to free
sleep(20)

# Create the tracker context to handle crashable libraries
try:
    with Tracker(path2, verbose=True):
        example_function_which_crashes()
except Exception:
    pass

print(
    "Successful: ", has_completed_successfully(path1),
    "Crashed gracefully: ", has_crashed_gracefully(path1),
    "Crashed ungracefully: ", has_crashed_ungracefully(path1)
)
# Successful:  True Crashed gracefully:  False Crashed ungracefully:  False

print(
    "Successful: ", has_completed_successfully(path2),
    "Crashed gracefully: ", has_crashed_gracefully(path2),
    "Crashed ungracefully: ", has_crashed_ungracefully(path2)
)
# Successful:  False Crashed gracefully:  True Crashed ungracefully:  False

plot_reports([path1, path2])

You can run it on Colab here.

The above example should generate a picture such as this one:

Alternative text

Note that there is some noise in the RAM and time measurements as it was executed on COLAB.

Authors and License

This package was developed by Luca Cappelletti and Tommaso Fontana and is released under MIT License.

Metadata

Release files for memory-time-tracker 0.0.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for memory-time-tracker 0.0.5
File Size Uploaded Provenance Actions
memory_time_tracker-0.0.5.tar.gz 8.5 kB Without provenance Details for memory_time_tracker-0.0.5.tar.gz

Release files / memory_time_tracker-0.0.5.tar.gz

Download URL memory_time_tracker-0.0.5.tar.gz
Size 8.5 kB
Tags Source
SHA-256 checksum
How to use checksums
b9c5f72d565bab1f3fec3b5f62e91fefe7242c001fd76b9278e557a426adb853
BLAKE2b-256 checksum
How to use checksums
943bf7c3162002089b9e11726dc08b8a1afcaca5fbda6a5603b330f72e658dc5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.8.0 pkginfo/1.8.2 readme-renderer/34.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.9 tqdm/4.63.1 importlib-metadata/4.11.3 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.7.9

Security for memory-time-tracker 0.0.5

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page