Skip to main content

A simple and versatile monitor/profiler for model training loops.

Project description

EpochMonitor ⏱️

PyPI version License: MIT

A simple, lightweight, and versatile profiler for monitoring your machine learning training loops. EpochMonitor helps you track training time, CPU/RAM usage, and NVIDIA GPU memory usage with minimal code changes.

Get crucial insights into your model's performance and resource consumption, all presented with a clean, dynamic progress bar.

[Image of a dynamic terminal progress bar]


Features

  • Easy Integration: Use it as a simple function decorator or a flexible context manager.
  • Comprehensive Metrics: Tracks epoch duration, total training time, CPU utilization, and RAM usage (in % and MB).
  • NVIDIA GPU Support: Automatically detects NVIDIA GPUs and monitors memory usage per epoch.
  • Dynamic Display: Uses tqdm to provide a clean, non-disruptive progress bar that updates in place.
  • Flexible Logging: Automatically saves a detailed history of all metrics to a CSV or JSON file for later analysis.
  • Multi-GPU Aware: Allows you to specify which GPU to monitor on systems with multiple cards.

Installation

You can install EpochMonitor directly from PyPI:

pip install epochmonitor

Usage

As a Decorator (Simplest Method)

Just add @EpochMonitor() on top of your training function.
The monitor object will be injected into your function as a keyword argument.

import time
from epochmonitor import EpochMonitor
from tqdm import tqdm

@EpochMonitor(log_file_prefix="my_model_log", file_format="json")
def train_my_model(epochs, learning_rate, monitor=None):
    for epoch in tqdm(range(epochs), desc="Training Model"):
        monitor.start_epoch()
        print(f"-> Training with lr={learning_rate}...")
        time.sleep(2)  # Simulating work
        monitor.end_epoch()

train_my_model(epochs=5, learning_rate=0.01)

As a Context Manager (More Control)

Use a with statement for explicit control.

import time
from epochmonitor import EpochMonitor
from tqdm import tqdm

def another_training_run(epochs):
    with EpochMonitor(log_file_prefix="context_run_log") as monitor:
        for epoch in tqdm(range(epochs), desc="Training Model"):
            monitor.start_epoch()
            time.sleep(1.5)  # Simulating work
            monitor.end_epoch()

another_training_run(epochs=3)

Listing Available GPUs

If you have multiple GPUs, list them first:

from epochmonitor import EpochMonitor

# Prints all detected NVIDIA GPUs and indices
EpochMonitor.list_gpus()

# Example: Monitor GPU at index 1
# @EpochMonitor(gpu_index=1)
# def train_on_second_gpu(...):
#     ...

Contributing

Contributions are welcome! Whether it's:

Reporting a bug 🐛

Suggesting a feature 💡

Submitting a pull request 📥

Please read the Contributing Guidelines before starting.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

epochmonitor-1.0.0.tar.gz (5.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

epochmonitor-1.0.0-py3-none-any.whl (6.1 kB view details)

Uploaded Python 3

File details

Details for the file epochmonitor-1.0.0.tar.gz.

File metadata

  • Download URL: epochmonitor-1.0.0.tar.gz
  • Upload date:
  • Size: 5.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.8

File hashes

Hashes for epochmonitor-1.0.0.tar.gz
Algorithm Hash digest
SHA256 57e36cf99a634bca6461bf2ad852210885ba6127c8050ddea17ae9e8c4235d76
MD5 10d467dfa054aac31d13849bedbe9824
BLAKE2b-256 b04d3379e7e8fed5e573827605f8f94a130903ce85d75bbeadacea905f658015

See more details on using hashes here.

File details

Details for the file epochmonitor-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: epochmonitor-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 6.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.8

File hashes

Hashes for epochmonitor-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 6851a50eae14f640d80dd24d953535f8bbf20ee2a4475c5c6f9dd02a19d0e234
MD5 42466d89ad7013290d95f572953a9f92
BLAKE2b-256 6249066d6f066036c807ab84dfb5b6f9a25ad44e1a79afb5b1d2e4104faf822a

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page