A simple and versatile monitor/profiler for model training loops.
Project description
EpochMonitor ⏱️
A simple, lightweight, and versatile profiler for monitoring your machine learning training loops. EpochMonitor helps you track training time, CPU/RAM usage, and NVIDIA GPU memory usage with minimal code changes.
Get crucial insights into your model's performance and resource consumption, all presented with a clean, dynamic progress bar.
[Image of a dynamic terminal progress bar]
Features
- Easy Integration: Use it as a simple function decorator or a flexible context manager.
- Comprehensive Metrics: Tracks epoch duration, total training time, CPU utilization, and RAM usage (in % and MB).
- NVIDIA GPU Support: Automatically detects NVIDIA GPUs and monitors memory usage per epoch.
- Dynamic Display: Uses
tqdmto provide a clean, non-disruptive progress bar that updates in place. - Flexible Logging: Automatically saves a detailed history of all metrics to a CSV or JSON file for later analysis.
- Multi-GPU Aware: Allows you to specify which GPU to monitor on systems with multiple cards.
Installation
You can install EpochMonitor directly from PyPI:
pip install epochmonitor
Usage
As a Decorator (Simplest Method)
Just add @EpochMonitor() on top of your training function.
The monitor object will be injected into your function as a keyword argument.
import time
from epochmonitor import EpochMonitor
from tqdm import tqdm
@EpochMonitor(log_file_prefix="my_model_log", file_format="json")
def train_my_model(epochs, learning_rate, monitor=None):
for epoch in tqdm(range(epochs), desc="Training Model"):
monitor.start_epoch()
print(f"-> Training with lr={learning_rate}...")
time.sleep(2) # Simulating work
monitor.end_epoch()
train_my_model(epochs=5, learning_rate=0.01)
As a Context Manager (More Control)
Use a with statement for explicit control.
import time
from epochmonitor import EpochMonitor
from tqdm import tqdm
def another_training_run(epochs):
with EpochMonitor(log_file_prefix="context_run_log") as monitor:
for epoch in tqdm(range(epochs), desc="Training Model"):
monitor.start_epoch()
time.sleep(1.5) # Simulating work
monitor.end_epoch()
another_training_run(epochs=3)
Listing Available GPUs
If you have multiple GPUs, list them first:
from epochmonitor import EpochMonitor
# Prints all detected NVIDIA GPUs and indices
EpochMonitor.list_gpus()
# Example: Monitor GPU at index 1
# @EpochMonitor(gpu_index=1)
# def train_on_second_gpu(...):
# ...
Contributing
Contributions are welcome! Whether it's:
Reporting a bug 🐛
Suggesting a feature 💡
Submitting a pull request 📥
Please read the Contributing Guidelines before starting.
License
This project is licensed under the MIT License. See the LICENSE file for details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file epochmonitor-1.0.0.tar.gz.
File metadata
- Download URL: epochmonitor-1.0.0.tar.gz
- Upload date:
- Size: 5.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.11.8
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
57e36cf99a634bca6461bf2ad852210885ba6127c8050ddea17ae9e8c4235d76
|
|
| MD5 |
10d467dfa054aac31d13849bedbe9824
|
|
| BLAKE2b-256 |
b04d3379e7e8fed5e573827605f8f94a130903ce85d75bbeadacea905f658015
|
File details
Details for the file epochmonitor-1.0.0-py3-none-any.whl.
File metadata
- Download URL: epochmonitor-1.0.0-py3-none-any.whl
- Upload date:
- Size: 6.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.11.8
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6851a50eae14f640d80dd24d953535f8bbf20ee2a4475c5c6f9dd02a19d0e234
|
|
| MD5 |
42466d89ad7013290d95f572953a9f92
|
|
| BLAKE2b-256 |
6249066d6f066036c807ab84dfb5b6f9a25ad44e1a79afb5b1d2e4104faf822a
|