discordflow 0.3.2

A lightweight MLflow clone that logs ML training metrics, parameters, and artifacts directly to Discord webhooks — with Forum thread support.

DiscordFlow 🚀

The MLflow you already have open on your phone. Log ML training metrics, parameters, and artifacts directly to Discord — no server required.

---

✨ What's in the Box

Feature	Details
📈 Metrics	log_metric() / log_metrics() per step/epoch
⚙️ Params	log_param() / log_params() — logged as blue embeds
🏷️ Tags	set_tag() — arbitrary key-value labels on a run
📁 Artifact upload	Any file up to 25 MB via log_artifact()
📄 Text artifact	log_text() — upload a string as a .txt / .csv attachment
📊 Figure upload	log_figure(fig) — send matplotlib figures as PNG into Discord
💬 Normal channel	start_run() — embeds posted directly in the channel
📋 Forum channel	start_forum_run() — each run gets its own dedicated Discord thread
🔄 Auto-resume	Threads are resumed automatically after Colab restarts
💾 State persistence	save() stores {run_name: thread_id} to JSON
🖥️ System metrics	Per-call, configurable: CPU, RAM, GPU, Disk, Network
⚡ Async logging	ThreadPoolExecutor — Discord calls never block training
❌ Error capture	Exceptions inside with blocks are posted to Discord
🖥️ Dry-run mode	dry_run=True prints to stdout — no real webhook calls

---

📦 Installation

# Core (no system metrics)
pip install discordflow

# + CPU, RAM, Disk, Network metrics
pip install "discordflow[system]"

# + NVIDIA GPU metrics
pip install "discordflow[system,gpu]"

Requires Python ≥ 3.8.

---

⚡ Quickstart

Get a webhook URL from Discord Server Settings → Integrations → Webhooks → New Webhook.

Normal Channel — start_run()

Use when your webhook points to a regular text / announcement channel.

from discordflow import DiscordFlow

dflow = DiscordFlow(WEBHOOK_URL, experiment_name="ResNet_Training")

with dflow.start_run("baseline") as run:
    run.log_params({"lr": 3e-4, "batch_size": 128, "epochs": 10})
    run.set_tag("dataset", "ImageNet")

    for epoch in range(1, 11):
        loss = train_one_epoch(model, ...)
        run.log_metrics(
            {"Train Loss": loss, "Val Acc": eval(model, ...)},
            step=epoch,
            system_metrics=["cpu", "ram"],   # ← hardware stats appended to each embed
        )

    run.log_artifact("best_model.pt")
    run.log_figure(fig, title="Loss Curve")

dflow.finish()  # flush async queue before exit

Forum Channel — start_forum_run()

Use when your webhook points to a Forum channel.
Each run automatically gets its own Discord thread — perfect for comparing many experiments.

dflow = DiscordFlow(FORUM_WEBHOOK_URL, experiment_name="LLM_FineTune")

with dflow.start_forum_run("lora_r16", description="LoRA rank=16 sweep") as run:
    run.log_params({"lora_rank": 16, "lr": 2e-4, "epochs": 3})

    for epoch in range(1, 4):
        run.log_metrics(
            {"Train Loss": ..., "Val Loss": ...},
            step=epoch,
            system_metrics=["cpu", "ram", "gpu"],  # ← GPU stats for Colab/NVIDIA
        )
        run.log_figure(fig, title=f"Epoch {epoch} curve")

# ✅ Summary + elapsed + final metrics auto-posted to the thread

dflow.save()    # ← persist thread IDs so you can resume after a restart
dflow.finish()

⚠️ Wrong channel type? If you call start_run() on a Forum webhook (or vice versa), DiscordFlow will raise a WebhookError with a message telling you exactly which method to switch to.

---

🖥️ Configurable System Metrics

Pass any combination to system_metrics= on log_metrics():

Key	Logged value	Requires
"cpu"	Usage % + clock speed	discordflow[system]
"ram"	Usage % + GB used / total	discordflow[system]
"gpu"	Util % + VRAM used / total, per GPU	discordflow[system,gpu]
"disk"	Usage % + GB used / total	discordflow[system]
"network"	Total MB ↑ sent / ↓ received	discordflow[system]

# Minimal — CPU + RAM only
run.log_metrics({"loss": 0.4}, step=1, system_metrics=["cpu", "ram"])

# Everything
run.log_metrics({"loss": 0.4}, step=1,
    system_metrics=["cpu", "ram", "gpu", "disk", "network"])

---

💾 Colab Restart Recovery

When your Colab runtime reconnects, thread IDs are restored automatically from the saved state file:

# Fresh runtime — state_file is loaded automatically on __init__
dflow = DiscordFlow(FORUM_WEBHOOK_URL, "LLM_FineTune",
                   state_file=".discordflow_state.json")  # default path

# start_forum_run now resumes the existing thread instead of creating a new one
with dflow.start_forum_run("lora_r16") as run:
    ...

Manual override (if you know the thread ID):

dflow.resume_run("lora_r16", thread_id="1234567890123456789")
dflow.save()

ZIP backup (download to PC, re-upload on next Colab session):

from discordflow.colab_utils import export_session, import_session

export_session(dflow)   # ← downloads discordflow_backup.zip to your machine
# --- On a fresh Colab runtime ---
import_session(dflow)   # ← upload the zip to restore all thread IDs

---

🎨 Custom Bot Identity

dflow = DiscordFlow(
    webhook_url     = WEBHOOK_URL,
    experiment_name = "ResNet_Training",
    username        = "TrainBot 🏋️",
    avatar_url      = "https://i.imgur.com/AfFp7pu.png",
)

---

📚 Full API Reference

DiscordFlow(webhook_url, experiment_name, ...)

Parameter	Default	Description
webhook_url	required	Discord webhook URL
experiment_name	"Default Experiment"	Shown in every embed footer
state_file	".discordflow_state.json"	JSON path for thread ID persistence (None to disable)
async_logging	True	Non-blocking background thread for webhook calls
dry_run	False	Print embeds to stdout, no actual HTTP requests
username	"DiscordFlow 🤖"	Bot display name in Discord
avatar_url	None	Bot profile picture URL

Channel Methods

Method	Channel	Returns	Description
start_run(run_name)	Normal	ActiveRun	Start a run, post embeds to channel
start_forum_run(run_name, description)	Forum	ForumActiveRun	Create/resume a forum thread for this run
resume_run(run_name, thread_id)	Forum	—	Manually re-link a run name to an existing thread
save(filepath=None)	Both	—	Persist {run_name: thread_id} to JSON
finish()	Both	—	Flush async queue and shut down executor

Logging Methods (on ActiveRun & ForumActiveRun)

run.log_param("lr", 3e-4)
run.log_params({"lr": 3e-4, "batch": 128, "optimizer": "AdamW"})
run.log_metric("loss", 0.42, step=5)
run.log_metrics({"loss": 0.42, "acc": 0.91}, step=5,
               system_metrics=["cpu", "ram"])
run.set_tag("author", "e27")
run.log_artifact("checkpoint.pt")
run.log_text("epoch,loss\n1,1.0\n2,0.5", filename="loss.csv")
run.log_figure(fig, title="Loss Curve")

Exceptions

Exception	When raised
WebhookError	Discord HTTP error (wrong channel type, network issue)
ArtifactTooLargeError	File exceeds 25 MB Discord limit
RunNotActiveError	Operation attempted with no active run

---

🧪 Local Testing (Dry Run)

dflow = DiscordFlow("ANY_URL", "test", dry_run=True)
with dflow.start_run("test_run") as run:
    run.log_metrics({"loss": 0.42, "acc": 0.91}, step=1)

Output is printed to stdout — no real HTTP calls, no webhook URL needed.

---

🗂️ Project Layout

discordflow/
├── core.py          # DiscordFlow main class
├── run.py           # ActiveRun + ForumActiveRun context managers
├── utils.py         # Colour palette, formatters, collect_system_metrics()
├── exceptions.py    # Custom exception hierarchy
├── colab_utils.py   # export_session / import_session for Colab
└── __init__.py      # Public API surface
colab_demo.py        # Full runnable Colab demo (switchable normal/forum mode)
example.py           # Minimal dry-run example

---

🤝 Contributing

git clone https://github.com/E27-25/discordflow.git
cd discordflow-project
pip install -e ".[dev]"

1. Fork → feature branch → PR.
2. Keep PRs focused; match the existing code style.

---

👤 Author

Watin Promfiy

• GitHub: @E27-25
• Project: github.com/E27-25/discordflow

---

📄 License

MIT © Watin Promfiy