slackker 1.3.0

Python package for monitoring your python script & Model training status in real-time on slack & telegram.

Introducing slackker! :fire:

Watching training metrics is a time killer and addictive. Have you ever found yourself walking back and forth to computer to monitor progress, only to find that the current epoch is not finished yet or that nothing has changed?

When you're in front of your screen, you start to look for patterns in the metrics to judge the progress, this way training spills over into the rest of your live. All the time the models are training, your brain works at 50% at most. So, I made slackker to make your life easy :grin:

slackker is a python package for monitoring your pipeline & ML model training status in real-time on Slack & Telegram. Features:

• Integrate within any .py function/script: You can integrate slackker with any pipeline built in python
• Real-time updates: Get updates on your progress in real-time on Slack & Telegram.
• Exported Plots: Exported plots of training metrics and send it to your Slack channel.
• Customizable: Customize the metrics you want to track and notify.
• Easy to use: Just import the package, setup the slack/telegram and you are good to go.

So now you don't have to sit in front of the machine all the time. You can quickly go and grab coffee :coffee: downstairs or run some errands and still keep tracking the progress while on the move without loosing your peace of mind.

Table of contents :notebook:

• Installation
• Getting started with slackker callbacks
  • Setup slackker
  • Create a Client
  • Use slackker callbacks for any python functions
  • Use slackker callbacks with Keras
  • Use slackker callbacks with Lightning
• Legacy API (deprecated)
• Support
• Citation
• Maintainer

Installation :arrow_down:

• Install slackker from UV is recommended. slackker is compatible with Python >= 3.10 and runs on Linux, MacOS X and Windows.
• Installing slackker in your environment is easy. Just use below command:

uv add slackker

OR

pip install slackker

Getting started with slackker callbacks

Setup slackker

• Slack: How to setup slackker for your slack channel
• Telegram: How to setup slackker for Telegram

Create a Client

All slackker callbacks now use a client object. Create one for your platform first, then pass it to any callback.

from slackker.core import SlackClient, TelegramClient

Slack

client = SlackClient(
    token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
    channel="C04AAB77ABC",
    verbose=0,
)

Telegram

client = TelegramClient(
    token="1234567890:AAAAA_A111BBBBBCCC2DD3eEe44f5GGGgGG",
    verbose=0,
)

Parameters (shared):

Parameter	Type	Default	Description
token	str	required	Slack app / Telegram bot token
channel	str	required (Slack only)	Slack channel ID to receive updates
chat_id	str	None (Telegram only)	Telegram chat ID. Auto-discovered if omitted
verbose	int	0	0 = no logging, 1 = info, 2 = debug

Use slackker callbacks for any python functions

Import

from slackker.core import SlackClient          # or TelegramClient
from slackker.callbacks.simple import SimpleCallback

Create the SimpleCallback object

client = SlackClient(
    token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
    channel="C04AAB77ABC",
)

slackker = SimpleCallback(client)

Wrap your function with the notifier decorator

@slackker.notifier
def your_function():
    return value_1, value_2

The following message will be sent to your channel when the function executes:

Function 'your_function' from Script: 'your_script.py' executed.
Execution time: 5.006 Seconds
Returned 2 outputs:
Output 0:
value_1

Output 1:
value_2

Send a notification with notify()

You can also use slackker.notify() anywhere in your script to send a custom notification:

slackker.notify(
    event="training_complete",
    value_1=arg1,
    value_2=f"This is argument 2 = {arg2}",
    status="completed",
)

The following message will be sent:

Notification: training_complete at 14-10-2024 12:15:54

value_1: arg1
value_2: This is argument 2 = arg2
status: completed

To send a file with the notification, pass the file path using attachment:

slackker.notify(
    event="training_complete",
    attachment="./artifacts/model.ckpt",
    best_val_loss=0.0123,
    epoch=20,
)

Async support

If you are working in an async context, use async_notify() directly:

await slackker.async_notify(event="step_done", accuracy=0.95)

Final code for python function

from slackker.core import SlackClient
from slackker.callbacks.simple import SimpleCallback

# Create client & SimpleCallback object
client = SlackClient(
    token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
    channel="C04AAB77ABC",
)
slackker = SimpleCallback(client)

@slackker.notifier
def your_function():
    return value_1, value_2

your_function()

slackker.notify(
    event="script_finished",
    attachment="./artifacts/summary.txt",
    value_1=value_1,
    value_2=value_2,
)

Use slackker callbacks with Keras

Import slackker for Keras

from slackker.core import SlackClient          # or TelegramClient
from slackker.callbacks.keras import KerasCallback

Create KerasCallback object

client = SlackClient(
    token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
    channel="C04AAB77ABC",
)

slackker = KerasCallback(
    client=client,
    model_name="Keras_NN",
    export="png",
    send_plot=True,
)

or with Telegram:

client = TelegramClient(
    token="1234567890:AAAAA_A111BBBBBCCC2DD3eEe44f5GGGgGG",
)

slackker = KerasCallback(
    client=client,
    model_name="Simple_NN",
    export="png",
    send_plot=True,
)

KerasCallback Parameters:

Parameter	Type	Default	Description
client	BaseClient	required	A SlackClient or TelegramClient instance
model_name	str	required	Name of your model (used in messages & plot titles)
export	str	"png"	Plot export format (eps, jpeg, jpg, pdf, pgf, png, ps, raw, rgba, svg, svgz, tif, tiff)
send_plot	bool	False	If True, sends training/validation plots when training ends

Call slackker object in model.fit()

history = model.fit(
    x_train, y_train,
    epochs=3,
    batch_size=16,
    verbose=1,
    validation_data=(x_val, y_val),
    callbacks=[slackker],
)

Final code for Keras

from slackker.core import SlackClient
from slackker.callbacks.keras import KerasCallback

# Train-Test split
x_train, x_test, y_train, y_test = train_test_split(x, y, train_size=0.8)
x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, train_size=0.8)

# Build keras model
model = Sequential()
model.add(Dense(8, activation='relu', input_shape=(IMG_WIDTH, IMG_HEIGHT, DEPTH)))
model.add(Dense(3, activation='softmax'))
model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])

# Create Client & KerasCallback
client = SlackClient(
    token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
    channel="C04AAB77ABC",
)

slackker = KerasCallback(
    client=client,
    model_name="SampleModel",
    export="png",
    send_plot=True,
)

# Pass slackker to model.fit() callbacks
history = model.fit(
    x_train, y_train,
    epochs=3,
    batch_size=16,
    verbose=1,
    validation_data=(x_val, y_val),
    callbacks=[slackker],
)

Use slackker callbacks with Lightning

Import slackker for Lightning

from slackker.core import SlackClient          # or TelegramClient
from slackker.callbacks.lightning import LightningCallback

Log your metrics to track

Log Training loop metrics

self.log("train_loss", loss, on_epoch=True)
self.log("train_acc", accuracy, on_epoch=True)

Make sure to set on_epoch=True in the training step.

Log Validation loop metrics

self.log("val_loss", loss)
self.log("val_acc", accuracy)

In the validation step on_epoch=True by default.

Create LightningCallback object

client = SlackClient(
    token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
    channel="C04AAB77ABC",
)

slackker = LightningCallback(
    client=client,
    model_name="Lightning NN",
    track_logs=["train_loss", "train_acc", "val_loss", "val_acc"],
    monitor="val_loss",
    export="png",
    send_plot=True,
)

or with Telegram:

client = TelegramClient(
    token="1234567890:AAAAA_A111BBBBBCCC2DD3eEe44f5GGGgGG",
)

slackker = LightningCallback(
    client=client,
    model_name="Lightning NN",
    track_logs=["train_loss", "train_acc", "val_loss", "val_acc"],
    monitor="val_loss",
    export="png",
    send_plot=True,
)

LightningCallback Parameters:

Parameter	Type	Default	Description
client	BaseClient	required	A SlackClient or TelegramClient instance
model_name	str	required	Name of your model (used in messages & plot titles)
track_logs	list[str]	required	List of metrics to track & report each epoch
monitor	str	None	Metric used to determine the best epoch
export	str	"png"	Plot export format (eps, jpeg, jpg, pdf, pgf, png, ps, raw, rgba, svg, svgz, tif, tiff)
send_plot	bool	False	If True, sends training plots when training ends

Call slackker object in Trainer module

trainer = Trainer(max_epochs=2, callbacks=[slackker])

Final code for Lightning

import torch
import torch.nn as nn
from torch.utils.data import DataLoader
import torchvision as tv
import torch.nn.functional as F
from lightning.pytorch import LightningModule, Trainer
from lightning.pytorch.callbacks import ModelCheckpoint

from slackker.core import SlackClient
from slackker.callbacks.lightning import LightningCallback

class LightningModel(LightningModule):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(28*28, 256)
        self.fc2 = nn.Linear(256, 128)
        self.out = nn.Linear(128, 10)

    def forward(self, x):
        batch_size, _, _, _ = x.size()
        x = x.view(batch_size, -1)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        return self.out(x)

    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=1e-3)

    def training_step(self, batch, batch_idx):
        x, y = batch
        y_hat = self.forward(x)
        loss = F.cross_entropy(y_hat, y)
        _, predictions = torch.max(y_hat, dim=1)
        accuracy = torch.sum(predictions == y) / y.shape[0]
        self.log("train_loss", loss, on_epoch=True)
        self.log("train_acc", accuracy, on_epoch=True)
        return loss

    def validation_step(self, batch, batch_idx):
        x, y = batch
        y_hat = self.forward(x)
        loss = F.cross_entropy(y_hat, y)
        _, predictions = torch.max(y_hat, dim=1)
        accuracy = torch.sum(predictions == y) / y.shape[0]
        self.log("val_loss", loss)
        self.log("val_acc", accuracy)
        return loss

train_data = tv.datasets.MNIST(".", train=True, download=True, transform=tv.transforms.ToTensor())
test_data = tv.datasets.MNIST(".", train=False, download=True, transform=tv.transforms.ToTensor())
train_loader = DataLoader(train_data, batch_size=128)
test_loader = DataLoader(test_data, batch_size=128)

model = LightningModel()

# Create Client & LightningCallback
client = SlackClient(
    token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
    channel="C04AAB77ABC",
)

slackker = LightningCallback(
    client=client,
    model_name="Lightning NN",
    track_logs=["train_loss", "train_acc", "val_loss", "val_acc"],
    monitor="val_loss",
    export="png",
    send_plot=True,
)

trainer = Trainer(max_epochs=2, callbacks=[slackker])
trainer.fit(model, train_loader, test_loader)

---

Legacy API (deprecated)

Note: The old Update, SlackUpdate, and TelegramUpdate classes still work but emit a DeprecationWarning. They will be removed in a future release. Please migrate to SimpleCallback and the new client-based API shown above.

Click to expand legacy usage examples

Basic callbacks (legacy)

from slackker.callbacks.basic import SlackUpdate   # or TelegramUpdate

# Slack
slackker = SlackUpdate(
    token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
    channel="C04AAB77ABC",
)

# Telegram
slackker = TelegramUpdate(token="1234567890:AAAAA_A111BBBBBCCC2DD3eEe44f5GGGgGG")

@slackker.notifier
def your_function():
    return value_1, value_2

slackker.notify(event="done", value_1=value_1)

Keras callbacks (legacy)

from slackker.callbacks.keras import SlackUpdate   # or TelegramUpdate

slackker = SlackUpdate(
    token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
    channel="C04AAB77ABC",
    ModelName="Keras_NN",
    export="png",
    SendPlot=True,
)

history = model.fit(..., callbacks=[slackker])

Lightning callbacks (legacy)

from slackker.callbacks.lightning import SlackUpdate   # or TelegramUpdate

slackker = SlackUpdate(
    token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
    channel="C04AAB77ABC",
    ModelName="Lightning NN",
    TrackLogs=["train_loss", "train_acc", "val_loss", "val_acc"],
    monitor="val_loss",
    export="png",
    SendPlot=True,
)

trainer = Trainer(max_epochs=2, callbacks=[slackker])
trainer.fit(model, train_loader, test_loader)

Support :sparkles:

If you get stuck, we’re here to help. The following are the best ways to get assistance working through your issue:

• Use our Github Issue Tracker for reporting bugs or requesting features. Contribution are the best way to keep slackker amazing :muscle:
• If you want to contribute please refer Contributor's Guide for how to contribute in a helpful and collaborative way :innocent:

Citation :page_facing_up:

Please cite slackker in your publications if this is useful for your project/research. Here is an example BibTeX entry:

@misc{siddheshgunjal2023slackker,
  title={slackker},
  author={Siddhesh Gunjal},
  year={2023},
  howpublished={\url{https://github.com/siddheshgunjal/slackker}},
}

Maintainer :sunglasses: