Machine learning dashboard that integrates with tensorflow. Great for monitoring training. Has tools for classification and images
Project description
MLDashboard
Getting Started
This guide assumes you already understand python and tensorflow.
Installation
pip install MLDashboard
Quick Start
To start, you need a dashboard.json config file. This should be in the same directory as your script. Here is an example:
{
"modules":[
[
["LossMetricsGraph", {}],
["LossMetricsNumerical", {}]
],
[
["StatusModule",{}],
["EmptyModule", {}]
]
]
}
NOTE: All code in this demo should be protected by
if __name__ == '__main__':
to prevent multiprocessing conflicts.
The dashboard can easily by added to an existing machine learning project. Import the dashboard as shown.
from MLDashboard.MLDashboardBackend import createDashboard
from MLDashboard.MLCallbacksBackend import DashboardCallbacks, CallbackConfig
from MLDashboard.MLCommunicationBackend import Message, MessageMode
Before training starts, create the dashboard.
#MAKE SURE YOU HAVE A DASHBOARD.JSON FILE IN THE SAME DIRECTORY AS YOUR SCRIPT
dashboardProcess, updatelist, returnlist = createDashboard(config='dashboard.json')
Connect the callbacks to your training.
config = CallbackConfig()
labels = list(range(0,10)) #labels should be customized for the data. This is for mnist number recognition
callback = DashboardCallbacks(updatelist, returnlist, model, x_train, y_train, x_test, y_test, labels, config)
model.fit(x_train, y_train, epochs=10, callbacks=[callback])
After training ends, you can send evaluation stats to the dashboard.
model.evaluate(x_test, y_test, batch_size=128, callbacks=[callback])
To exit the dashboard cleanly, use the following code:
updatelist.append(Message(MessageMode.End, {}))
print("Exiting cleanly...")
dashboardProcess.join()
print("Dashboard exited.")
#This handles any extra data that the dashboard sent, such as save commands
callback.HandleRemaingCommands()
Here is a full example with python code:
import os
os.environ["TF_CPP_MIN_LOG_LEVEL"] = '2' #stops agressive error message printing
import tensorflow as tf
from tensorflow import keras
from MLDashboard.MLDashboardBackend import createDashboard
from MLDashboard.MLCallbacksBackend import DashboardCallbacks, CallbackConfig
from MLDashboard.MLCommunicationBackend import Message, MessageMode
def run():
print("Starting interactive dashboard demo...")
print("Setting up dashboard...")
#Create dashboard and return communication tools (this starts the process)
#MAKE SURE YOU HAVE A DASHBOARD.JSON FILE IN THE SAME DIRECTORY AS YOUR SCRIPT
dashboardProcess, updatelist, returnlist = createDashboard(config='dashboard.json')
print("Loading data...")
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
print("Formatting data...")
x_train = x_train.reshape(-1, 784).astype("float32") / 255.0
x_test = x_test.reshape(-1, 784).astype("float32") / 255.0
print("Sampling data...")
# Limit the train data to 10000 samples
x_train = x_train[:10000]
y_train = y_train[:10000]
# Limit test data to 1000 samples
x_test = x_test[:1000]
y_test = y_test[:1000]
print("Creating model...")
model = keras.Sequential([keras.layers.Dense(128, activation='relu'), keras.layers.Dense(10)])
model.compile(optimizer='adam', metrics=["accuracy"],
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True))
print("Creating callbacks...")
#Callbacks require update and return list for communicating with dashboard
#Model and datasets are useful for sending that data to certain modules
config = CallbackConfig()
labels = list(range(0,10))
callback = DashboardCallbacks(updatelist, returnlist, model, x_train, y_train, x_test, y_test, labels, config)
model.fit(x_train, y_train, epochs=50, callbacks=[callback])
print("Evaluating model...")
#This is connected to the callback so the data is sent to the dashboard
model.evaluate(x_test, y_test, batch_size=128, callbacks=[callback])
updatelist.append(Message(MessageMode.End, {}))
print("Exiting cleanly...")
dashboardProcess.join()
print("Dashboard exited.")
#This handles any extra data that the dashboard sent, such as save commands
callback.HandleRemaingCommands()
if __name__ == '__main__':
run()
Other guides:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for MLDashboard-1.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 39c09796ffeb43c7ff48031f943d641028e380d3fcc49c9c3aa971f80c70a77d |
|
MD5 | bffd8bc412bdbf91a9612ac56d7f300d |
|
BLAKE2b-256 | d6f129386c04aba6be8e9b5627ad7b8d5ac1895bb675233a43ac96a2a6553cf3 |