Software Hut Logger

A package containing the logger demo for Software Hut using Hugging Face 🤗 Transformers Trainer.

Important Notes:

• If you find anything is broken or confusing, please open an issue so that all teams can benefit from any explanations and fixes.
• This module was developed on linux and is untested on Windows. I would recommend using WSL2 if you don't have a linux machine available but Windows might work. I won't be able to support Windows specific issues.
• You won't need to run the training script but if you really want to, linux/WSL2 and CUDA>=12.1 on an NVIDIA GPU are required.

Index

• Installation & Setup: Instructions for setting up the environment.
• Usage: How to use the software after installation.
• Commands: List of available commands and their descriptions.
• Metadata File Contents: Details about the contents of the metadata file.

TODO

• Finish documentation
• Add system monitoring

1. Installation & Setup

# Create a virtual environment
python3 -m venv shl-demo-venv
source shl-demo-venv/bin/activate
pip install software-hut-logger

NOTE: If you want to edit the code or rerun the training script for any reason, let me know and I'll add instructions.

2. Usage

If installed correctly, the package should be available as a command-line tool.

# Both commands are equivalent
shl <command> [ --project-name <project-name> ] [ --experiment-name <experiment-name> ] [ --run-name <run-name> ] [...command-specific-options]

software-hut-logger <command> [ --project-name <project-name> ] [ --experiment-name <experiment-name> ] [ --run-name <run-name> ] [...command-specific-options]

NOTE: Anywhere that "-" is used in an argument name, "_" is also valid.

# argument1 is required
<argument1>

# One of argument1 or argument2 must be specified
<argument1 | argument2>

# argument1 is optional
[argument1]

# argument1 is required and argument2 is optional
<argument2> [argument1]

# If argument1 is specified, then argument2 is required
[argument1 <argument2>]

2.1 Commands

There are three commands you can use but only two that you'll probably need to use.

2.1.1 start-server

Starts a mock endpoint for you to explore uploading runs to. It is currently implemented with an API key -- if you chose to implement authentication in a different way, let me know and I'll add support for that.

shl server <start [--upload-url <upload-url>] [--upload-port <upload-port>] [-q | --quiet] [--api-key <api-key>] [--workers <num-workers>] | stop [--pid-file <pid-file>]>  

Minimal Example:

# Starts the demo-server at http://0.0.0.0:8000
shl server start

# Stops the demo-server -- only needed if you started the server with the `-q` flag
shl server stop

2.1.2 upload-run

Uploads a log file to the specified endpoint.

shl upload-run [--run-dir <run-dir>] [--api-key <api-key>] [--upload-url <upload-url>] [--upload-port <upload-port>]

Minimal Example:

# Uploads example_log to the demo-server at http://0.0.0.0:8000
shl upload-run

2.1.3 build-dataset-example

Builds an example dataset from the English-German portion of the WMT14 dataset. By default, a jsonl that is 1.44GB will be created. If you want something smaller to work with, you can specify a number of samples to save (try ~1000-10,000).

shl build-dataset-example [--save-dir <save-dir>] [--num-samples <num-samples>]

Minimal Example:

shl build-dataset-example

2.1.4 train

🚧 I'll add details here if you find you need to run the training script. 🚧

3. Metadata File Contents

The metadata file contains all of the information about the run.