ml-logger

Logging utility for ML experiments

Why

People use different tools for logging experimental results - Tensorboard, Wandb etc to name a few. Working with different collaborators, I will have to switch my logging tool with each new project. So I made this simple tool that provides a common interface to logging results to different loggers.

Installation

• pip install "mllogger[all]"

If you want to use only the filesystem logger, use pip install "mllogger"

Install from source

• git clone git@github.com:shagunsodhani/ml-logger.git
• cd ml-logger
• pip install ".[all]"

Alternatively, pip install "git+https://git@github.com/shagunsodhani/ml-logger.git@master#egg=ml_logger[all]"

If you want to use only the filesystem logger, use pip install . or pip install "git+https://git@github.com/shagunsodhani/ml-logger.git@master#egg=ml_logger".

Documentation

https://shagunsodhani.github.io/ml-logger

Use

• Make a logbook_config:

  from ml_logger import logbook as ml_logbook
  logbook_config = ml_logbook.make_config(
      logger_dir = <path to write logs>,
      wandb_config = <wandb config or None>,
      tensorboard_config = <tensorboard config or None>,
      mlflow_config = <mlflow config or None>)
  

  The API for make_config can be accessed here.

• Make a LogBook instance:

  logbook = ml_logbook.LogBook(config = logbook_config)
  

• Use the logbook instance:

  log = {
      "epoch": 1,
      "loss": 0.1,
      "accuracy": 0.2
  }
  logbook.write_metric_log(log)
  

  The API for write_metric_log can be accessed here.

Note

• If you are writing to wandb, the log must have a key called step. If your log already captures the step but as a different key (say epoch), you can pass the wandb_key_map argument (set as {epoch: step}). For more details, refer the documentation here.

• If you are writing to mlflow, the log must have a key called step. If your log already captures the step but as a different key (say epoch), you can pass the mlflow_key_map argument (set as {epoch: step}). For more details, refer the documentation here.

• If you are writing to tensorboard, the log must have a key called main_tag or tag which acts as the data Identifier and another key called global_step. These keys are described here. If your log already captures these values but as different key (say mode for main_tag and epoch for global_step), you can pass the tensorboard_key_map argument (set as {mode: main_tag, epoch: global_step}). For more details, refer the documentation here.

Dev Setup

• pip install -e ".[dev]"
• Install pre-commit hooks pre-commit install
• The code is linted using:
  • black
  • flake8
  • mypy
• Tests can be run locally using nox

Acknowledgements

• Config for circleci, pre-commit, mypy etc are borrowed/modified from Hydra