Murakami

Murakami is a tool for creating an automated internet measurement service, running in a Docker container. A Murakami measurement container can be configured to automatically run supported tests four times a day using a randomized schedule, and export each test result to a local storage device, to one or more remote servers via SCP, or to a Google Cloud Storage bucket. Results are saved as individual files in JSON new line format (.jsonl).

Supported Measurement Tests

• M-Lab Network Diagnostic Tool (NDT) - both ndt5 and ndt7 protocols, using the libndt client library
• Neubot DASH - Dynamic Adaptive Streaming over HTTP, hosted by M-Lab
• speedtest-cli - Command line interface for testing internet bandwidth using speedtest.net

Murakami Deployment Scenarios

Murakami supports three types of container deployment scenarios on supported systems:

• Standalone: a single Murakami device, configured on-device
• Local Managed: one or more Murakami devices in a single network, configured individually on-device, or using a Mozilla WebThings Gateway on the same network, via the M-Lab WebThings Murakami Add-on
• Balena Cloud: one or more Murakami devices, configured and managed by the Balena Cloud IoT platform

Supported Operating Systems

Murakami supports Linux operating systems like Ubuntu, Debian, etc. Windows is not supported. Mac OS is supportable as a standalone device, but is not completely compatible with the Webthings interface. Balena cloud has not beed tested on Mac OS.

TO DO: test deployment on MacOS

Murakami Configurations and Customization

A Murkami container can be configured flexibly depending on the deployment scenario. If you simply run a Murakami container using M-Lab's pre-built images on Dockerhub, by default all tests are configured to run four times daily at randomized intervals, but no data exporters enabled. This section focuses on which options can be configured using the file murakami.toml OR using environment variables. Examples are provided in later sections of this document.

The Murakami container can be customized using:

• environment variables on the command line when running docker (Standalone, Local Managed)
• environment variables in a file, passed on the command line when running docker (Standalone, Local Managed)
• customizing the murakami.toml configuration file (Standalone, Local Managed, Balena Cloud)
• environment variables in your Balena Cloud project, or per device (Balena Cloud)

The table below summarizes the options you can configure in murakami.toml and how to format the options as command line variables at the command line.

Multiple exporters of any type are supported as well. For example if you wanted to define two different SCP servers or GCS storage buckets where data should be exported, the config file exporters section might look like this:

[exporters]

  [exporters.gcs1]
  type = "gcs"
  enabled = true
  target = "gs://murakami-storage-bucket-archive/"
  service_account = "murakami-test-gcs@mlab-sandbox.iam.gserviceaccount.com"
  key = "/murakami/keys/murakami-gcs-serviceaccount.json"

  [exporters.gcs2]
  type = "gcs"
  enabled = true
  target = "gs://murakami-storage-bucket-access/"
  service_account = "murakami-test-gcs@mlab-sandbox.iam.gserviceaccount.com"
  key = "/murakami/keys/murakami-gcs-serviceaccount.json"

OR as environment variables:

MURAKAMI_EXPORTERS_GCS1_TYPE = "gcs"
MURAKAMI_EXPORTERS_GCS1_ENABLED = "true"
MURAKAMI_EXPORTERS_GCS1_TARGET = "gs://murakami-storage-bucket-archive/"
MURAKAMI_EXPORTERS_GCS1_SERVICE_ACCOUNT = "murakami-test-gcs@mlab-sandbox.iam.gserviceaccount.com"
MURAKAMI_EXPORTERS_GCS1_KEY = "/murakami/keys/murakami-gcs-serviceaccount.json"

MURAKAMI_EXPORTERS_GCS2_TYPE = "gcs"
MURAKAMI_EXPORTERS_GCS2_ENABLED = "true"
MURAKAMI_EXPORTERS_GCS2_TARGET = "gs://murakami-storage-bucket-access/"
MURAKAMI_EXPORTERS_GCS2_SERVICE_ACCOUNT = "murakami-test-gcs@mlab-sandbox.iam.gserviceaccount.com"
MURAKAMI_EXPORTERS_GCS2_KEY = "/murakami/keys/murakami-gcs-serviceaccount.json"

M-Lab Supported Dockerhub Images and Tags

TO DO Describe how we structure tags for different system architectures here.