MFLIB Measurement Framework Library

Welcome to the FABRIC Measurement Framework Library. MFLib makes it easy to install monitoring systems to a FABRIC experimenter's slice. The monitoring system makes extensive use of industry standards such as Prometheus, Grafana, Elastic Search and Kibana while adding customized monitoring tools and dashboards for quick setup and visualization.

Documentation Resources

For more information about FABRIC vist fabric-testbed.net

Example Jupyter Notebooks

FABRIC Jupyter Examples GitHub repository contains many examples for using FABRIC from simple slice setup to advanced networking setups. Look for the MFLib section. These notebooks are designed to be easily used on the FABRIC JupyterHub

FABRIC Learn Site

FABRIC Knowledge Base

MFLib Python Package Documentation

Documentation for the package is presented in serveral different forms (and maybe include later in this document):

• ReadTheDocs
• MFLib.pdf in the source code/GitHub.
• Or you may build the documentation from the source code. See Sphinx Documentation later in this document.

MFLib Installation

Instaling via PIP

MFLib may be installed using PIP and PyPI fabrictestbed-mflib

pip install --user fabrictestbed-mflib

To use the REST-backed direct mode, install the optional server dependencies:

pip install --user "fabrictestbed-mflib[rest]"

Installing via Source Code

If you need a development version, clone the git repo, then use pip to install.

git clone https://github.com/fabric-testbed/mflib.git
cd mflib
pip install --user .

To include the REST server dependencies from source:

pip install --user ".[rest]"

Running Without FABlib

The library can now run in two modes:

1. Slice mode: current behavior, where MFLib is constructed with a FABlib slice and can bootstrap topology-aware measurement infrastructure.
2. Direct mode: MFLib is constructed with a REST endpoint for an already reachable measurement node, and service lifecycle calls run through that API instead of FABlib SSH helpers.

Direct mode is intended for operations that already collapse to a single measurement node, such as:

• create()
• update()
• info()
• start()
• stop()
• remove()
• service file upload and download helpers

Topology-aware bootstrap steps still require a FABlib slice today because they depend on slice networks, interface discovery, and fan-out to experiment nodes.

FastAPI Server on the Measurement Node

Run this on the measurement node:

export MFLIB_API_TOKEN="replace-me"
python3 -m mflib.rest_api --host 0.0.0.0 --port 8000

The server intentionally exposes only the node operations MFLib already uses: command execution, file upload/download, directory upload, and basic metadata.

Direct Client Example

from mflib.mflib import MFLib

mfl = MFLib(
	node_api_url="http://meas-node.example:8000",
	node_api_token="replace-me",
	slice_name="my-slice",
)

result = mfl.create("prometheus")
print(result)

Suggested Path Forward

Use a staged split between transport and topology:

1. Keep init() and addMeasNode() in FABlib mode. They are topology-aware and already encode slice bootstrap behavior.
2. Move service lifecycle and file movement through the new node transport seam. This is now supported by either FABlib or the REST adapter.
3. Run the FastAPI service behind an SSH tunnel, VPN, or internal FABRIC-only address space. The API executes shell commands and should not be exposed broadly.
4. If you want fully slice-free bootstrap later, add a second REST service on the control side that supplies topology data and hosts inventory to the measurement node, rather than trying to infer it locally on the node.

Building & Deploying

Spinx Documentation

This package is documented using sphinx. The source directories are already created and populated with reStructuredText ( .rst ) files. The build directories are deleted and/or are not included in the repository,

API documentation can also be found at https://fabrictestbed-mflib.readthedocs.io/.

Build HTML Documents

Install the extra packages required to build API docs: (sphinx, furo theme, and myst-parser for parsing markdown files):

pip install -r docs/requirements.txt

Build the documentation by running the following command from the root directory of the repo.

./create_html_doc.sh

The completed documentation may be accessed by clicking on /docs/build/html/index.html. Note that the HTML docs are not saved to the repository.

Build PDF Document

Latex must be installed. For Debian use: Latex must be installed. For Debian use:

sudo apt install texlive-latex-extra sudo apt install latexmk

Changed to conda install for environments with out sudo access.

conda install -c conda-forge tectonic

Run the bash script to create the MFLIB.pdf documentation. MFLIB.pdf will be placed in the root directory of the repository.

./create_pdf_doc.sh

Note you may just hit return for the ? warnings about .svg files.

Distribution Package

MFLib package is created using Flit Be sure to create and commit the PDF documentation to GitHub before building and publishing to PyPi. The MFLib.pdf is included in the distributition.

To build python package for PyPi run

./create_release.sh

Uploading to PyPI

First test the package by uploading to test.pypi.org then test the install.

flit publish --repository testpypi 

Note that if the package has already been published to testpypi, it cannot be published again until the version is updated. There is not an obvious error for this. Instead you will just get the following error:

requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://test.pypi.org/legacy/

Once install is good, upload to PiPy

flit publish

Note that Flit places a .pypirc file in your home directory if you do not already have one. Flit may also store your password in the keyring which may break if the password is changed. see Flit Controlling package uploads. The password can also be added to the .pypirc file. If password contains % signs it will break the .pypirc file.