FlorDB: Log-Forward Metadata Management for AI/ML Training and Evaluation

FlorDB starts with the print and logging output of the scripts you already run as part of model training, and, over time, grows with you into sustained experiment tracking, model evaluation, and some measure of reproducibility. No new schema or service to adopt.

🌻 Why FlorDB?

• Starting from an Existing Project
  Add import flordb as flor to a .py script you run. FlorDB captures the run's print and logging output with each run tied to the code that produced it. You can query these values with flor.io().

• Experiment Tracking with Logging Statements
  flor.log(n, v) records what a run produces: loss, accuracy, anything you'd print. flor.arg(n, v) records what it consumes: learning rate, batch size, random seed, each settable from the command line. You can query these values with flor.dataframe().

• Evaluation: Pull the Model or Push the Code
  Missed a metric? Load a past run's checkpoint in a notebook and measure it, or add the log statement and replay past runs to retrieve it.

• Reproducibility Without Friction
  Every run is versioned via Git, replays reuse the forward run's hyperparameters and seed, and one checkpoint per run is mirrored automatically.

• Keep Run History With Your Project
  Your run history stays local, alongside your code. FlorDB keeps a record of your experiments as you work, with no server to set up or maintain.

Keep using the tools you already work with: Make, Airflow, Slurm, Jupyter, VSCode, or a plain terminal.

📦 Installation

pip install flordb

For contributors or bleeding-edge features:

git clone https://github.com/ucbrise/flor.git
cd flor
pip install -e .

🪵 Already using print and logging? Add one import

Requires a Git repository for automatic versioning.

Add one import to the script you run. The rest of your code stays as it is:

import flordb as flor          # <-- the only new line

for epoch in range(3):
    print(f"epoch {epoch} | loss: {1.0 / (epoch + 2):.4f}")

Your output prints as before. When the run ends, FlorDB commits it and says so; the captured lines are queryable with flor.io().

→ Automatic log capture: channels and turning captured text into real metric columns.

FlorDB commits to its own git branch

Run from main and FlorDB creates and switches to flor.branch (or a numbered variant), keeping auto-commits off your working branches. You stay on that flor branch after the run: subsequent runs accumulate history.

Prefer to name the branch yourself? Create it with a flor. prefix, such as flor.experiment, and FlorDB commits there instead of creating one. Exploring several leads? Give each its own flor. branch.

→ Working on Flor Branches: saving changes, pushing your branch, and bringing code back for review.

🧪 Track Experiments with the Flor API

Use flor.arg to declare inputs, flor.log to record named values, and flor.loop to attach iteration context. Query these records with flor.dataframe().

First Log in 30 Seconds

Requires a Git repository for automatic versioning.

mkdir flor_sandbox
cd flor_sandbox
git init
ipython

import flordb as flor
flor.log("message", "Hello ML World!")

message: Hello ML World!

Run committed successfully.

Retrieve logs anytime:

flor.dataframe("message")

         projid              tstamp filename   source          message
0  flor_sandbox 2025-10-13 18:13:48  ipython  forward  Hello ML World!

Record hyperparameters and per-iteration metrics

Record learning rate and batch size for the run, loss at each training step, and validation accuracy after each epoch:

import flordb as flor

lr = flor.arg("lr", 1e-3)                     # CLI-settable, recorded with the run
batch_size = flor.arg("batch_size", 32)

for epoch in flor.loop("epoch", range(epochs)):
    for x, y in flor.loop("step", trainloader):
        ...
        flor.log("loss", loss.item())
    flor.log("val_acc", validate(net))

    torch.save({"model": net.state_dict()}, "ckpt.pth")   # flor keeps each run's copy

Change hyperparameters from the CLI:

python train.py --kwargs lr=5e-4 batch_size=64

View metrics across runs:

flor.dataframe("lr", "batch_size", "loss")

        projid                     tstamp  filename   source  epoch  step      lr batch_size    loss
0  ml_tutorial 2026-08-13 11:27:06.417615  train.py  forward      0     0  0.0005         64     0.5
1  ml_tutorial 2026-08-13 11:27:06.417615  train.py  forward      0     1  0.0005         64  0.3333
2  ml_tutorial 2026-08-13 11:27:06.417615  train.py  forward      1     0  0.0005         64  0.3333
3  ml_tutorial 2026-08-13 11:27:06.417615  train.py  forward      1     1  0.0005         64    0.25
4  ml_tutorial 2026-08-13 11:27:06.417615  train.py  forward      2     0  0.0005         64    0.25
5  ml_tutorial 2026-08-13 11:27:06.417615  train.py  forward      2     1  0.0005         64     0.2

The epoch and step columns come from the named flor.loop calls. Each row pairs a logged loss with its epoch, step, and the run's lr and batch_size. FlorDB combines them automatically.

→ Experiment tracking: declaring inputs, recording metrics, and naming your loops with the Flor API.

→ Checkpoints: what gets copied, and how replay treats your checkpoint file.

🔍 Evaluate Past Runs

New questions come up after training: a metric you forgot to log, or a bias that only surfaced in production. Load a past run's checkpoint in Jupyter and evaluate the model, or add the flor.log statement to your script and replay past runs to record it.

Pull the model into Jupyter

Load saved models in a notebook to evaluate new metrics and compare past runs.

→ Jupyter walkthrough and comparison notebook.

Replay with hindsight logging

Forgot to log gradient norms? Add the statement to the script now:

flor.log("grad_norm", ...)

python -m flordb replay --apply grad_norm

FlorDB walks the historical versions, splices your new statement into each one, re-executes it from the start, and records the recovered values.

→ Replay: choosing which runs to log, replaying one run on a different device, and replaying from a fresh clone.

📁 What FlorDB Writes

FlorDB stores run records, checkpoints, and its query cache in .flor/. Run records are tracked in Git; checkpoints and the cache stay local. FlorDB never pushes—you choose what to share.

→ Storage: the file layout, what syncs, and how to rebuild the query cache after checkout.

📚 Publications

FlorDB is based on research from UC Berkeley’s RISE Lab continued at Arizona State University.

• Flow with FlorDB: Incremental Context Maintenance for the Machine Learning Lifecycle (CIDR 2025)
• The Management of Context in the ML Lifecycle (UCB Tech Report 2024)
• Hindsight Logging for Model Training (PVLDB 2021)

🛠 License

Apache v2 License — free to use, modify, and distribute.

💡 Get Involved

FlorDB is actively developed. Contributions, issues, and real-world use cases are welcome!

make test        # full suite, including real forward runs and replays
make test-fast   # unit tests only (~1s)

Email: rogarcia@berkeley.edu (or) rolando.garcia@asu.edu