dftracer-utils

Tools for reading, indexing, querying and analyzing DFTracer traces: a Python query API over a SIMD columnar DataFrame engine, a plugin ABI for analyses that ride the scan, and the CLI utilities around them.

Quickstart

pip install dftracer-utils

from dftracer.utils import TraceViewer

view = TraceViewer("traces/")            # a directory of .pfw / .pfw.gz files
df = (
    view.filter('cat == "POSIX"')
        .group_by("name")
        .agg("count", "sum:dur", "max:dur")
        .collect()                        # runs the plan, returns a DataFrame
)
print(df.sort_values("sum_dur", ascending=False).head(3).to_pandas())

    name  count  sum_dur  max_dur
0  pread     16      280       22
1   read     13      244       25
2  fread      8      156       24

The frame is the engine's own DataFrame: the pandas and polars spellings work on it (import dftracer.utils.pandas as pd), and .to_pandas() / .to_arrow() hand the buffers over without a copy.

Documentation

Full documentation is available at Read the Docs:

• Getting started: install, first query, first pipeline, first plugin.
• Guides: task recipes, the pandas and polars surfaces, plans, joins, the memory budget, performance.
• API reference and the C ABI.
• Plugins: analyses that ride the fused scan.

To build documentation locally:

pip install .
cd docs
pip install -r requirements.txt
make html

See docs/README.md for detailed documentation building instructions.

Building

Prerequisites

• CMake 3.20 or higher
• C++20 compatible compiler (GCC 11+, Clang 14+)
• zlib development library
• pkg-config

Build

mkdir build && cd build
cmake ..
make

Installation

mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=<LOCATION>
make
make install

Developers Guide

Please see Developers Guide for more information how to test, run coverage, etc.

Citation

If you use this software, please cite:

Ray A. O. Sinurat, William Nixon, Haryadi S. Gunawi, Nikoli Dryden, and Hariharan Devarajan. 2026. HORATIO: Bridging Management and Analysis of Traces at Scale. In The International Conference on Scalable Scientific Data Management 2026 (SSDBM 2026), August 11-13, 2026, San Diego, CA, USA. ACM, New York, NY, USA. doi:10.1145/3828820.3828825

@inproceedings{sinurat2026horatio,
  author    = {Sinurat, Ray A. O. and Nixon, William and Gunawi, Haryadi S. and Dryden, Nikoli and Devarajan, Hariharan},
  title     = {HORATIO: Bridging Management and Analysis of Traces at Scale},
  year      = {2026},
  isbn      = {979-8-4007-2708-5},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  doi       = {10.1145/3828820.3828825},
  booktitle = {The International Conference on Scalable Scientific Data Management 2026 (SSDBM 2026)},
  location  = {San Diego, CA, USA},
  series    = {SSDBM 2026},
}

License

dftracer-utils is distributed under the terms of the MIT license. All new contributions must be made under this license.

See LICENSE and NOTICE for details.

SPDX-License-Identifier: MIT

LLNL-CODE-2024514 — Applied Storage Research (ASR)