include-what-costs

A tool to analyze C++ header dependencies and compile-time costs.

Installation

Using pixi (recommended):

pixi install
pixi run include-what-costs --help

Or with pip:

pip install -e .

Dependencies

• prmon - for memory/time benchmarking (must be in PATH)
• graphviz - for graph rendering (specifically twopi for radial layout)

Commands

analyze - Analyze header dependencies and benchmark costs

Builds the complete include dependency graph and optionally benchmarks compile costs.

include-what-costs analyze \
    --root path/to/header.h \
    --compile-commands build/compile_commands.json \
    --output results/

With benchmarking (measures RSS and compile time for each header):

include-what-costs analyze \
    --root path/to/header.h \
    --compile-commands build/compile_commands.json \
    --benchmark \
    --output results/

Benchmark only the top N headers (by depth, then preprocessed size):

include-what-costs analyze \
    --root path/to/header.h \
    --compile-commands build/compile_commands.json \
    --benchmark 50 \
    --output results/

consolidate - Find headers exposing external dependencies

Identifies which of your headers expose a specific external dependency (e.g., DD4hep) and estimates the cost of removing that exposure.

include-what-costs consolidate \
    --root path/to/header.h \
    --compile-commands build/compile_commands.json \
    --prefix /home/user/project \
    --pattern DD4hep

Output shows:

• Which of your headers include the external dependency
• How many of your other headers would be affected by removing each include
• Estimated memory savings from removing the dependency

trace - Find include path between headers

Finds and displays the shortest include path(s) between two headers. Useful for understanding why a particular header is being included.

include-what-costs trace \
    --root path/to/header.h \
    --compile-commands build/compile_commands.json \
    --to "DD4hep/Objects.h"

By default traces from the root header. Use --from to trace from a different starting point:

include-what-costs trace \
    --root path/to/header.h \
    --compile-commands build/compile_commands.json \
    --from "MyHeader.h" \
    --to "DD4hep/Objects.h"

Shows up to 10 shortest paths by default. Use -n to control:

# Show only 1 path
include-what-costs trace ... --to "SomeHeader.h" -n 1

# Show up to 20 paths
include-what-costs trace ... --to "SomeHeader.h" -n 20

Example output:

8 shortest path(s) of length 8, 5 more not shown:

Path 1:
JIT_includes.h
  -> TrackLike.h
    -> KalmanFitResult.h
      -> Measurement.h
        -> DeMuonChamber.h
          -> DeMuonChamber.h
            -> DeIOV.h
              -> DD4hep/Handle.h

Path 2:
...

Common Options

These options are shared across all subcommands:

Option	Description
--root	Root header file to analyze (required)
--compile-commands	Path to compile_commands.json (required)
--prefix	Path prefix for filtering/display (can be repeated)
--wrapper	Wrapper command for gcc (e.g., ./Rec/run)
--config	Path to YAML config file

Subcommand-specific options

analyze:

Option	Description
--output	Output directory (default: results)
--benchmark [N]	Benchmark headers. Without N: all headers. With N: top N by (depth, preprocessed size)

consolidate:

Option	Description
--pattern	Substring pattern to match external headers (required)
--output	Optional JSON output path

trace:

Option	Description
--from	Source header substring (defaults to --root)
--to	Target header substring (required)
-n, --max-paths	Maximum paths to show (default: 10)

Output Files (analyze)

File	Description
include_graph.json	Full dependency graph and analysis data
include_graph.html	Interactive HTML visualization
header_costs.json	Per-header RSS and compile time (if benchmarked)
header_costs.csv	Same in CSV format
summary.txt	Human-readable summary

Using a Config File

YAML config files can specify common options:

root: path/to/header.h
compile-commands: build/compile_commands.json
wrapper: ./run_env.sh
prefix:
  - /home/user/project
  - /home/user/other
output: results/
benchmark: 50  # or true for all

include-what-costs analyze --config my_config.yaml
include-what-costs trace --config my_config.yaml --to "SomeHeader.h"

LHCb Usage

For analyzing JIT functor compilation includes:

cd ~/stack

# Run analysis with benchmarking
pixi run -m include-what-costs/ include-what-costs analyze \
    --root Rec/Phys/FunctorCore/include/Functors/JIT_includes.h \
    --compile-commands Rec/build.x86_64_v3-el9-gcc13-opt/compile_commands.json \
    --wrapper ./Rec/run \
    --prefix ~/stack \
    --benchmark \
    --output results/

# Find what's pulling in DD4hep
pixi run -m include-what-costs/ include-what-costs consolidate \
    --root Rec/Phys/FunctorCore/include/Functors/JIT_includes.h \
    --compile-commands Rec/build.x86_64_v3-el9-gcc13-opt/compile_commands.json \
    --wrapper ./Rec/run \
    --prefix ~/stack \
    --pattern DD4hep

# Trace path to a specific DD4hep header
pixi run -m include-what-costs/ include-what-costs trace \
    --root Rec/Phys/FunctorCore/include/Functors/JIT_includes.h \
    --compile-commands Rec/build.x86_64_v3-el9-gcc13-opt/compile_commands.json \
    --wrapper ./Rec/run \
    --prefix ~/stack \
    --to DD4hep/Handle.h

How It Works

1. Extract compile flags from compile_commands.json (auto-detects the right source file based on the root header path)

2. Run gcc -H to get the complete include hierarchy (stderr contains the include tree with depth indicated by dots)

3. Supplement edges by parsing #include directives directly from headers (gcc -H only shows first inclusion of each header)

4. Benchmark headers (if requested) by compiling minimal .cpp files that include just that header, measuring RSS and time with prmon

5. Generate outputs: JSON data, HTML visualization, and summary