Skip to main content

Inserts markers in C/C++ code. Markers expose information about the optimizations a compiler has applied, e.g., whether a piece of code was dead code eliminated.

Project description

program-markers inserts markers (as function calls) in C or C++ source code to enable differential testing for missed optimizations:

  1. Instrument a program.
  2. Compile it with two or more compilers (or compiler versions, or optimization levels, etc).
  3. For each output: the markers whose corresponding calls are still present in the generated assembly are alive, the remaining are dead.
  4. Differential testing: compare the sets of alive and dead markers across outputs.

There are two kinds of markers supported:

  • DCE (Dead Code Elimination) Markers
  • VR (Value Range) Markers

A DCEMarker tests if a compiler dead code eliminates a piece of code (basic block). For example,

if (Cond){
  DCEMarker0_();
  STMT0;
  STMT1;
  //...
}

If call DCEMarker0_(); is not present in the generated assembly code then the compiler determined that Cond must always be false and therefore the body of this if statement is dead.

A VRMarker tests if a compiler can determine subsets of the value ranges of variables, for example:

if (!(LB <= a and a <= UB))
    VRMarker0_();

If call VRMarker0_(); is not present in the generated assembly code then the compiler determined that a's value is always LB <= a <= UB. Currently only integer variables are instrumented.

To build just the clang tool

Prerequisites: cmake, make, clang/llvm 13/14.

mkdir build
cd build
cmake .. 
cmake --build . [--parallel]
cmake --install . --prefix=/where/to/install/

Usage

cat test.c
int foo(int a) {
    if (a == 0)
        return 1;
    else {
        a = 5;
    }

    return a;
}


dead-instrument test.c --


cat test.c | clang-format
#if defined DisableDCEMarker0_
#define DCEMARKERMACRO0_ ;
#elif defined UnreachableDCEMarker0_
#define DCEMARKERMACRO0_ __builtin_unreachable();
#else
#define DCEMARKERMACRO0_ DCEMarker0_();
void DCEMarker0_(void);
#endif
#if defined DisableDCEMarker1_
#define DCEMARKERMACRO1_ ;
#elif defined UnreachableDCEMarker1_
#define DCEMARKERMACRO1_ __builtin_unreachable();
#else
#define DCEMARKERMACRO1_ DCEMarker1_();
void DCEMarker1_(void);
#endif
int foo(int a) {
  if (a == 0)
  {
    DCEMARKERMACRO1_
    return 1;
  }
  else {
    DCEMARKERMACRO0_
    a = 5;
  }

  return a;
}

Individual markers can be disabled or turned into unreachables (useful for helping the compiler optimize parts known to be dead):

gcc -E -P -DDisableDCEMarker0_ -DUnreachableDCEMarker1_ test.c | clang-format
int foo(int a) {
  if (a == 0) {
    __builtin_unreachable();
    return 1;
  } else {
    ;
    a = 5;
  }
  return a;
}

Passing --ignore-functions-with-macros to program-markers will cause it to ignore any functions that contain macro expansions.

Value range markers can be emitted instead by using --mode=vr:

cat test.c
int foo(int a) {
  if (a == 0)
    return 1;
  return 0;
}

program-markers --mode=vr test.c --

cat test.c | clang-format 
#if defined DisableVRMarker0_
#define VRMARKERMACRO0_(VAR, TYPE)
#elif defined UnreachableVRMarker0_
#define VRMARKERMACRO0_(VAR, TYPE)                                                   \
  if (!(VRMarkerLowerBound0_ <= (VAR) && (VAR) <= VRMarkerUpperBound0_))       \
    __builtin_unreachable();
#else
#define VRMARKERMACRO0_(VAR, TYPE)                                                   \
  if (!(VRMarkerLowerBound0_ <= (VAR) && (VAR) <= VRMarkerUpperBound0_))       \
    VRMarker0_();
void VRMarker0_(void);
#endif
#ifndef VRMarkerLowerBound0_
#define VRMarkerLowerBound0_ 0
#endif
#ifndef VRMarkerUpperBound0_
#define VRMarkerUpperBound0_ 0
#endif
int foo(int a) {
  VRMARKERMACRO0_(a,"int")
  if (a == 0)
    return 1;
  return 0;
}

The ranges that each marker test can be adjucted via macros:

gcc -E -P -DVRMarkerLowerBound0_=-4 test.c | clang-format
void VRMarker0_(void);
int foo(int a) {
  if (!(-4 <= (a) && (a) <= 0))
    VRMarker0_();
  if (a == 0)
    return 1;
  return 0;
}

Python wrapper

pip install program-markers

To use the instrumenter in python import from program_markers.instrumenter import instrument_program: instrument_program(program: diopter.SourceProgram, ignore_functions_with_macros: bool) -> InstrumentedProgram.

Building the python wrapper

Local build
./build_python_wheel_local.sh #this will build the current branch
pip install .

Docker based build

docker run --rm -e REVISION=REV -v `pwd`:/io theodort/manylinux-with-llvm:latest /io/build_python_wheel_docker.sh

This will build multiple wheels for REV with multiple python versions. The output is stored in the wheelhouse directory. The docker image is based on https://github.com/thetheodor/manylinux-with-llvm.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

program_markers-0.5.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (16.8 MB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.28+ x86-64

program_markers-0.5.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (16.8 MB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.28+ x86-64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page