flopscope

NumPy-compatible math primitives with analytical FLOP counting

Built for the ARC Whitebox Estimation Challenge by AIcrowd

flopscope is a drop-in replacement for a subset of NumPy that counts floating-point operations as you compute. Algorithms submitted to the ARC Whitebox Estimation Challenge are scored by their analytical FLOP cost, not wall-clock time, so researchers can focus on algorithmic innovation rather than hardware tuning. Every arithmetic call deducts from a fixed budget; exceed it and execution stops immediately.

Why flopscope?

import numpy as np


depth, width = 5, 256

# Weight init
scale = np.sqrt(2 / width)
weights = [np.random.randn(width, width) * scale for _ in range(depth)]

# Forward pass
x = np.random.randn(width)
h = x
for i, W in enumerate(weights):
    h = np.einsum("ij,j->i", W, h)
    if i < depth - 1:
        h = np.maximum(h, 0)
# Total FLOPs? No idea.

import flopscope as flops
import flopscope.numpy as fnp

depth, width = 5, 256

# Weight init
scale = fnp.sqrt(2 / width)
weights = [fnp.random.randn(width, width) * scale for _ in range(depth)]

# Forward pass
x = fnp.random.randn(width)
h = x
for i, W in enumerate(weights):
    h = fnp.einsum("ij,j->i", W, h)
    if i < depth - 1:
        h = fnp.maximum(h, 0)
flops.budget_summary()  # 12,459,522 FLOPs

Key Features

• NumPy-compatible API -- import flopscope.numpy as fnp for counted NumPy ops, with import flopscope as flops for budgets and symmetry helpers
• Analytical FLOP counting -- deterministic, hardware-independent cost tracking
• Budget enforcement -- operations are checked before execution; exceeding the budget raises a clear error
• Symmetry-aware einsum -- automatic FLOP savings for repeated operands and declared symmetry groups
• Transparent diagnostics -- inspect per-operation costs, cumulative budget usage, and detailed summaries at any time
• Inspect costs analytically -- flops.einsum_accumulation_cost(...) and flops.reduction_accumulation_cost(...) return the exact FLOP count for an einsum or reduction without running the op
• Truncated SVD -- top-k singular value decomposition with O(m * n * k) cost

What's Supported

Blocked operations (I/O, config, and system calls) raise a helpful AttributeError with a link to the docs.

flopscope arrays are immutable (like JAX): item assignment such as arr[i] = ... or arr[i] += ... raises TypeError. Build results functionally instead — collect the pieces and fnp.stack(...) them, or use a whole-array update (arr = arr + x). See the Immutable arrays guide.

Quick Start

Installation

Latest stable release from PyPI (see release notes):

uv add flopscope
# or
pip install flopscope

For the server-side toolkit (heavy install plus the flopscope-server process for remote-execution architectures):

pip install "flopscope[server]"
# equivalently:
pip install flopscope-server

For the lightweight client (proxies all calls to a remote flopscope-server over ZMQ; no numpy dependency):

pip install flopscope-client

The client occupies the same import flopscope namespace as the main package — install it instead of flopscope, not alongside. All three packages are released in lockstep at the same version. See CHANGELOG.md for the release notes.

Latest development version from git:

uv add git+https://github.com/AIcrowd/flopscope.git
# or
pip install git+https://github.com/AIcrowd/flopscope.git

Basic Usage

import flopscope as flops
import flopscope.numpy as fnp

depth, width = 5, 256

with flops.BudgetContext(flop_budget=10**8, wall_time_limit_s=5.0) as budget:
    # Weight init
    scale = fnp.sqrt(2 / width)
    weights = [fnp.random.randn(width, width) * scale for _ in range(depth)]

    # Forward pass
    x = fnp.random.randn(width)
    h = x
    for i, W in enumerate(weights):
        h = fnp.einsum("ij,j->i", W, h)
        if i < depth - 1:
            h = fnp.maximum(h, 0)

    print(budget.summary())  # current context summary

flops.budget_summary()  # accumulated session/global summary

flopscope FLOP Budget Summary
=============================
  Total budget:             100,000,000
  Used:                      12,459,522  (12.5%)
  Remaining:                 87,540,478  (87.5%)

  By operation:
    random.randn           10,493,952  ( 84.2%)  [6 calls]
    einsum                   1,308,160  ( 10.5%)  [5 calls]
    multiply                   655,360  (  5.3%)  [5 calls]
    maximum                      2,048  (  0.0%)  [4 calls]
    sqrt                             2  (  0.0%)  [1 call]

  Total Wall Time:     ...s
  Flopscope Backend:   ...s  (...%)
  Flopscope Overhead:  ...s  (...%)
  Residual Wall Time:  ...s  (...%)

Budget summary semantics

Local and remote summary dictionaries have the same public schema and accounting meaning. Remote summaries are read-only snapshots of authoritative server state. The global remote view is the current server summary epoch.

The one-participant-process epoch is a WHEST evaluator deployment contract, not a security property of every server invocation. A conforming evaluator launches the server with --token-fd, keeps the minted token in its trusted control channel, performs a post-smoke, pre-scoring reset, and couples server replacement to participant-process replacement. Under that contract, smoke work is excluded and participant code cannot reset the epoch because it never receives the control token. Standalone tokenless local/development servers intentionally allow lifecycle control operations and are not a hardened participant boundary. A client connected to a peer that does not advertise authoritative summaries gets an actionable compatibility error rather than partial or zero-filled data.

Summary construction is O(K) in the operation and optional namespace buckets returned, independent of historical call count. The formatted budget_summary() and BudgetContext.summary() views render those same dictionaries; by_namespace=True adds the namespace mapping without changing the flat totals.

Summary inspection is itself measured FLOPScope work. A summary call takes its snapshot before committing that call's inspection overhead, so the overhead appears in the next snapshot or a later final-close snapshot. A final close response cannot recursively include its own post-boundary serialization.

BudgetContext.summary_dict() has three states: before first entry it returns a canonical empty mapping; while active it reads the authoritative context and normalizes only the four top-level timing fields to the client-owned context's end-to-end timing; after close it returns a defensive copy of the cached final mapping. Live scalar timing properties retain their established None/zero-until-close behavior. After close, every scalar timing property is read from the same cached mapping, so the scalar and dictionary views cannot diverge.

Plan Your Budget Before Executing

# Query FLOP costs without running anything (no BudgetContext needed)
cost = flops.accounting.einsum_cost("ij,jk->ik", shapes=[(256, 256), (256, 256)])
print(f"Matmul cost: {cost:,}")  # 33,488,896

cost = flops.accounting.svd_cost(m=256, n=256, k=10)
print(f"SVD cost: {cost:,}")  # 2,621,440

For symmetry-aware inspection that takes actual array inputs (and reflects declared symmetry), use the accumulation cost APIs:

import numpy as np

A = np.zeros((256, 256))
B = np.zeros((256, 256))
# Returns an AccumulationCost decomposition without running the op
cost = flops.einsum_accumulation_cost("ij,jk->ik", A, B)
print(cost.total)  # 33,488,896

cost = flops.reduction_accumulation_cost(A, op_factor=1)
print(cost.total)  # 65,535

Symmetry Savings

When you pass the same array object multiple times, flopscope automatically detects the symmetry and reduces the FLOP count:

with flops.BudgetContext(flop_budget=10**8) as budget:
    X = fnp.ones((100, 100))

    # Gram matrix: both operands are the same X.
    # flopscope auto-detects this and induces S2{j,k} on the output,
    # giving ~1/2 the dense cost (since R[j,k] = R[k,j]).
    R = fnp.einsum("ij,ik->jk", X, X)
    print(f"Cost with equal-operand detection: {budget.flops_used:,}")

This works for any einsum where the same Python object appears at multiple operand positions. See the exploit-symmetry guide for more examples including triple products and block symmetries.

How It Works

1. FLOPs are tracked automatically. A global default budget activates on first use, or you can wrap code in an explicit BudgetContext for a custom limit. Some ops are free (0 FLOPs) -- zeros, empty, transpose, and other pure-view/uninitialized-allocation operations. Ops that materialize a new buffer (reshape, copy, concatenate, full) are charged numel per element, like any other write; see the FLOP Counting Model for the full free-tier rule.
2. FLOP costs are analytical. Costs are computed from tensor shapes, not measured from execution. A matmul of (m, k) @ (k, n) always costs m * n * (2k - 1) FLOPs (k multiplies plus k - 1 adds per output element) regardless of hardware.
3. Budget is checked before execution. If an operation would exceed the remaining budget, BudgetExhaustedError is raised and the operation does not run.
4. All tensors are plain numpy.ndarray. Standard flopscope arrays are regular NumPy arrays with no hidden state. SymmetricTensor is a lightweight ndarray subclass that carries symmetry metadata for einsum savings — it works everywhere a normal array does.

Sharp Edges

Budget is always active. A global default budget (1e15 FLOPs, configurable via FLOPSCOPE_DEFAULT_BUDGET env var) activates automatically. Use an explicit BudgetContext to set a custom limit.

Only numeric dtypes are supported. flopscope bills bool, integer, float, and complex arrays; every other dtype — object (data that coerces to dtype=object: a None in a list, mixed types, ragged nesting), str_, bytes_, datetime64, timedelta64, and structured/void — raises UnsupportedDtypeError wherever it reaches a registered operation — as an operand, an explicit dtype=, a fill value or distribution parameter, or an out= destination — including fnp.array/fnp.asarray/fnp.astype/fnp.fromiter/fnp.require and every random sampler. Clean the data with plain NumPy first (np.array(x, dtype=np.float64)), or keep ragged data as a Python list of numeric arrays.

31 operations are blocked. Config and system-level functions (set_printoptions, geterr, and text/legacy I/O like savetxt/loadtxt) raise AttributeError by design. Weight I/O is supported: use fnp.savez/fnp.load (and the flops.Module base class) to persist numeric state safely — see the Saving & Loading guide. Loading data is free (0 FLOPs) and never unpickles.

sort, argsort, trace, and random sampling all have analytical FLOP costs based on their algorithmic complexity.

Nested explicit BudgetContexts are not allowed. Opening an explicit BudgetContext while another explicit BudgetContext is already active raises RuntimeError. However, opening an explicit context while only the global default is active is fine — the explicit context temporarily replaces the default.

Cost is analytical, not wall-clock. Two operations with the same shapes always report the same FLOP cost, regardless of data values, cache effects, or hardware. This is intentional -- it makes scores reproducible across machines.

SymmetricTensor propagation rules. Symmetry metadata (used by einsum for FLOP savings) propagates through reshaping, slicing, and unary pointwise operations (e.g., exp, log). Binary pointwise operations (e.g., add, multiply) intersect the symmetry groups of both operands. Reductions may drop symmetry on the reduced axis. If the result doesn't carry the symmetry you expect, declare symmetry groups explicitly with flops.as_symmetric().

Documentation

Getting Started

• Installation & Setup
• Your First Budget

How-To Guides

• Use Einsum
• Exploit Symmetry
• Use Linear Algebra
• Plan Your Budget
• Debug Budget Overruns
• Migrate from NumPy

Concepts

• FLOP Counting Model
• Operation Categories

Development

• Contributor Guide

API Reference

• Full API Reference

For AI Agents:

• Cheat Sheet -- compact reference for fast lookup

Development

git clone https://github.com/AIcrowd/flopscope.git
cd flopscope
make install
make test                      # core test suite
make docs-serve                # local docs at http://127.0.0.1:8000

For the monorepo layout, client/server workflows, and generated-doc rules, see Contributor Guide.

Citation

@misc{flopscope2026,
  title  = {flopscope: NumPy-compatible math primitives with FLOP counting},
  author = {AIcrowd},
  year   = {2026},
  url    = {https://github.com/AIcrowd/flopscope},
  note   = {Built for the ARC Whitebox Estimation Challenge}
}

License

MIT