RunScope

A progress bar that doesn't lie. Calibrated ETAs and completion intelligence for long-running Python jobs.

from runscope import trange           # drop-in for tqdm

for i in trange(48000, key="terrain_analysis"):
    process(tile(i))

terrain_analysis ▕████████░░░░░░░░░░░░▏  38% 17492/48000 · 1h20m left (1h12m–1h31m) · high · [Padawan]

Ordinary progress bars assume the rest of your job looks like the part that already ran. That assumption breaks exactly when it matters — when the expensive work is at the end. RunScope gives you an honest range instead of a fake exact number, and it learns your recurring jobs so each run's estimate gets better than the last.

Install

pip install runscope        # free, local, zero dependencies

Sample usage & output

import runscope

# 1) the simplest thing — wrap any loop
for row in runscope.track(rows, key="clean_dataset"):
    clean(row)

# 2) give it a way to sample the future (measure_fn) and RunScope asks, before the
#    run, what kind of workload this is - and adapts (Jedi for uneven/back-loaded,
#    Padawan for steady). Pass prompt=False to skip the question.
for tile in runscope.track(tiles, key="satellite",
                           weight=lambda t: t.bytes,
                           measure_fn=lambda t: probe_cost(t)):
    process(tile)

What you see in the terminal:

$ python satellite.py
RunScope - Use the Force? (Jedi mode samples the future for a sharper ETA)
  workload:  1) back-loaded  2) uneven  3) slow phase  4) steady
  1/2/3 = best results · 4 = steady, no need · Enter = may the Force be with you!
  [1-4, Enter for Jedi]: 1
The Force is with you. Sampling the future...

RunScope [Jedi]: sampled 50 upcoming items to gauge the real workload ahead.
satellite ▕██████████░░░░░░░░░░▏  50% 24000/48000 · 1h41m left (1h34m–1h48m) · high · [Jedi]
done in 3h29m · 48000 items · [Jedi] called it within 2% (estimated 3h25m)

# a recurring job, on its 4th run, learns from history automatically:
RunScope [Master]: recognized this job from 3 past runs - adjusting by x1.18.
nightly ▕███████░░░░░░░░░░░░░▏  35% 4200/12000 · 22m left (20m–24m) · high · [Master]

Padawan (current run), Master (recurring history) and Jedi (future sampling) all show an honest range and a [Mode] tag. You never pick Padawan or Master — they turn on by themselves; Jedi is the only opt-in.

Use it

1. Wrap any loop

import runscope

for item in runscope.track(items, key="my_job"):
    process(item)

2. Drop-in for tqdm

from runscope import trange
for i in trange(10000, key="my_job"):
    ...

3. Tell it how "big" each item is (stronger estimates)

for path in runscope.track(files, key="ingest", weight=lambda p: p.stat().st_size):
    process(path)

4. Peek at the future for known-heterogeneous jobs

# checks a tiny representative sample of the REMAINING work up front, so a
# back-loaded job can't ambush you with a 3x longer runtime at the end
for item in runscope.track(items, key="my_job", weight=size_of, measure=True):
    process(item)

5. Instrument an existing script without editing it

runscope run train.py        # transparently upgrades tqdm bars in the script

How it works (you never have to think about this)

Three layers combine automatically:

The key is what ties runs of the same job together so RunScope can learn. Use a stable name for recurring jobs (key="nightly_terrain").

The three modes (you never pick — it uses the best it can)

Padawan and Master are automatic and always free. Jedi is opt-in — the first time you use measure=True, RunScope asks once (and remembers). Jedi is free for everyone through September 30, 2026 while we test.

Pricing (after the free period)

Padawan + Master stay free, forever, offline. Jedi becomes part of RunScope Pro:

• Pro — $4/month: unlimited Jedi, cloud run history across machines, and job-finished / ETA-blowout alerts. Priced to cover our cloud costs, not to get rich.

(Introductory pricing, subject to change. Nothing you can do today with Padawan or Master will ever be gated.)

What it's good at (and what it isn't)

RunScope is for enumerable work — loops over files, records, images, tiles, simulations, parameter grids, API calls. That covers a huge amount of scientific and data-processing work.

It does not try to predict the runtime of an arbitrary opaque operation with no sub-steps and no history. When there isn't enough information to estimate honestly, it tells you so instead of inventing a number. That restraint is on purpose.

The science

RunScope's estimators are not heuristics someone made up. They come from a research program that tested dozens of ETA methods against a simple baseline under preregistered pass/fail gates and kept only what won by a required margin. The core finding: for enumerable jobs, a size-weighted estimate is very hard to beat — except by measuring a small sample of the unexecuted work, which cut remaining-time error 40–90% on hard, heterogeneous workloads. That measurement is Jedi mode.

Jedi is design-based sampling, not machine learning. It draws a small representative sample of the remaining items (stratified by observable size, or systematically across the run when sizes are uniform) and forms a Horvitz-Thompson estimate of the remaining cost. Because the sample is chosen by design, the estimate is unbiased and comes with a measurable margin of error rather than a blind guess. The approach applies the same design-based / adaptive-cluster sampling ideas the author used for ecological abundance estimation:

Hariharan, Aneesh; Gallucci, Vincent; Heberer, Craig. Estimation of relative efficiency of adaptive cluster vs traditional sampling designs applied to arrival of sharks. arXiv:1304.2460 (2013).

The insight is the same in both settings: when the quantity you care about is concentrated in places you haven't looked yet, a well-designed sample of the unobserved population beats extrapolating from what you happened to see first.

License

Apache-2.0.