proxybench 0.1.0

Enterprise proxy performance benchmark — measure success rate, latency and throughput across concurrency and protocols (HTTP/HTTPS/SOCKS5). Built on proxy-builder-sdk.

proxybench

An open-source proxy performance benchmark. Measure a proxy's real success rate, latency, and throughput across concurrency and protocols (HTTP / HTTPS / SOCKS5) — with a fair, reproducible methodology and raw per-request data you can audit.

Built and maintained by the team at aethyn.io, but vendor-neutral by design: Aethyn is scored by the exact same numbers as every other provider, and it builds endpoints via proxy-builder-sdk so one config benchmarks Aethyn, Bright Data, Oxylabs, Decodo, IPRoyal, SOAX, NetNut — or any proxy at all — identically.

pip install proxybench

# benchmark any proxy in seconds — just give it the URL
proxybench run --proxy-url "http://user:pass@host:port" --target-url https://api.ipify.org

You get a live summary table plus raw.jsonl (every request), summary.json, and reports in runs/<timestamp>/.

Benchmark any proxy

Three ways to point it at a proxy — no account or SDK provider required:

1. A raw URL — the fastest path. HTTP or SOCKS5; the scheme is authoritative:

proxybench run --proxy-url "http://user:pass@1.2.3.4:8080"  --target-url https://api.ipify.org
proxybench run --proxy-url "socks5://user:pass@1.2.3.4:1080" --protocols socks5 --target-url https://api.ipify.org

2. Your own gateway (host + port + creds, in a config file):

providers:
  - name: my-gateway
    host: gw.myprovider.com
    port: 9000
    credentials: { username: { env: PROXY_USERNAME }, password: { env: PROXY_PASSWORD } }

3. A named provider (Aethyn or any proxy-builder-sdk provider), with full targeting — and an optional gateway host override for self-hosted/staging:

PROXY_USERNAME=... PROXY_PASSWORD=... \
proxybench run --provider aethyn --workers 8,32 --requests 200

What it measures

• Success rate — three ways, side by side (strict / hard-fail / soft-fail), each with a 95% Wilson confidence interval, and a mandatory content-nonce assertion so a "200" from a block page never counts as success.
• Latency, 4 segments — connect / TLS / TTFB / total, reported as p50/p90/p95/p99 (never mean-only — proxy latency is heavy-tailed).
• Throughput — successful req/s per concurrency level, with concurrency-sweep knee detection (the worker count past which more concurrency stops helping).
• Provider comparison — rank providers per (protocol, concurrency), and flag when the top two are statistically indistinguishable (CIs overlap) — no cherry-picking.
• Error taxonomy — timeouts, resets, 403/429/407, TLS, proxy-auth.

Commands

proxybench run [config.yaml] [flags]     # run a benchmark
proxybench resume runs/<id>              # continue an interrupted run (skips finished cells)
proxybench compare runs/<a> runs/<b>     # diff two runs, cell by cell
proxybench presets                       # list built-in run presets

Presets scale the run to intent — --preset quick (smoke), sweep (concurrency curve), or thorough (big samples, tight CIs); anything you set explicitly overrides them.

Config

Flags override a YAML file; a file can extends: a preset.

extends: sweep              # optional: inherit a built-in preset, then override below
run:
  engine: thread            # thread pool (default) | async
  workers: [8, 32, 128]     # a list runs a concurrency sweep
  rate_limit_rps: 50        # global safety cap (on by default)
  total_requests: 200
  warmup_requests: 20
providers:
  - { name: aethyn,  credentials: { username: { env: AETHYN_USER }, password: { env: AETHYN_PASS } } }
  - { name: oxylabs, credentials: { username: { env: OXY_USER },    password: { env: OXY_PASS } } }
  # - { url: "http://user:pass@host:port" }        # or a raw proxy URL
targeting: { country: us, session: run1, ttl: 10, tier: elite }
protocols: [http, socks5]
targets: { url: "https://api.ipify.org/?format=json", nonce_param: null }
report: { formats: [json, csv, markdown, html], out_dir: runs }

Every run writes raw.jsonl (one line per request); report.formats selects any of json / csv / markdown / html on top. The HTML report is self-contained (bars, CIs, knee — no external assets).

Benchmark / stress-test your own endpoint

The public echo targets are for light sanity checks. To push real load through a proxy against a URL you control (load-testing your own app/API through residential IPs), point proxybench at it and assert authorization:

proxybench run \
  --proxy-url "http://user:pass@host:port" \
  --target-url https://your-app.example/health \
  --i-am-authorized \
  --workers 64 --requests 5000 --rate-limit-rps 200

--i-am-authorized confirms you own or are permitted to test that host — the allowlist blocks arbitrary third-party targets so proxybench can't be used to hammer someone else's site. In a config file, set it once:

targets: { url: "https://your-app.example/health", authorized: true }

Responsible use

proxybench is a measurement tool, not a load/DoS tool. It ships with a global rate limiter (on by default) and a target allowlist — it refuses to hammer arbitrary third-party hosts. Point it at endpoints you own or are permitted to test; --i-am-authorized is your explicit assertion for custom targets. See ETHICS.md.

Links

• aethyn.io — residential proxies; builds and maintains proxybench
• proxy-builder-sdk — the provider layer proxybench builds endpoints with
• CHANGELOG · ETHICS

License

MIT