perferox
Perferox is a simple performance fuzzer for AI systems software, to try to spot performance bugs in prod systems. It can be used as a CI, always running agent, or extensive performance testing for package releases.
It is geared towards:
- Kernel libraries
- RL infra projects
- Distributed training libraries
- Model serving engines
What it does
Perferox is an agentic loop that tests the absolute edges of a system, by using it in increasingly niche ways, looking for recent PRs that regressed perf or capabilities, trying different combinations of settings, trying less-used/accessible chipsets, etc. in an effort of finding hidden regressions that many users would not report, or that often go unnoticed.
Not everyone opens a github issue when something goes wrong...
How it works
A main agent spawns subagents which each live in persistent tmux sessions, and spin up independent VMs through RunPod or Lambda, and based off the instructions of the main agent, tries attacking a certain vulnerability the main agent thinks is worthwhile prodding about, and comes back with the results, and updates a local DB with detailed information on experiments ran, and points out when it notices an anomaly.
All this information is easily visible through the TUI, and can also quickly be seen via CLI The goal is to keep this very much hackable, such that you can customize it for your small OSS project, or your niche cloud provider that you have a bunch of credits for, or your mess of local machines.
I am building this in a way such that the abstractions allow easy extensibility beyond basic functionality, and there are comments just for agents to guide them towards implementing idiomatic code for your setup.
RunPod and Lambda are supported today through runpodctl and lambda-labs. I plan on adding support for other major projects (vLLM, DeepSpeed, flashinfer, etc.) and other neoclouds with a CLI or MCP.
Because of this, I cannot imagine this going beyond 10 kLOC, (As of Jul. 9th we are hovering ~2 kLOC)
Extra Info
As of Jul. 9th, this is built for SGLang and supports RunPod and Lambda. In the future, I plan on making it generalized such that it is very easy to point an agent at perferox source code and it can customize it for whatever project you are working on.
It is clearly pre-beta right now
1 maintainer
Metadata
1 maintainer
Release files for perferox 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| perferox-0.1.1.tar.gz | 15.6 MB | Without provenance | Details for perferox-0.1.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| perferox-0.1.1-py3-none-any.whl | Python 3 | none | any | Details for perferox-0.1.1-py3-none-any.whl |
Total release size: 31.3 MB
Release files / perferox-0.1.1.tar.gz
| Download URL | perferox-0.1.1.tar.gz |
|---|---|
| Size | 15.6 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
1f9cbbea46e47e8cafe22d84fdeb656ece9c828adbff55e00576d5348ce399f0
|
|
BLAKE2b-256 checksum How to use checksums |
3250e0bb4f7044072b1ab1206deef05de4b940199367e7017f5c312dc1ed9139
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.22 {"installer":{"name":"uv","version":"0.9.22","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / perferox-0.1.1-py3-none-any.whl
| Download URL | perferox-0.1.1-py3-none-any.whl |
|---|---|
| Size | 15.7 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9d22083876fcbb655117bdaafb1429515396bb1be2642d1f4f9552c50cec7f64
|
|
BLAKE2b-256 checksum How to use checksums |
693dca43b9b4a577a8f0059e03a77caa0e21d1b83642bfe9b32007d02760a636
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.22 {"installer":{"name":"uv","version":"0.9.22","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for perferox 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
