PerfSage SignalPilot
Deploy went fine. Errors didn't.
Open-source Kubernetes RCA that answers "why did errors spike after my last deploy?" in under five minutes — by correlating deploy diffs, events, metrics, logs, and git into ranked findings with copy-pastekubectlfixes.
Landing page: perfsage.com/signalpilot · Sample report: examples/sample-report.html
Quick start
pip install perfsage-signalpilot
If PyPI is not yet available in your region, install from the v1.0.0 release:
pip install https://github.com/perfsage/signalpilot/releases/download/v1.0.0/perfsage_signalpilot-1.0.0-py3-none-any.whl
Then:
kubectl apply -f deploy/signalpilot-rbac.yaml
signalpilot analyze my-namespace --deployment my-app --output report.html
CI gate (exit 1 on HIGH+ findings):
signalpilot gate my-namespace --deployment my-app --junit-xml results.xml
From source:
git clone https://github.com/perfsage/signalpilot
cd signalpilot && pip install -e .
Why SignalPilot?
| kubectl | Grafana / APM | SignalPilot | |
|---|---|---|---|
| Deploy context | One object at a time | Metrics without diff | Deploy diff fused into every finding |
| Cross-source evidence | Manual tab-switching | Separate dashboards | Events + metrics + logs + git in one report |
| Actionable output | Raw YAML/events | Charts | Ranked findings + copy-paste kubectl fixes |
| Post-deploy RCA time | Hours (typical war room) | Still manual correlation | Under 5 minutes for typical regressions |
| Cluster agents | N/A | Often required | Read-only RBAC only — no app pod agents |
| Cost | Free | Enterprise tiers | MIT open source |
Not another dashboard. Analysis you can act on. Core RCA uses deterministic rules — optional LLM polish only if you want it.
How it works
SignalPilot runs observe → correlate → explain → recommend → verify → learn:
- Observe — parallel collectors across K8s API, metrics-server, logs, cAdvisor, Prometheus, network, and optional git
- Correlate — deterministic rules fuse cross-source evidence (e.g. OOMKilled + memory at 94% of limit + git commit touching heap code → undersized memory limit)
- Explain — plain-English narrative per finding; optional LLM polish
- Recommend — ranked, copy-paste
kubectlfixes - Verify — baseline before fix, compare after next deploy: Fixed / Regressed / Unchanged
- Learn — revision-keyed history in
.signalpilot/
Signal sources
| Tier | Source | Always-on? |
|---|---|---|
| 0 | Deploy diff (image/env/resources/probes/configmap) | Yes |
| 0 | Git repo correlation (commit SHA → suspect commits) | Optional (--git-repo) |
| 1 | K8s API: restarts, OOMKilled, CrashLoopBackOff, probes | Yes |
| 1 | K8s Events: FailedScheduling, BackOff, Unhealthy | Yes |
| 1 | metrics-server: CPU/memory saturation vs limits | Yes |
| 1 | Container logs: drain3 clustering, new-error detection | Yes |
| 2 | kubelet/cAdvisor: CPU throttling, memory working-set | Yes |
| 2 | Network: endpoint readiness, DNS failures | Yes |
| 4 | Prometheus: p95/p99, error rate, CFS throttle | Auto-detect |
| 4 | Loki, OTel/Jaeger traces | Auto-detect |
RCA rules
| Rule | Trigger signals | Typical fix |
|---|---|---|
oom_killed |
OOMKilled + mem working-set near limit | Raise memory limit |
cpu_throttled |
CFS throttle ratio > 30% ± latency regression | Raise CPU limit/request |
crash_loop |
CrashLoopBackOff + log patterns + config diff | Check env vars, rollback |
image_pull_error |
ImagePullBackOff / ErrImagePull | Fix image tag, rollback |
probe_failure |
Readiness/liveness probe failing | Fix probe path/port/timing |
pending_unschedulable |
Pending pod + FailedScheduling events | Reduce requests, add capacity |
code_regression |
New log fingerprints after deploy ± git suspect | Rollback, investigate commit |
network_latency |
TCP retransmits + DNS failures | Investigate network policy, CoreDNS |
More commands
# Git correlation
signalpilot analyze my-namespace --deployment my-app --git-repo https://github.com/org/app
# Web dashboard
signalpilot serve
# Watch for deploys (auto-analyze)
signalpilot watch my-namespace --output-dir ./reports/
Prerequisites
- Python 3.12+
kubectlconfigured with cluster access- RBAC:
kubectl apply -f deploy/signalpilot-rbac.yaml - Optional: Prometheus (auto-detected at common cluster URLs)
Configuration
All settings via env vars (SIGNALPILOT_*) or .env file:
SIGNALPILOT_PROMETHEUS_URL=http://prometheus:9090
SIGNALPILOT_LLM_PROVIDER=openai # optional
SIGNALPILOT_LLM_API_KEY=sk-... # optional
SIGNALPILOT_SLACK_WEBHOOK_URL=https://hooks.slack.com/...
SIGNALPILOT_GATE_SEVERITY_THRESHOLD=high
SIGNALPILOT_BASELINE_WINDOW_S=1800
SIGNALPILOT_DATA_DIR=.signalpilot
PerfSage product ladder
- PerfSage Reveal — JMeter JTL analysis in the lab
- SLO Reporter — CI gates on load tests
- SignalPilot — post-deploy RCA in production (this repo)
Development
bash install.sh --dev
source .venv/bin/activate
pytest tests/unit/
pytest tests/e2e/ # requires kubectl + sample apps
bash scripts/reset_scenarios.sh # deploy 16 test scenarios
License
MIT © PerfSage
Metadata
Release files for perfsage-signalpilot 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| perfsage_signalpilot-1.0.0.tar.gz | 241.4 kB | Without provenance | Details for perfsage_signalpilot-1.0.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| perfsage_signalpilot-1.0.0-py3-none-any.whl | Python 3 | none | any | Details for perfsage_signalpilot-1.0.0-py3-none-any.whl |
Total release size: 318.1 kB
Release files / perfsage_signalpilot-1.0.0.tar.gz
| Download URL | perfsage_signalpilot-1.0.0.tar.gz |
|---|---|
| Size | 241.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
88e3bbc8c2e43969726bb8a9ed831d1a8a5cc457c02fddda8575c52ebb3a064e
|
|
BLAKE2b-256 checksum How to use checksums |
53da468ebe6d3b2b76bd2481072c9e39c71430e1563fca1cbd7db98b7f2959cb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.13
|
Release files / perfsage_signalpilot-1.0.0-py3-none-any.whl
| Download URL | perfsage_signalpilot-1.0.0-py3-none-any.whl |
|---|---|
| Size | 76.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
195da1df88a1505e6ec7d80490e652916c13876ecb3a81cf06a25d1e9a5bbd24
|
|
BLAKE2b-256 checksum How to use checksums |
dfd91291f37dcefa034a589acd5619b901d5b8b5b206e90b608f58ecfa57fe5f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.13
|
Security for perfsage-signalpilot 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.