presidio-hardened-arch-translucency 0.10.0

MVP implementation of architectural translucency for Docker/Kubernetes replication layer analysis

presidio-hardened-arch-translucency

v0.10.0 — Architectural Translucency Analyzer for Docker & Kubernetes

Architectural translucency (Stantchev, ~2005) is the ability to monitor and control non-functional properties — especially performance — architecture-wide in a cross-layered way. The core insight: the same measure (replication) has different implications on throughput ω(δ) and response time when applied at different layers.

This CLI tool (pat) helps you choose the replication layer that gives the highest performance gain with the lowest overhead for your workload.

---

Agent Skill — use pat from Claude Code, Cursor, etc.

pat ships with an Agent Skill for AI coding assistants. When an assistant edits a Kubernetes manifest, an HPA, Terraform for EKS/GKE/AKS/ECS/Fargate, or reasons about replica counts and cost-per-request, the skill triggers the assistant to invoke pat and ground its recommendation in the architectural translucency model instead of guessing.

Project-local (auto-discovered in this repo)

Assistant	Path
Claude Code	.claude/skills/pat/SKILL.md
Cursor	.cursor/skills/pat/SKILL.md

An assistant working in this repo discovers them automatically.

Personal install (works across all your projects)

# Claude Code
cp -r .claude/skills/pat ~/.claude/skills/pat

# Cursor
cp -r .cursor/skills/pat ~/.cursor/skills/pat

What the skill does

• Triggers on Kubernetes manifests (Deployment, HPA, StatefulSet, ReplicaSet), Docker Compose deploy.replicas, and Terraform for ECS/Fargate/EKS/GKE/AKS/ACI
• Runs pat analyze, pat cost, pat slo, or pat what-if with inputs gathered from code and context
• Injects the recommendation into the assistant's plan or PR description, citing the architectural translucency model so reviewers can reproduce it
• Refuses to fabricate rps or avg_latency_ms — asks the user instead

The skill is MIT-licensed (same as pat itself) and adds no runtime dependencies; it's plain Markdown that instructs the assistant to shell out to the pat CLI you already installed.

---

Replication Layers (Docker/Kubernetes)

Layer	Description	Fixed Overhead	Coordination Cost
container	New Docker container (process-level isolation)	2%	Low
pod	Kubernetes Pod (shared network namespace)	5%	Moderate
deployment	Kubernetes Deployment/ReplicaSet	10%	High
node	Cluster node (full VM/bare-metal)	18%	Highest

---

Installation

Requires Python ≥ 3.10.

pip install presidio-hardened-arch-translucency

Or with uv:

uv pip install presidio-hardened-arch-translucency

---

Quick Start

# Analyze a 500 req/s workload with 80ms avg latency, currently at container level
pat analyze --requests-per-second 500 --avg-latency-ms 80 --current-layer container

Output:

╭──────────── Presidio Architectural Translucency — Recommendation ────────────╮
│ Recommended layer:  container                                                 │
│ Optimal replicas:   4                                                         │
│ Throughput gain:    +45.2%                                                    │
│ Response-time Δ:    -38.1%                                                    │
│ Est. throughput:    500 req/s                                                  │
│ Est. response time: 49.4 ms                                                   │
│                                                                               │
│ New Docker container (process-level isolation, shared kernel)                 │
╰───────────────────────────────────────────────────────────────────────────────╯

Baseline: 714 req/s @ 80.0 ms  (current layer: container)

Show all layers

pat analyze --requests-per-second 500 --avg-latency-ms 80 \
    --current-layer container --show-all

Layer	Replicas	Throughput	Δ Throughput	Response Time	Δ RT	Recommended
container	4	500	+45.2%	49.4 ms	-38.1%	✓
pod	3	500	+42.0%	55.2 ms	-31.0%
deployment	2	500	+38.1%	68.3 ms	-14.6%
node	1	357	0.0%	80.0 ms	0.0%

---

Dynamic Scaling Analysis (v0.3.0)

pat what-if — HPA Lag Model

Projects the performance trough that occurs between a load spike and the moment new Kubernetes pods become Ready. Shows throughput, latency, p99, and missed requests during the HPA scale-out window.

pat what-if \
  --current-rps 50 --spike-rps 200 \
  --avg-latency-ms 80 --current-layer container \
  --output hpa-event.png

Three stacked panels are saved to hpa-event.png:

• Throughput (req/s) — actual served vs demand, with trough annotation
• Avg latency (ms) — how response time degrades during the trough
• p99 latency (ms) — tail behaviour before and after pods are Ready

Optional overrides: --hpa-poll-s (default 15 s), --pod-startup-s (default 30 s), --cold-start-s (default 0 s), --replicas-before, --replicas-after.

---

pat slo — SLO Compliance Check

Checks whether a p99 latency SLO is met in steady-state and during an HPA trough across all four replication layers.

pat slo \
  --requests-per-second 50 \
  --avg-latency-ms 80 \
  --p99-target-ms 500 \
  --spike-multiplier 3.0

Output table shows steady p99, trough p99, and SLO verdict per layer. The recommendation panel advises the minimum HPA minReplicas needed to eliminate the trough breach.

---

Cost-Aware Analysis (v0.4.0 / v0.5.0)

pat cost — Performance-Per-Dollar Ranking

Cross-layer cost analysis showing hourly cost, cost-per-request, and ROI score for every replication layer.

pat cost \
  --requests-per-second 500 \
  --avg-latency-ms 80 \
  --current-layer container \
  --cost-per-container-hour 0.02 \
  --cost-per-pod-hour 0.05 \
  --cost-per-deployment-hour 0.10 \
  --cost-per-node-hour 0.50

Layer	Replicas	Δ Throughput	Δ RT	Cost/hr	Cost/req	ROI score	Best ROI
container	4	+45.2%	-38.1%	$0.0800	$0.000044	1 027	✓
pod	3	+42.0%	-31.0%	$0.1500	$0.000083	504
deployment	2	+38.1%	-15%	$0.2000	$0.000111	343
node	1	0.0%	0.0%	$0.5000	$0.000278	0

ROI score = throughput-gain-% / cost-per-request (higher = better performance-per-dollar).

pat analyze — Cost columns

Add --cost-per-replica-hour to include cost columns in --show-all output:

pat analyze --requests-per-second 500 --avg-latency-ms 80 \
    --current-layer container --show-all --cost-per-replica-hour 0.02

pat what-if — Trough revenue impact

Add --cost-per-request to see the estimated revenue cost of the HPA trough:

pat what-if --current-rps 50 --spike-rps 200 --avg-latency-ms 80 \
    --current-layer container --cost-per-request 0.001

Output includes:

  Missed reqs   ~1 350
  Trough cost   ~$1.35 revenue impact

pat slo — Min-cost layer that meets SLO

pat slo now shows a Cost/hr column and identifies the cheapest layer that satisfies your p99 target.

---

Cloud Billing Integration (v0.5.0 / v0.6.0)

Replaces manual --cost-per-*-hour flags with live cloud prices fetched from public APIs (no credentials required for on-demand/reserved). Results are cached locally at ~/.pat/pricing-cache.json.

pat cost --cloud aws — EC2 on-demand pricing (v0.5.0)

pat cost \
  --requests-per-second 500 \
  --avg-latency-ms 80 \
  --current-layer container \
  --cloud aws \
  --region us-east-1 \
  --instance-type m5.large

Per-layer costs use packing ratios: 16 containers/node, 8 pods/node.

pat cost --cloud aws --show-reserved — Reserved pricing (v0.6.0)

Add --show-reserved to render 1-year and 3-year No Upfront reserved pricing alongside on-demand in separate tables:

pat cost -r 500 -l 80 -c container \
  --cloud aws --region us-east-1 --instance-type m5.large \
  --show-reserved

pat cost --cloud aws --spot — Spot pricing (v0.6.0)

Requires boto3 and AWS credentials (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY):

pip install "presidio-hardened-arch-translucency[spot]"

pat cost -r 500 -l 80 -c container \
  --cloud aws --region us-east-1 --instance-type m5.large \
  --spot

Spot prices use a 5-minute cache TTL and are annotated with an interruption-risk warning.

pat cost --cloud aws --fargate — Fargate task pricing (v0.5.0)

pat cost -r 500 -l 80 -c container \
  --cloud aws --region us-east-1 --fargate --vcpu 0.5 --memory-gb 1

pat cost --cloud gcp — GCP Compute Engine pricing (v0.6.0)

Fetches from the public GCP Pricing Calculator JSON (no credentials required):

pat cost -r 500 -l 80 -c container \
  --cloud gcp --region us-central1 --machine-type n2-standard-4

# Add --spot for preemptible pricing with interruption-risk annotation
pat cost -r 500 -l 80 -c container \
  --cloud gcp --region us-central1 --machine-type n2-standard-4 --spot

pat cost --cloud azure — Azure VM pricing (v0.6.0)

Fetches from the official Azure Retail Prices API (no credentials required):

pat cost -r 500 -l 80 -c container \
  --cloud azure --region eastus --sku-name "D2s v3"

# Add --spot for Azure Spot VM pricing
pat cost -r 500 -l 80 -c container \
  --cloud azure --region eastus --sku-name "D2s v3" --spot

Cache control

Flag	Effect
(default)	On-demand/reserved: 24 h cache; spot: 5 min cache
--no-cache	Force a fresh API fetch (use in CI cost-gate pipelines)

---

Autoresearch: calibrate → observe → optimize (v0.7.0 / v0.8.0)

The static commands above (analyze, cost, slo, what-if) reason from the analytical model. The autoresearch commands close the loop with measured data: calibrate the model to your workload, record a rolling history of live measurements, and project demand forward into a proactive scaling recommendation — optionally emitted as an apply-able HPA manifest.

All three share ~/.pat/: the fitted model lives at ~/.pat/model.json (or a project-local .pat-model.json, which takes precedence), and the rolling observation history lives in a SQLite store at ~/.pat/observations.db.

pat calibrate — fit the model to measured points (v0.7.0)

Fits the per-replica capacity model (concurrency κ and coordination overhead β) to two or more measured rps:latency_ms:replicas points from your APM, load tests, or prior pat demo runs. Writes ~/.pat/model.json; afterwards the static commands use your fitted parameters and stop emitting the envelope warning. No Docker required.

pat calibrate --observation 100:50:2 --observation 300:80:5

Prints a per-observation prediction/residual table plus overall R² and RMSE.

pat calibrate --benchmark — measure the points with Docker (v0.9.0)

Don't have measured points to hand? Let pat measure them. Benchmark mode sweeps a set of replica counts on the local Docker daemon — starting that many copies of the same Monte Carlo workload pat demo uses, load-testing each — then fits the model to the measured throughput/latency at every count. Requires a running Docker daemon; the workload containers are published to 127.0.0.1 only.

# Sweep 1, 2, 4 replicas, measure each, and fit (defaults to 1 2 4)
pat calibrate --benchmark --layer container \
  --replicas 1 --replicas 2 --replicas 4

# Tune the load applied at each replica count
pat calibrate --benchmark --requests 80 --concurrency 16 --iterations 200000

At least two distinct replica counts are required (one point cannot constrain both parameters). Pass --layer to write per-layer parameters, exactly as in analytical mode. --observation and --benchmark are mutually exclusive.

pat observe — record a rolling measurement history (v0.8.0)

Records a single workload observation into the SQLite store, or lists recent rows. Single-shot by design — it takes one measurement and exits; schedule recurring collection externally (cron, launchd, a Kubernetes CronJob).

# Record one measurement (all fields required when recording)
pat observe --layer container \
  --rps 480 --avg-latency-ms 78 --p99-latency-ms 190 \
  --throughput 470 --replicas 4

# List the most recent observations
pat observe --list --limit 20

The store is source-agnostic — supply numbers from any source. Tag the origin with --source (defaults to manual), and override the store path with --db.

pat observe --prometheus — scrape one sample from Prometheus (v0.8.0)

Scrapes a single sample (rps, p99, replica count) from the Prometheus HTTP API and records it with source='prometheus'. Still single-shot — schedule repeats externally. Prometheus does not know the replication layer, so --layer is required.

export PAT_PROMETHEUS_TOKEN=...   # optional bearer token; env only, never a flag
pat observe --prometheus http://prometheus.monitoring.svc:9090 --layer deployment

The bearer token is read from PAT_PROMETHEUS_TOKEN only — never passed as a CLI argument and never logged.

pat optimize — proactive scaling recommendation (v0.8.0)

Reads the observation store, projects demand a few minutes ahead, and recommends the replica count to serve it.

# Simple moving average over the most-recent samples (default)
pat optimize --model sma --window 10 --horizon-minutes 10

# ARIMA forecast with a 95% confidence interval + replica range
pat optimize --model arima --horizon-minutes 15

--model arima fits a statsmodels ARIMA with a 95% confidence interval and emits a replica range; it auto-falls back to SMA when fewer than 30 samples are available. Restrict to one layer with --layer, or override the store with --db.

pat optimize --emit-hpa-patch — emit an apply-able HPA (v0.8.0)

Instead of the summary panel, emit a sanitised HorizontalPodAutoscaler manifest to stdout for a target Deployment. minReplicas is the point recommendation; maxReplicas is the ARIMA upper-CI bound when available.

pat optimize --model arima --emit-hpa-patch \
  --target my-api --namespace production > hpa-patch.yaml
kubectl apply -f hpa-patch.yaml

The target and namespace are validated as RFC 1123 names; no user input is echoed raw into the manifest.

Workflow example — the observe → optimize loop

# 1. (Once) calibrate the model to a couple of measured operating points
pat calibrate --observation 100:50:2 --observation 300:80:5

# 2. (Recurring, e.g. a */1 * * * * cron entry) record a live sample each minute
pat observe --prometheus http://prometheus:9090 --layer deployment

# 3. After ~30+ samples accumulate, project demand and recommend replicas
pat optimize --model arima --horizon-minutes 15

# 4. When you're ready to act, emit the HPA and apply it
pat optimize --model arima --emit-hpa-patch \
  --target my-api --namespace production | kubectl apply -f -

Until ~30 samples exist, step 3 transparently falls back to SMA, so the loop is useful from the very first observations.

---

Monitoring integration: Prometheus exporter (v0.10.0)

The first step of the monitoring-integration arc ("The Translucency Control Plane"). pat export publishes the model's per-layer recommendations as Prometheus metrics on a read-only /metrics endpoint, so they can be scraped into Grafana alongside the metrics you already run — pat never mutates infrastructure, it only exposes.

# Serve metrics on http://127.0.0.1:9847/metrics (Ctrl-C to stop)
pat export --requests-per-second 500 --avg-latency-ms 80 --current-layer container

# Print the exposition once and exit (no server) — handy for CI / a quick look
pat export -r 500 -l 80 -c container --once

Exposed gauges (per layer where applicable): pat_recommended_replicas, pat_estimated_throughput_rps, pat_response_time_ms, pat_throughput_gain_ratio, pat_layer_recommended, plus pat_workload_* inputs and pat_build_info.

Forecast metrics from the observation store (--predict)

With --predict, the exporter also runs an optimize pass over your observation store (pat observe) on every scrape and exposes the live forecast — turning the exporter from a static view into the moving front of the observe → predict → visualize loop:

# Expose forecast metrics alongside the analysis (SMA by default)
pat export -r 500 -l 80 -c container --predict

# Use ARIMA (refits each scrape — prefer SMA for frequent intervals)
pat export -r 500 -l 80 -c container --predict --model arima --horizon-minutes 15

Adds pat_predicted_rps{model}, pat_predicted_recommended_replicas{layer}, pat_observed_rps/pat_observed_latency_ms, pat_optimize_trend_ratio, pat_optimize_horizon_minutes, and pat_optimize_samples (reads 0 on an empty store). With --model arima it also exposes 95% CI bounds (pat_predicted_rps_lower/_upper and the matching replica bounds). --window, --predict-layer, and --db tune the SMA window, the observation layer, and the store path.

Scrape it from Prometheus:

scrape_configs:
  - job_name: pat
    static_configs:
      - targets: ["127.0.0.1:9847"]

Security. The server is read-only (only GET is implemented — any other method returns 501) and binds 127.0.0.1 by default. Binding a routable interface requires an explicit opt-in:

pat export -r 500 -l 80 -c container --host 0.0.0.0 --listen-public

Metric names are fixed (never user input) and label values are escaped. Use --layer to expose a per-layer calibrated fit (see pat calibrate --layer).

Cost metrics (--cost-per-replica-hour)

Pass a uniform replica cost to add per-layer cost gauges (pat_cost_per_request, pat_hourly_cost_usd):

pat export -r 500 -l 80 -c container --cost-per-replica-hour 0.02

(For live cloud pricing across AWS/GCP/Azure, use pat cost — the exporter keeps a single uniform rate to stay scrape-cheap and network-free.)

Official Grafana dashboard

A ready-to-import dashboard lives at grafana/pat-dashboard.json. It visualises observed-vs-predicted demand (with the ARIMA CI band), recommended replicas per layer, response time, throughput gain, and cost-per-request.

Import it via Grafana → Dashboards → New → Import, upload the JSON, and pick your Prometheus data source when prompted. Pair it with pat export --predict (and --cost-per-replica-hour for the cost panel) for the full picture.

---

Live Demonstrator

pat demo spins up real Docker containers and measures throughput, latency, and CPU across three replication variants, then outputs:

1. A results table and PNG comparison chart
2. An HPA Lag Projection — what happens if load spikes 3× (v0.3.0)
3. A Cost Analysis panel — cost/req per variant and best-ROI layer (v0.5.0)

Requirements: Docker daemon running locally.

# Install with demo extras
pip install "presidio-hardened-arch-translucency[demo]"

# Run the demo (defaults: 4 replicas, 40 requests, 8 concurrent threads)
pat demo

# Custom run with cost override
pat demo --replicas 6 --requests 80 --concurrency 12 \
    --cost-per-container-hour 0.05 --output results.png

Variants compared:

Variant	Description
1 — Single container	Baseline: one container handles all traffic
2 — N containers (round-robin)	Manual container-level replication, client-side LB
3 — N workers + nginx	Simulated Kubernetes Deployment with nginx reverse proxy

Example output:

╭───── Architectural Translucency — Measured Results ──────╮
│ Variant                    Workers  Throughput  Avg Lat   │
│ 1 — Single container            1        8.2    612 ms    │
│ 2 — 4 containers (round-robin)  4       28.7    167 ms ✓  │
│ 3 — nginx LB (4 workers)        5       22.4    213 ms    │
╰──────────────────────────────────────────────────────────╯

Architectural Translucency Insight:
  Manual container replication minimises coordination overhead…

╭──────────── HPA Lag Projection (if load spikes 3×) ─────────────╮
│ TROUGH  (0 s – 45 s)                                             │
│   Throughput    8.2 req/s  (9 % of spike demand)                 │
│   p99 latency   4,896 ms                                         │
│   Missed reqs   ~3,321                                           │
│ STEADY STATE  (after 45 s — 3 replicas)                          │
│   Throughput    24.6 req/s                                        │
│   p99 latency   1,102 ms                                         │
│ → Set HPA minReplicas = 3 to eliminate the trough.               │
╰──────────────────────────────────────────────────────────────────╯

╭────────────────── v0.5.0 Cost Analysis ──────────────────────────╮
│ Best measured variant:  2 — 4 containers (round-robin)           │
│   Cost/req  $0.000077  ·  Cost/hr  $0.0800                       │
│ Analytical best-ROI layer:  container                            │
│   Replicas  4  ·  Throughput gain  +45.2%                        │
│   Cost/req  $0.000044  ·  Cost/hr  $0.0800  ·  ROI score  1027   │
╰──────────────────────────────────────────────────────────────────╯

Two PNG files are saved: demo-results.png (bar chart) and demo-results-hpa.png (3-panel HPA time-series).

---

Security — Presidio Hardening

This toolkit ships with mandatory Presidio security extensions:

Feature	Description
Input sanitization	All workload parameters are bounds-checked and type-validated
Secure logging	Recommendations logged without sensitive data
CVE/dependency audit	pip-audit check on normal command execution (--skip-audit to disable; help/version exits skip network audit)
Security event logging	"Presidio architectural-translucency recommendation applied" emitted
Output sanitization	User-supplied values are never echoed raw into output
Dependabot	Automated dependency updates via .github/dependabot.yml
CodeQL	Static analysis via .github/workflows/codeql.yml

---

CLI Reference

Usage: pat [OPTIONS] COMMAND [ARGS]...

Options:
  -V, --version         Show version and exit.
  -v, --verbose         Enable debug logging.
  --skip-audit          Skip the on-run CVE dependency audit.
  --help                Show this message and exit.

Commands:
  analyze     Analyze workload and recommend the optimal replication layer.
  what-if     Project the HPA scale-out trough for a load spike.
  slo         Check p99 SLO compliance in steady state and during a trough.
  cost        Rank layers by cost-per-request and performance-per-dollar.
  calibrate   Fit the model to measured rps:latency:replicas points.
  observe     Record one workload observation (or --list recent ones).
  optimize    Proactive scaling recommendation from observed history.
  demo        Run the live Docker demonstrator.

pat analyze Options:
  -r, --requests-per-second FLOAT   Observed workload in req/s  [required]
  -l, --avg-latency-ms FLOAT        Current average latency in ms  [required]
  -c, --current-layer TEXT          Current layer (container|pod|deployment|node)  [required]
  --show-all                        Show all layers in a comparison table

Run pat <command> --help for the full option list of any command.

---

Theory: Architectural Translucency Model

The model is based on the replication performance equations from Stantchev's work:

Intensity after replication:

ι(δ) = rps/δ  +  α·rps  +  β·rps·ln(δ)

Throughput:

ω(δ) = min(base_capacity · δ · efficiency(δ), rps)
efficiency(δ) = 1 - α - β·ln(δ)

Response time (M/M/δ approximation):

RT(δ) = avg_latency / (1 - ρ)  +  coordination_overhead
ρ = ι(δ) / base_capacity

Where α (fixed overhead) and β (coordination cost) are layer-specific parameters calibrated for Docker/Kubernetes realities.

The cross-layer recommendation maximises ω(δ) gain while penalising response-time degradation — the central principle of architectural translucency.

---

Development

uv venv .venv && source .venv/bin/activate
uv pip install -e ".[dev]"

# Format + lint
ruff format . && ruff check . --fix

# Tests with coverage
pytest

---

Roadmap

Version	Theme
v0.1.0	MVP — layer analysis & recommendation
v0.2.0	Multi-Python CI hardening
v0.3.0	HPA lag model (pat what-if, pat slo)
v0.4.0	Cost-aware replication analysis (pat cost)
v0.5.0	Cloud billing integration — AWS on-demand pricing
v0.6.0	Cloud billing — AWS reserved/spot + GCP + Azure
v0.7.0	Autoresearch — pat calibrate + observation store + SMA predictions
v0.8.0	Autoresearch — pat observe/pat optimize, Prometheus source, ARIMA + HPA patch emitter
v0.9.0	Per-layer + Docker-benchmark pat calibrate, ARIMA order bounds, observe daemon, security audit
v0.10.0	Monitoring integration — read-only Prometheus exporter (pat export)

Full deliberation and feature details: PRESIDIO-REQ.md

---

License

MIT — see LICENSE.

References

• V. Stantchev, "Effects of Replication on Web Service Performance in WebSphere," Technical Report, ICSI — International Computer Science Institute, Berkeley, CA, USA.
• V. Stantchev, C. Schröpfer, "Negotiating and Enforcing QoS and SLAs in Grid and Cloud Computing," in Advances in Grid and Pervasive Computing (GPC 2009), Lecture Notes in Computer Science, vol. 5529, Springer, 2009.
• V. Stantchev, M. Malek, "Architectural translucency in service-oriented architectures," IEE Proceedings — Software, vol. 153, no. 1, pp. 31–37, 2006. DOI: 10.1049/ip-sen:20050017

---

SDLC

This repository is developed under the Presidio hardened-family SDLC: https://github.com/presidio-v/presidio-hardened-docs/blob/main/sdlc/sdlc-report.md.