api-gate-keeper

Python-based API test harness for web application security and reliability checks.

About

apptest is a CLI that discovers a web app's API surface from its live OpenAPI schema and runs a broad set of security and reliability checks against it — authentication robustness, injection, authorization/IDOR, JWT misuse, mass assignment, SSRF, rate limiting, and more — with safe-by-default execution and CI-friendly reporting (JSON, Markdown, JUnit, SARIF).

It's built for teams that want a repeatable, automatable security signal in CI/CD without standing up a full scanning platform.

What It Does

• Discovers API endpoints automatically from OpenAPI specs (or manually seeded paths)
• Supports multiple auth strategies: auto, bearer/API key, basic, username/password login, or none
• Runs 20+ security test categories (auth, authz/BOLA, JWT, SSRF, mass assignment, schema conformance, workflow chains, soak/reliability profiling, and more)
• Classifies every check as pass, fail, skipped, or error — with conservative defaults to keep CI signal trustworthy
• Emits JSON, Markdown (summary/detail/failures), JUnit XML, and SARIF reports for CI gating and dashboards
• Ships quick / standard / strict profiles so you can start fast and dial up coverage over time

Navigate

• Quick Start
• Advanced Usage
• Learn More

Quick Start

First-time run should only require three choices:

• target URL (--base-url)
• auth type (API key, username/password, or no auth)
• profile (quick, standard, or strict)

60-Second Path

Use this if you just want a first successful run with minimal setup.

python3 -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt && pip install -e .
apptest --base-url https://your-target.example --api-key <your-api-key> --profile quick

Install and run:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -e .

Recommended first command (safe defaults):

apptest --base-url https://your-target.example --api-key <your-api-key> --profile quick

If the target does not expose an OpenAPI spec, you can seed known endpoints manually:

apptest --base-url https://your-target.example --api-key <your-api-key> \
	--seed-endpoint GET:/api/v1/resources.json \
	--seed-endpoint GET:/api/v1/accounts.json \
	--profile quick

Run multiple targets in one command (parallel):

apptest --targets https://service-a.example,https://service-b.example --api-key <your-api-key> --profile quick

Which Profile Should I Pick?

Auth alternatives:

# username/password
apptest --base-url https://your-target.example --username user@example.com --password secret --profile quick

# public endpoints only
apptest --base-url https://your-target.example --profile quick --config apptest.yaml

If you use the no-auth flow, set this in apptest.yaml:

auth:
	type: none

Minimal starter apptest.yaml (onboarding-focused):

base_url: https://your-target.example
auth:
	type: bearer
	header: "Authorization: Bearer ${TARGET_TOKEN}"
profile: quick

Common First-Run Issues

• 401/403 responses on most checks: verify API key/token, or switch to username/password auth.
• Connection or DNS failures: confirm --base-url is reachable from your environment.
• TLS/certificate errors in non-production environments: retry with --insecure only for test environments.
• Very few tests executed: ensure auth is valid and avoid auth.type: none unless testing public endpoints.

Advanced Usage

Authentication

Supported auth modes:

Auto (default):

• Uses --api-key if provided.
• Otherwise attempts login with --username/--password.

Bearer / API key header:

apptest --base-url https://your-target.example --api-key <your-api-key>

Username + password login:

apptest --base-url https://your-target.example --username user@example.com --password secret

HTTP basic auth (via config):

auth:
	type: basic
	username: ${APP_USERNAME}
	password: ${APP_PASSWORD}

No auth (public API checks only):

auth:
	type: none

Browser-copied session headers / cookies:

apptest \
	--base-url https://your-target.example \
	--request-header 'X-Requested-With: XMLHttpRequest' \
	--cookie-header 'sessionid=abc123; other_cookie=value' \
	--profile quick

Environment variable fallback is supported (precedence: CLI args > env vars > built-in defaults):

• APPTEST_BASE_URL
• APPTEST_API_KEY
• APPTEST_USERNAME
• APPTEST_PASSWORD
• APPTEST_AUTH_TYPE
• APPTEST_AUTH_HEADER
• APPTEST_AUTH_USERNAME
• APPTEST_AUTH_PASSWORD
• APPTEST_CATEGORIES
• APPTEST_SOAK_SAMPLES
• APPTEST_SOAK_MAX_5XX_RATE
• APPTEST_SOAK_MAX_EXCEPTION_RATE
• APPTEST_SOAK_MAX_P95_MS

Example using env vars only:

APPTEST_BASE_URL=https://your-target.example \
APPTEST_USERNAME=your-user@example.com \
APPTEST_PASSWORD=your-password \
apptest

Config File (apptest.yaml)

If apptest.yaml exists in the current directory, it is auto-loaded. You can also pass a custom path with --config.

Precedence is:

• CLI args
• Environment variables
• apptest.yaml
• Built-in defaults

Example apptest.yaml:

base_url: https://your-target.example
auth:
	type: bearer
	header: "Authorization: Bearer ${TARGET_TOKEN}"
profile: quick
timeout: 20
output_dir: reports
include_unsafe: false
unsafe_categories: all
categories: all
soak_samples: 12
soak_max_5xx_rate: 0.15
soak_max_exception_rate: 0.05
soak_max_p95_ms: 2500

Auth modes for auth.type: auto, bearer, api_key, basic, none.

For bearer and api_key, set auth.header (for example Authorization: Bearer <token> or x-api-key: <key>).

For browser-driven apps that require session cookies or extra request headers, you can also set:

auth:
	type: none
	cookie_header: ${TARGET_COOKIE_HEADER}
	extra_headers:
		X-Requested-With: XMLHttpRequest
		Accept: application/json

CLI --request-header values override auth.extra_headers entries with the same header name.

Run with auto-loaded config:

apptest

Run with explicit config path:

apptest --config ./apptest.yaml

Initialize a starter config file:

apptest init

Initialize at custom path:

apptest init --path ./config/apptest.yaml

Overwrite existing config:

apptest init --force

Default Runtime Parameters

• base URL: https://your-target.example
• auth mode: auto
• username: your-user@example.com (used by auto login fallback)
• password: your-password (used by auto login fallback)

Example Commands

Run with defaults (safe mode — skips mutating methods):

apptest --base-url https://your-target.example

Run with profile presets:

# reduced category set, safe mode
apptest --profile quick --base-url https://your-target.example

# all categories, safe mode
apptest --profile standard --base-url https://your-target.example

# all categories, unsafe enabled by default
apptest --profile strict --base-url https://your-target.example

Run specific categories only:

apptest --base-url https://your-target.example --categories auth_robustness
apptest --base-url https://your-target.example --categories token_security
apptest --base-url https://your-target.example --categories privilege_escalation
apptest --base-url https://your-target.example --categories file_upload_security --include-unsafe --unsafe-categories file_upload_security
apptest --base-url https://your-target.example --categories ssrf_and_redirects --include-unsafe --unsafe-categories ssrf_and_redirects
apptest --base-url https://your-target.example --categories rate_limit_and_resource_abuse
apptest --base-url https://your-target.example --categories error_leakage_and_debug_surface
apptest --base-url https://your-target.example --categories mass_assignment --include-unsafe --unsafe-categories mass_assignment
apptest --base-url https://your-target.example --categories schema_contract_and_validation --include-unsafe --unsafe-categories schema_contract_and_validation
apptest --base-url https://your-target.example --categories jwt_security
apptest --base-url https://your-target.example --categories contract_conformance
apptest --base-url https://your-target.example --categories bola_matrix
apptest --base-url https://your-target.example --categories workflow_chains --include-unsafe --unsafe-categories workflow_chains
apptest --base-url https://your-target.example --categories soak_profile
apptest --base-url https://your-target.example --categories baseline,owasp,authz

Tune soak profile thresholds/samples (CLI):

apptest \
	--base-url https://your-target.example \
	--categories soak_profile \
	--soak-samples 8 \
	--soak-max-5xx-rate 0.25 \
	--soak-max-exception-rate 0.10 \
	--soak-max-p95-ms 3500

Soak tuning precedence is:

• CLI flags
• environment variables
• config file keys
• category defaults

Tune soak profile thresholds via environment variables:

APPTEST_SOAK_SAMPLES=8 \
APPTEST_SOAK_MAX_5XX_RATE=0.25 \
APPTEST_SOAK_MAX_EXCEPTION_RATE=0.10 \
APPTEST_SOAK_MAX_P95_MS=3500 \
apptest --base-url https://your-target.example --categories soak_profile

Run with API key instead of username/password:

apptest --base-url https://your-target.example --api-key <your-api-key>

Run with explicit auth header from config/env:

APPTEST_AUTH_TYPE=bearer \
APPTEST_AUTH_HEADER="Authorization: Bearer <token>" \
apptest --base-url https://your-target.example

Run with custom target and creds:

apptest --base-url https://your-target.example --username your-user --password your-pass

Run with unsafe methods enabled for specific categories:

apptest --base-url https://your-target.example --include-unsafe --unsafe-categories baseline

Run with all categories fully unlocked:

apptest --base-url https://your-target.example --include-unsafe --unsafe-categories all

Disable TLS verification (only for test environments):

apptest --base-url https://your-target.example --insecure

Fail CI when findings are present:

apptest --base-url https://your-target.example --fail-on-findings

Fail CI only when thresholds are exceeded:

apptest --base-url https://your-target.example --max-failed 0 --max-errors 0

Generate JUnit XML output for CI test dashboards:

apptest --base-url https://your-target.example --junit-out reports/apptest-junit.xml

Generate SARIF output for GitHub code scanning:

apptest --base-url https://your-target.example --sarif-out reports/apptest.sarif.json

Write reports to explicit paths:

apptest \
	--base-url https://your-target.example \
	--categories auth_robustness \
	--json-out reports/auth_robustness.json \
	--md-summary-out reports/auth_robustness.summary.md \
	--md-detail-out reports/auth_robustness.detail.md \
	--sarif-out reports/auth_robustness.sarif.json

Upload SARIF to GitHub Code Scanning:

- name: Run apptest with SARIF
	run: |
		apptest \
			--base-url "$APPTEST_BASE_URL" \
			--api-key "$APPTEST_API_KEY" \
			--sarif-out reports/apptest.sarif.json

- name: Upload SARIF to GitHub Code Scanning
	if: always()
	uses: github/codeql-action/upload-sarif@v3
	with:
		sarif_file: reports/apptest.sarif.json

Output

Reports are written to reports/ — four files per run, prefixed with the target hostname:

• <target-host>-<timestamp>-api-security-report.json — raw results
• <target-host>-<timestamp>-api-security-summary.md — per-category totals
• <target-host>-<timestamp>-api-security-detail.md — per-test detail table with PASS/FAIL/SKIP
• <target-host>-<timestamp>-api-security-failures.md — failures/errors only, with a prioritized Remediation Plan summary followed by the detail table

Categories Implemented

• baseline API execution
• penetration payload probes
• SQL injection probes
• OWASP-oriented checks
• additional security checks (CORS, basic rate probe)
• authorization and IDOR checks
• authentication robustness checks (invalid credential rejection, failed-login throttle signal)
• token security checks (malformed token, tampered token, missing Bearer prefix; skipped when bearer token auth is not in use)
• privilege escalation checks (likely admin endpoint access with current token)
• file upload security checks (multipart endpoint discovery, unsafe filename traversal and absolute path probes)
• SSRF and redirect checks (ServiceNow instance URL rejection for internal targets, redirect-style parameter discovery)
• rate-limit and resource-abuse checks (burst probes and oversized pagination or time-window requests)
• error leakage and debug surface checks (debug route exposure, unauthenticated debug access, stack-trace/internal error leakage markers)
• mass assignment checks (sensitive writable fields on authenticated JSON mutation endpoints)
• schema contract and validation checks (missing required fields, wrong types, unknown fields)
• JWT-specific token misuse checks (alg:none, stripped signatures, future nbf, kid abuse patterns)
• response contract conformance checks (status-code drift, undocumented response fields, missing required response fields)
• BOLA matrix checks with harvested live IDs (neighbor ID probes and cross-service ID references)
• workflow lifecycle chain checks (create-read-update-delete flow health and auth enforcement)
• soak profile checks on safe endpoints (repeated request stability and latency thresholds; auth-required endpoints with no successful responses are classified as skipped to reduce false positives)

GitHub Actions Integration

Sample workflow (.github/workflows/apptest-scan.yml):

name: Web App Security Scan (apptest)

on:
	pull_request:
		branches: [ main, develop ]
	push:
		branches: [ main, develop ]
	workflow_dispatch:
		inputs:
			base_url:
				description: "Target web app base URL"
				required: false
				default: "https://your-target.example"

jobs:
	apptest-scan:
		runs-on: ubuntu-latest
		timeout-minutes: 30

		env:
			DEFAULT_BASE_URL: https://your-target.example

		steps:
			- name: Checkout
				uses: actions/checkout@v4

			- name: Set up Python
				uses: actions/setup-python@v5
				with:
					python-version: "3.11"

			- name: Install apptest
				run: |
					python -m pip install --upgrade pip
					pip install -e .

			- name: Resolve target URL
				id: target
				shell: bash
				run: |
					if [ -n "${{ github.event.inputs.base_url }}" ]; then
						echo "base_url=${{ github.event.inputs.base_url }}" >> "$GITHUB_OUTPUT"
					elif [ -n "${{ vars.APPTEST_BASE_URL }}" ]; then
						echo "base_url=${{ vars.APPTEST_BASE_URL }}" >> "$GITHUB_OUTPUT"
					else
						echo "base_url=${DEFAULT_BASE_URL}" >> "$GITHUB_OUTPUT"
					fi

			- name: Run apptest (API key preferred)
				shell: bash
				env:
					APPTEST_API_KEY: ${{ secrets.APPTEST_API_KEY }}
					APPTEST_USERNAME: ${{ secrets.APPTEST_USERNAME }}
					APPTEST_PASSWORD: ${{ secrets.APPTEST_PASSWORD }}
				run: |
					set -euo pipefail
					BASE_URL="${{ steps.target.outputs.base_url }}"

					if [ -n "${APPTEST_API_KEY:-}" ]; then
						apptest \
							--base-url "$BASE_URL" \
							--api-key "$APPTEST_API_KEY" \
							--include-unsafe \
							--unsafe-categories baseline
					else
						apptest \
							--base-url "$BASE_URL" \
							--username "${APPTEST_USERNAME}" \
							--password "${APPTEST_PASSWORD}" \
							--include-unsafe \
							--unsafe-categories baseline
					fi

			- name: Enforce gate from JSON report
				shell: bash
				run: |
					set -euo pipefail
					python - <<'PY'
					import json, glob, sys
					report = sorted(glob.glob("reports/*-api-security-report.json"))[-1]
					data = json.load(open(report))
					failed = data.get("summary", {}).get("failed", 0)
					errors = data.get("summary", {}).get("errors", 0)
					print(f"Gate check: failed={failed}, errors={errors}")
					if failed > 0 or errors > 0:
							sys.exit(1)
					PY

			- name: Upload reports
				if: always()
				uses: actions/upload-artifact@v4
				with:
					name: apptest-reports
					path: reports/

Required repository configuration:

• Secret APPTEST_API_KEY (recommended), or both APPTEST_USERNAME and APPTEST_PASSWORD
• Optional repository variable APPTEST_BASE_URL

Simple Workflow (URL + Username + Password Only)

Use this minimal option if you want the easiest integration path.

name: apptest-simple

on:
	workflow_dispatch:
	pull_request:
		branches: [ main ]

jobs:
	apptest:
		runs-on: ubuntu-latest
		steps:
			- uses: actions/checkout@v4

			- uses: actions/setup-python@v5
				with:
					python-version: "3.11"

			- name: Install
				run: |
					python -m pip install --upgrade pip
					pip install -e .

			- name: Run apptest
				env:
					APPTEST_BASE_URL: ${{ secrets.APPTEST_BASE_URL }}
					APPTEST_USERNAME: ${{ secrets.APPTEST_USERNAME }}
					APPTEST_PASSWORD: ${{ secrets.APPTEST_PASSWORD }}
				run: |
					apptest \
						--base-url "$APPTEST_BASE_URL" \
						--username "$APPTEST_USERNAME" \
						--password "$APPTEST_PASSWORD"

			- name: Upload reports
				if: always()
				uses: actions/upload-artifact@v4
				with:
					name: apptest-reports
					path: reports/

Required secrets for the simple workflow:

• APPTEST_BASE_URL
• APPTEST_USERNAME
• APPTEST_PASSWORD

Learn More

• Executive Summary — the problem, the approach, and why it matters, in a few paragraphs
• Design Document — full technical design: discovery, auth, execution, categories, reporting, and extension points
• Test Categories — every check explained: what it tests, how, and why
• Security Risk Brief — what each test category protects against, why it matters, and the business impact of leaving it unaddressed
• Deck Notes — copy/paste-ready slide content for internal readouts