Skip to main content

audit-packs

PyPI version Python Downloads License GitHub Repository

audit-packs banner

An evidence-first Compliance Intelligence Engine that transforms security scanner findings into standardized, evidence-backed compliance artifacts (an evidence-backed PR summary comment, OSCAL assessment results, SARIF, and control coverage reports).


🔍 The Problem

Modern security scanners (such as Checkov, Semgrep, Trivy, tfsec, and gitleaks) are highly effective at identifying infrastructure-as-code (IaC) misconfigurations and software vulnerabilities. However, they are scanner-centric and do not directly answer the critical questions that governance, risk, and compliance (GRC) teams or auditors ask:

  • Which compliance controls (e.g., NIST 800-53, SOC 2, HIPAA, GDPR) are affected by this finding?
  • Where is the exact code/configuration evidence to prove compliance or violation?
  • How do we avoid false-positive alerts bloating our engineering workflows?

💡 The Solution

audit-packs bridges the gap by providing a scanner-agnostic mapping, enrichment, and consensus layer. Rather than replacing existing detection tools, it takes their output (typically via SARIF), normalizes it, enriches it with codebase/git evidence context, maps the findings to GRC control requirements, and optionally filters noise using an AI consensus ensemble.


📦 Installation

To run audit-packs locally as a CLI tool:

# Install audit-packs CLI via pip
pip install audit-packs

# Or install in an isolated environment via pipx (recommended)
pipx install audit-packs

Note: Detection is delegated to best-in-class open-source engines. For the scanners to run, ensure they are installed on your system path, or inject them into the pipx environment:

pipx inject audit-packs checkov semgrep

🚀 Quick Start

1. Initialize Configuration

Bootstrap your repository with local config files — an interactive wizard writes an audit-models.yaml model-router file and records your framework selection. It downloads nothing; the compliance packs ship bundled with the CLI:

audit-packs --init

2. Run a Compliance Scan

Scan your workspace and map findings to NIST 800-53 and SOC 2 frameworks:

audit-packs --frameworks nist-800-53,soc2

📋 CLI Flags

These flags belong to the audit-packs command-line scanner (argparse). Run audit-packs --help for the authoritative list.

Flag Default Description
--frameworks nist-800-53 Comma/newline-separated framework pack IDs to evaluate (e.g., nist-800-53,soc2).
--fail-on high Minimum finding severity that exits non-zero: low, medium, high, critical.
--scan-mode both Scan scope: diff (PR-changed lines only), full (entire posture), or both.
--base-ref origin/main Base git ref to diff against for diff-mode scanning.
--workspace $GITHUB_WORKSPACE or . Repository workspace directory to scan.
--packs-dir bundled Directory of custom compliance pack YAMLs.
--rules-path bundled Directory of Semgrep rule files.
--scanners-dir none Directory of declarative scanner plugin definitions.
--adjudication-mode off AI consensus adjudication: off, advisory (score only), or enforce (suppress below threshold).
--confidence-threshold 0.70 Composite confidence threshold (0.0–1.0) applied under enforce mode.
--codeql-sarif none Directory of CodeQL SARIF files to ingest (skipped if absent).
--trivy / --no-trivy from env Enable/disable Trivy filesystem scanning.
--trivy-image none Docker image tag to scan with Trivy.
--tfsec / --no-tfsec from env Enable/disable tfsec Terraform scanning.
--gitleaks / --no-gitleaks from env Enable/disable gitleaks secret detection.
--allow-repo-config false Trust repo-supplied config (ast-rules/, scanner plugins, model base_url/api_key_env). Off by default.
--slack-webhook $SLACK_WEBHOOK_URL Slack webhook URL for scan alerts.
--jira-url $JIRA_URL Jira Server/Cloud URL for issue creation.
--jira-email $JIRA_EMAIL Jira account email for API auth.
--jira-token $JIRA_API_TOKEN Jira API token.
--jira-project $JIRA_PROJECT Jira project key.
--init N/A Interactive configuration wizard (writes local config — see Quick Start).
--validate-policy N/A Validate custom compliance packs and Semgrep rule schemas, then exit.

Notes:

  • --trivy/--tfsec/--gitleaks default from their env vars (TRIVY_ENABLED, TFSEC_ENABLED, GITLEAKS_ENABLED); when those are unset on the CLI the scanner is off. The GitHub Action enables them by default via the corresponding inputs.
  • Output emission is controlled by environment variables, not CLI flags: EMIT_OSCAL, EMIT_COVERAGE, EMIT_SARIF (all default true). These map to the emit-oscal/emit-coverage/emit-sarif Action inputs.

🧰 pack Subcommands

Manage custom framework packs via the audit-packs pack command space. See the pack management guide for details.

Subcommand Usage Description
pack init audit-packs pack init <pack-id> [--output-dir packs] Scaffold a new framework pack directory.
pack validate audit-packs pack validate <pack-path> Validate a pack directory against the schema.
pack test audit-packs pack test <pack-path> --fixture <dir> [--scanners-dir <dir>] Dry-run pack mappings against scan fixtures.
pack publish audit-packs pack publish <pack-path> [--output-dir .] Package a pack into a distributable tarball.
pack install audit-packs pack install <source> [--output-dir <dir>] Install a pack from a URL, owner/repo@version, or local tarball.

🌐 GitHub Action Integration

audit-packs is designed to run seamlessly in GitHub Action pipelines to block compliance regressions on pull requests.

# .github/workflows/compliance-audit.yml
name: Compliance Audit

on:
  pull_request:

jobs:
  audit:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write # Required to post the PR summary comment

    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0 # Required for diff-only scanning

      - name: Run Audit Packs compliance check
        uses: prakharsingh/audit-packs@v0
        with:
          frameworks: nist-800-53,soc2
          fail-on: high

Action Inputs

Generated from action.yml. Input names use hyphens (GitHub Action convention) and may differ from the CLI flag names above.

Input Default Description
frameworks Required Newline/comma-separated framework IDs to assess.
min-confidence 0.70 Composite score threshold (0.0–1.0); findings below are suppressed in enforce mode. Maps to the --confidence-threshold CLI flag.
adjudication-mode off off (no LLM calls), advisory (score shown, nothing suppressed), or enforce (suppress below threshold).
models-config audit-models.yaml Path to a model-routing YAML file (provider, model, base_url, api_key_env per role). Falls back to built-in defaults if absent.
detector-model "" Override the detector role's model (sets DETECTOR_MODEL).
verifier-model "" Override the verifier role's model (sets VERIFIER_MODEL).
adversarial-model "" Override the challenger role's model (sets CHALLENGER_MODEL).
judge-model "" Override the consensus role's model (sets CONSENSUS_MODEL).
codeql-sarif "" Directory of CodeQL SARIF files from github/codeql-action/analyze. Skipped if empty.
ast-rules ast-rules Path to custom AST rule modules (executed by the built-in AST engine).
trivy-enabled true Run Trivy filesystem scan for IaC misconfigs.
trivy-image "" Docker image tag to scan with Trivy. Empty = skip image scan.
tfsec-enabled true Run tfsec for Terraform security checks.
gitleaks-enabled true Run gitleaks to detect hardcoded secrets and credentials.
fail-on high Minimum severity that blocks the PR: low, medium, high, critical.
base-ref origin/main Git ref to diff against.
scan-mode both diff, full, or both.
emit-oscal true Emit OSCAL assessment-results JSON.
emit-coverage true Emit coverage matrix as Markdown and HTML.
emit-sarif true Emit aggregate SARIF for upload.
seo-title Audit Packs Control Coverage Matrix HTML title for coverage.html.
seo-description Compliance control coverage report generated by audit-packs. HTML meta description for coverage.html.
seo-canonical-url "" Canonical URL for coverage.html when published.
allow-repo-config false Trust repo-supplied config (ast-rules/, scanner plugins, audit-models.yaml base_url/api_key_env). The scanned repo is untrusted by default.
redact-engines "" Comma-separated engines whose raw output is fully masked before any output sink. Masking is applied post-adjudication — the LLM still sees raw evidence for these engines.
github-token ${{ github.token }} Token used to post PR review comments.
pr-number triggering PR Pull request number to comment on. Empty on non-PR events.

Action Outputs

Exposed as GitHub Action step outputs (from action.yml).

Output Description
oscal-path Workspace-relative path to the OSCAL JSON output file.
coverage-md-path Workspace-relative path to the Markdown coverage matrix.
coverage-html-path Workspace-relative path to the HTML coverage matrix.
sarif-path Workspace-relative path to the aggregate SARIF file.

📊 Supported Compliance Frameworks

Compliance frameworks are defined as declarative YAML packs. The following packs are supported:

Framework Pack ID Automated Controls
NIST SP 800-53 Rev 5 nist-800-53 21 (Canonical baseline)
SOC 2 Type II soc2 39 (Technical criteria)
ISO/IEC 27001:2022 iso27001 7
PCI-DSS v4.0 pci-dss 8
FedRAMP Moderate fedramp 8
HIPAA Security Rule hipaa 6
GDPR gdpr 5
Custom Org-Policy org-policy Configurable

📤 Output Artifacts

  • PR Summary Comment: Posts one updatable summary comment containing control mappings and evidence for all diff-filtered findings in the PR.
  • OSCAL Assessment Results: Machine-readable oscal.json compliant with NIST SP 800-53 GRC tooling workflows.
  • Coverage Reports: Beautiful coverage.md and coverage.html containing an audit-ready compliance matrix.
  • Aggregated SARIF: A combined audit-packs.sarif file containing all scanner findings mapped to controls.

📦 Ecosystem Architecture

audit-packs is built as a modular ecosystem consisting of five Python packages:

Package PyPI Link Role Standalone?
audit-packs PyPI Main CLI & Action entrypoint Yes
audit-packs-core PyPI Primitives, diff parsing, normalization No
audit-packs-mapping PyPI Compliance pack loader & OSCAL exporter No
audit-packs-evidence PyPI Evidence collectors & heuristic agents No
audit-packs-ai PyPI LLM consensus & confidence scoring No

🤝 Contributing & Backtrack Links

📄 License

This project is licensed under the Apache-2.0 License. See the LICENSE file in the main repository for details.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

audit_packs-0.9.0.tar.gz (46.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

audit_packs-0.9.0-py3-none-any.whl (49.2 kB view details)

Uploaded Python 3

File details

Details for the file audit_packs-0.9.0.tar.gz.

File metadata

  • Download URL: audit_packs-0.9.0.tar.gz
  • Upload date:
  • Size: 46.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for audit_packs-0.9.0.tar.gz
Algorithm Hash digest
SHA256 ad4e0fc7dfe8984fc4d7499b2187939ac8861cbb7d77ce8624bbd79bb7d8dd0b
MD5 d5861557d2836cfe8dc65f7f0d2ccd70
BLAKE2b-256 ad53481958d1d21e447b93629be63895937748acef813b5047e357fca26dc82f

See more details on using hashes here.

File details

Details for the file audit_packs-0.9.0-py3-none-any.whl.

File metadata

  • Download URL: audit_packs-0.9.0-py3-none-any.whl
  • Upload date:
  • Size: 49.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for audit_packs-0.9.0-py3-none-any.whl
Algorithm Hash digest
SHA256 83a946ef9338acb2b93dd794c8acb716b9e29ecbf7ad6fc65f3cae11ed825a02
MD5 4728de12bc30bf3e9e945405f73046fc
BLAKE2b-256 2aabd278fccd747df6bfe4b5d7e4796b8bfbe422464089a38954998d454e2b54

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page