audit-packs
An evidence-first Compliance Intelligence Engine that transforms security scanner findings into standardized, evidence-backed compliance artifacts (an evidence-backed PR summary comment, OSCAL assessment results, SARIF, and control coverage reports).
🔍 The Problem
Modern security scanners (such as Checkov, Semgrep, Trivy, tfsec, and gitleaks) are highly effective at identifying infrastructure-as-code (IaC) misconfigurations and software vulnerabilities. However, they are scanner-centric and do not directly answer the critical questions that governance, risk, and compliance (GRC) teams or auditors ask:
- Which compliance controls (e.g., NIST 800-53, SOC 2, HIPAA, GDPR) are affected by this finding?
- Where is the exact code/configuration evidence to prove compliance or violation?
- How do we avoid false-positive alerts bloating our engineering workflows?
💡 The Solution
audit-packs bridges the gap by providing a scanner-agnostic mapping, enrichment, and consensus layer. Rather than replacing existing detection tools, it takes their output (typically via SARIF), normalizes it, enriches it with codebase/git evidence context, maps the findings to GRC control requirements, and optionally filters noise using an AI consensus ensemble.
📦 Installation
To run audit-packs locally as a CLI tool:
# Install audit-packs CLI via pip
pip install audit-packs
# Or install in an isolated environment via pipx (recommended)
pipx install audit-packs
Note: Detection is delegated to best-in-class open-source engines. For the scanners to run, ensure they are installed on your system path, or inject them into the
pipxenvironment:pipx inject audit-packs checkov semgrep
🚀 Quick Start
1. Initialize Configuration
Bootstrap your repository with local config files — an interactive wizard writes an audit-models.yaml model-router file and records your framework selection. It downloads nothing; the compliance packs ship bundled with the CLI:
audit-packs --init
2. Run a Compliance Scan
Scan your workspace and map findings to NIST 800-53 and SOC 2 frameworks:
audit-packs --frameworks nist-800-53,soc2
📋 CLI Flags
These flags belong to the audit-packs command-line scanner (argparse). Run audit-packs --help for the authoritative list.
| Flag | Default | Description |
|---|---|---|
--frameworks |
nist-800-53 |
Comma/newline-separated framework pack IDs to evaluate (e.g., nist-800-53,soc2). |
--fail-on |
high |
Minimum finding severity that exits non-zero: low, medium, high, critical. |
--scan-mode |
both |
Scan scope: diff (PR-changed lines only), full (entire posture), or both. |
--base-ref |
origin/main |
Base git ref to diff against for diff-mode scanning. |
--workspace |
$GITHUB_WORKSPACE or . |
Repository workspace directory to scan. |
--packs-dir |
bundled | Directory of custom compliance pack YAMLs. |
--rules-path |
bundled | Directory of Semgrep rule files. |
--scanners-dir |
none | Directory of declarative scanner plugin definitions. |
--adjudication-mode |
off |
AI consensus adjudication: off, advisory (score only), or enforce (suppress below threshold). |
--confidence-threshold |
0.70 |
Composite confidence threshold (0.0–1.0) applied under enforce mode. |
--codeql-sarif |
none | Directory of CodeQL SARIF files to ingest (skipped if absent). |
--trivy / --no-trivy |
from env | Enable/disable Trivy filesystem scanning. |
--trivy-image |
none | Docker image tag to scan with Trivy. |
--tfsec / --no-tfsec |
from env | Enable/disable tfsec Terraform scanning. |
--gitleaks / --no-gitleaks |
from env | Enable/disable gitleaks secret detection. |
--allow-repo-config |
false |
Trust repo-supplied config (ast-rules/, scanner plugins, model base_url/api_key_env). Off by default. |
--slack-webhook |
$SLACK_WEBHOOK_URL |
Slack webhook URL for scan alerts. |
--jira-url |
$JIRA_URL |
Jira Server/Cloud URL for issue creation. |
--jira-email |
$JIRA_EMAIL |
Jira account email for API auth. |
--jira-token |
$JIRA_API_TOKEN |
Jira API token. |
--jira-project |
$JIRA_PROJECT |
Jira project key. |
--init |
N/A | Interactive configuration wizard (writes local config — see Quick Start). |
--validate-policy |
N/A | Validate custom compliance packs and Semgrep rule schemas, then exit. |
Notes:
--trivy/--tfsec/--gitleaksdefault from their env vars (TRIVY_ENABLED,TFSEC_ENABLED,GITLEAKS_ENABLED); when those are unset on the CLI the scanner is off. The GitHub Action enables them by default via the corresponding inputs.- Output emission is controlled by environment variables, not CLI flags:
EMIT_OSCAL,EMIT_COVERAGE,EMIT_SARIF(all defaulttrue). These map to theemit-oscal/emit-coverage/emit-sarifAction inputs.
🧰 pack Subcommands
Manage custom framework packs via the audit-packs pack command space. See the pack management guide for details.
| Subcommand | Usage | Description |
|---|---|---|
pack init |
audit-packs pack init <pack-id> [--output-dir packs] |
Scaffold a new framework pack directory. |
pack validate |
audit-packs pack validate <pack-path> |
Validate a pack directory against the schema. |
pack test |
audit-packs pack test <pack-path> --fixture <dir> [--scanners-dir <dir>] |
Dry-run pack mappings against scan fixtures. |
pack publish |
audit-packs pack publish <pack-path> [--output-dir .] |
Package a pack into a distributable tarball. |
pack install |
audit-packs pack install <source> [--output-dir <dir>] |
Install a pack from a URL, owner/repo@version, or local tarball. |
🌐 GitHub Action Integration
audit-packs is designed to run seamlessly in GitHub Action pipelines to block compliance regressions on pull requests.
# .github/workflows/compliance-audit.yml
name: Compliance Audit
on:
pull_request:
jobs:
audit:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write # Required to post the PR summary comment
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Required for diff-only scanning
- name: Run Audit Packs compliance check
uses: prakharsingh/audit-packs@v0
with:
frameworks: nist-800-53,soc2
fail-on: high
Action Inputs
Generated from action.yml. Input names use hyphens (GitHub Action convention) and may differ from the CLI flag names above.
| Input | Default | Description |
|---|---|---|
frameworks |
Required | Newline/comma-separated framework IDs to assess. |
min-confidence |
0.70 |
Composite score threshold (0.0–1.0); findings below are suppressed in enforce mode. Maps to the --confidence-threshold CLI flag. |
adjudication-mode |
off |
off (no LLM calls), advisory (score shown, nothing suppressed), or enforce (suppress below threshold). |
models-config |
audit-models.yaml |
Path to a model-routing YAML file (provider, model, base_url, api_key_env per role). Falls back to built-in defaults if absent. |
detector-model |
"" |
Override the detector role's model (sets DETECTOR_MODEL). |
verifier-model |
"" |
Override the verifier role's model (sets VERIFIER_MODEL). |
adversarial-model |
"" |
Override the challenger role's model (sets CHALLENGER_MODEL). |
judge-model |
"" |
Override the consensus role's model (sets CONSENSUS_MODEL). |
codeql-sarif |
"" |
Directory of CodeQL SARIF files from github/codeql-action/analyze. Skipped if empty. |
ast-rules |
ast-rules |
Path to custom AST rule modules (executed by the built-in AST engine). |
trivy-enabled |
true |
Run Trivy filesystem scan for IaC misconfigs. |
trivy-image |
"" |
Docker image tag to scan with Trivy. Empty = skip image scan. |
tfsec-enabled |
true |
Run tfsec for Terraform security checks. |
gitleaks-enabled |
true |
Run gitleaks to detect hardcoded secrets and credentials. |
fail-on |
high |
Minimum severity that blocks the PR: low, medium, high, critical. |
base-ref |
origin/main |
Git ref to diff against. |
scan-mode |
both |
diff, full, or both. |
emit-oscal |
true |
Emit OSCAL assessment-results JSON. |
emit-coverage |
true |
Emit coverage matrix as Markdown and HTML. |
emit-sarif |
true |
Emit aggregate SARIF for upload. |
seo-title |
Audit Packs Control Coverage Matrix |
HTML title for coverage.html. |
seo-description |
Compliance control coverage report generated by audit-packs. |
HTML meta description for coverage.html. |
seo-canonical-url |
"" |
Canonical URL for coverage.html when published. |
allow-repo-config |
false |
Trust repo-supplied config (ast-rules/, scanner plugins, audit-models.yaml base_url/api_key_env). The scanned repo is untrusted by default. |
redact-engines |
"" |
Comma-separated engines whose raw output is fully masked before any output sink. Masking is applied post-adjudication — the LLM still sees raw evidence for these engines. |
github-token |
${{ github.token }} |
Token used to post PR review comments. |
pr-number |
triggering PR | Pull request number to comment on. Empty on non-PR events. |
Action Outputs
Exposed as GitHub Action step outputs (from action.yml).
| Output | Description |
|---|---|
oscal-path |
Workspace-relative path to the OSCAL JSON output file. |
coverage-md-path |
Workspace-relative path to the Markdown coverage matrix. |
coverage-html-path |
Workspace-relative path to the HTML coverage matrix. |
sarif-path |
Workspace-relative path to the aggregate SARIF file. |
📊 Supported Compliance Frameworks
Compliance frameworks are defined as declarative YAML packs. The following packs are supported:
| Framework | Pack ID | Automated Controls |
|---|---|---|
| NIST SP 800-53 Rev 5 | nist-800-53 |
21 (Canonical baseline) |
| SOC 2 Type II | soc2 |
39 (Technical criteria) |
| ISO/IEC 27001:2022 | iso27001 |
7 |
| PCI-DSS v4.0 | pci-dss |
8 |
| FedRAMP Moderate | fedramp |
8 |
| HIPAA Security Rule | hipaa |
6 |
| GDPR | gdpr |
5 |
| Custom Org-Policy | org-policy |
Configurable |
📤 Output Artifacts
- PR Summary Comment: Posts one updatable summary comment containing control mappings and evidence for all diff-filtered findings in the PR.
- OSCAL Assessment Results: Machine-readable
oscal.jsoncompliant with NIST SP 800-53 GRC tooling workflows. - Coverage Reports: Beautiful
coverage.mdandcoverage.htmlcontaining an audit-ready compliance matrix. - Aggregated SARIF: A combined
audit-packs.sariffile containing all scanner findings mapped to controls.
📦 Ecosystem Architecture
audit-packs is built as a modular ecosystem consisting of five Python packages:
| Package | PyPI Link | Role | Standalone? |
|---|---|---|---|
audit-packs |
Main CLI & Action entrypoint | Yes | |
audit-packs-core |
Primitives, diff parsing, normalization | No | |
audit-packs-mapping |
Compliance pack loader & OSCAL exporter | No | |
audit-packs-evidence |
Evidence collectors & heuristic agents | No | |
audit-packs-ai |
LLM consensus & confidence scoring | No |
🤝 Contributing & Backtrack Links
- GitHub Repository: https://github.com/prakharsingh/audit-packs
- Issue Tracker: https://github.com/prakharsingh/audit-packs/issues
- Contributing Guidelines: Refer to the repository CONTRIBUTING.md.
📄 License
This project is licensed under the Apache-2.0 License. See the LICENSE file in the main repository for details.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file audit_packs-0.8.0.tar.gz.
File metadata
- Download URL: audit_packs-0.8.0.tar.gz
- Upload date:
- Size: 43.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.1.0 CPython/3.13.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
efaf2cc46ef6e419f01c771c6d79c9893bda7a80b5f3615062373556547c203e
|
|
| MD5 |
d4a1705ed6cbbda496fe6466b2a34f68
|
|
| BLAKE2b-256 |
385a6d74ffb7b726bf473c6a595c3bd2cd883eaa44bf7ba174f0182c9ac3efd3
|
File details
Details for the file audit_packs-0.8.0-py3-none-any.whl.
File metadata
- Download URL: audit_packs-0.8.0-py3-none-any.whl
- Upload date:
- Size: 46.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.1.0 CPython/3.13.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
0b2f3fd2a7907a10f10e255aa53216d1373ef7d2542317d968580f778d8882e5
|
|
| MD5 |
69a80ef869bc5de6ea02edcb553f92bd
|
|
| BLAKE2b-256 |
76846e11e303404d2bbafd2ab460b1557f3517298b84bcb2c75af8e1122efddf
|