audit-packs
An evidence-first Compliance Intelligence Engine that transforms security scanner findings into standardized, evidence-backed compliance artifacts (an evidence-backed PR summary comment, OSCAL assessment results, SARIF, and control coverage reports).
🔍 The Problem
Modern security scanners (such as Checkov, Semgrep, Trivy, tfsec, and gitleaks) are highly effective at identifying infrastructure-as-code (IaC) misconfigurations and software vulnerabilities. However, they are scanner-centric and do not directly answer the critical questions that governance, risk, and compliance (GRC) teams or auditors ask:
- Which compliance controls (e.g., NIST 800-53, SOC 2, HIPAA, GDPR) are affected by this finding?
- Where is the exact code/configuration evidence to prove compliance or violation?
- How do we avoid false-positive alerts bloating our engineering workflows?
💡 The Solution
audit-packs bridges the gap by providing a scanner-agnostic mapping, enrichment, and consensus layer. Rather than replacing existing detection tools, it takes their output (typically via SARIF), normalizes it, enriches it with codebase/git evidence context, maps the findings to GRC control requirements, and optionally filters noise using an AI consensus ensemble.
📦 Installation
To run audit-packs locally as a CLI tool:
# Install audit-packs CLI via pip
pip install audit-packs
# Or install in an isolated environment via pipx (recommended)
pipx install audit-packs
Note: Detection is delegated to best-in-class open-source engines. For the scanners to run, ensure they are installed on your system path, or inject them into the
pipxenvironment:pipx inject audit-packs checkov semgrep
🚀 Quick Start
1. Initialize Configuration
Bootstrap your repository with local config files — an interactive wizard writes an audit-models.yaml model-router file and records your framework selection. It downloads nothing; the compliance packs ship bundled with the CLI:
audit-packs --init
2. Run a Compliance Scan
Scan your workspace and map findings to NIST 800-53 and SOC 2 frameworks:
audit-packs --frameworks nist-800-53,soc2
📋 CLI Flags
These flags belong to the audit-packs command-line scanner (argparse). Run audit-packs --help for the authoritative list.
| Flag | Default | Description |
|---|---|---|
--frameworks |
nist-800-53 |
Comma/newline-separated framework pack IDs to evaluate (e.g., nist-800-53,soc2). |
--fail-on |
high |
Minimum finding severity that exits non-zero: low, medium, high, critical. |
--scan-mode |
both |
Scan scope: diff (PR-changed lines only), full (entire posture), or both. |
--base-ref |
origin/main |
Base git ref to diff against for diff-mode scanning. |
--workspace |
$GITHUB_WORKSPACE or . |
Repository workspace directory to scan. |
--packs-dir |
bundled | Directory of custom compliance pack YAMLs. |
--rules-path |
bundled | Directory of Semgrep rule files. |
--scanners-dir |
none | Directory of declarative scanner plugin definitions. |
--adjudication-mode |
off |
AI consensus adjudication: off, advisory (score only), or enforce (suppress below threshold). |
--confidence-threshold |
0.70 |
Composite confidence threshold (0.0–1.0) applied under enforce mode. |
--codeql-sarif |
none | Directory of CodeQL SARIF files to ingest (skipped if absent). |
--trivy / --no-trivy |
from env | Enable/disable Trivy filesystem scanning. |
--trivy-image |
none | Docker image tag to scan with Trivy. |
--tfsec / --no-tfsec |
from env | Enable/disable tfsec Terraform scanning. |
--gitleaks / --no-gitleaks |
from env | Enable/disable gitleaks secret detection. |
--allow-repo-config |
false |
Trust repo-supplied config (ast-rules/, scanner plugins, model base_url/api_key_env). Off by default. |
--slack-webhook |
$SLACK_WEBHOOK_URL |
Slack webhook URL for scan alerts. |
--jira-url |
$JIRA_URL |
Jira Server/Cloud URL for issue creation. |
--jira-email |
$JIRA_EMAIL |
Jira account email for API auth. |
--jira-token |
$JIRA_API_TOKEN |
Jira API token. |
--jira-project |
$JIRA_PROJECT |
Jira project key. |
--init |
N/A | Interactive configuration wizard (writes local config — see Quick Start). |
--validate-policy |
N/A | Validate custom compliance packs and Semgrep rule schemas, then exit. |
Notes:
--trivy/--tfsec/--gitleaksdefault from their env vars (TRIVY_ENABLED,TFSEC_ENABLED,GITLEAKS_ENABLED); when those are unset on the CLI the scanner is off. The GitHub Action enables them by default via the corresponding inputs.- Output emission is controlled by environment variables, not CLI flags:
EMIT_OSCAL,EMIT_COVERAGE,EMIT_SARIF(all defaulttrue). These map to theemit-oscal/emit-coverage/emit-sarifAction inputs.
🧰 pack Subcommands
Manage custom framework packs via the audit-packs pack command space. See the pack management guide for details.
| Subcommand | Usage | Description |
|---|---|---|
pack init |
audit-packs pack init <pack-id> [--output-dir packs] |
Scaffold a new framework pack directory. |
pack validate |
audit-packs pack validate <pack-path> |
Validate a pack directory against the schema. |
pack test |
audit-packs pack test <pack-path> --fixture <dir> [--scanners-dir <dir>] |
Dry-run pack mappings against scan fixtures. |
pack publish |
audit-packs pack publish <pack-path> [--output-dir .] |
Package a pack into a distributable tarball. |
pack install |
audit-packs pack install <source> [--output-dir <dir>] |
Install a pack from a URL, owner/repo@version, or local tarball. |
🌐 GitHub Action Integration
audit-packs is designed to run seamlessly in GitHub Action pipelines to block compliance regressions on pull requests.
# .github/workflows/compliance-audit.yml
name: Compliance Audit
on:
pull_request:
jobs:
audit:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write # Required to post the PR summary comment
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Required for diff-only scanning
- name: Run Audit Packs compliance check
uses: prakharsingh/audit-packs@v0
with:
frameworks: nist-800-53,soc2
fail-on: high
Action Inputs
Generated from action.yml. Input names use hyphens (GitHub Action convention) and may differ from the CLI flag names above.
| Input | Default | Description |
|---|---|---|
frameworks |
Required | Newline/comma-separated framework IDs to assess. |
min-confidence |
0.70 |
Composite score threshold (0.0–1.0); findings below are suppressed in enforce mode. Maps to the --confidence-threshold CLI flag. |
adjudication-mode |
off |
off (no LLM calls), advisory (score shown, nothing suppressed), or enforce (suppress below threshold). |
models-config |
audit-models.yaml |
Path to a model-routing YAML file (provider, model, base_url, api_key_env per role). Falls back to built-in defaults if absent. |
detector-model |
"" |
Override the detector role's model (sets DETECTOR_MODEL). |
verifier-model |
"" |
Override the verifier role's model (sets VERIFIER_MODEL). |
adversarial-model |
"" |
Override the challenger role's model (sets CHALLENGER_MODEL). |
judge-model |
"" |
Override the consensus role's model (sets CONSENSUS_MODEL). |
codeql-sarif |
"" |
Directory of CodeQL SARIF files from github/codeql-action/analyze. Skipped if empty. |
ast-rules |
ast-rules |
Path to custom AST rule modules (executed by the built-in AST engine). |
trivy-enabled |
true |
Run Trivy filesystem scan for IaC misconfigs. |
trivy-image |
"" |
Docker image tag to scan with Trivy. Empty = skip image scan. |
tfsec-enabled |
true |
Run tfsec for Terraform security checks. |
gitleaks-enabled |
true |
Run gitleaks to detect hardcoded secrets and credentials. |
fail-on |
high |
Minimum severity that blocks the PR: low, medium, high, critical. |
base-ref |
origin/main |
Git ref to diff against. |
scan-mode |
both |
diff, full, or both. |
emit-oscal |
true |
Emit OSCAL assessment-results JSON. |
emit-coverage |
true |
Emit coverage matrix as Markdown and HTML. |
emit-sarif |
true |
Emit aggregate SARIF for upload. |
seo-title |
Audit Packs Control Coverage Matrix |
HTML title for coverage.html. |
seo-description |
Compliance control coverage report generated by audit-packs. |
HTML meta description for coverage.html. |
seo-canonical-url |
"" |
Canonical URL for coverage.html when published. |
allow-repo-config |
false |
Trust repo-supplied config (ast-rules/, scanner plugins, audit-models.yaml base_url/api_key_env). The scanned repo is untrusted by default. |
redact-engines |
"" |
Comma-separated engines whose raw output is fully masked before any output sink. Masking is applied post-adjudication — the LLM still sees raw evidence for these engines. |
github-token |
${{ github.token }} |
Token used to post PR review comments. |
pr-number |
triggering PR | Pull request number to comment on. Empty on non-PR events. |
Action Outputs
Exposed as GitHub Action step outputs (from action.yml).
| Output | Description |
|---|---|
oscal-path |
Workspace-relative path to the OSCAL JSON output file. |
coverage-md-path |
Workspace-relative path to the Markdown coverage matrix. |
coverage-html-path |
Workspace-relative path to the HTML coverage matrix. |
sarif-path |
Workspace-relative path to the aggregate SARIF file. |
📊 Supported Compliance Frameworks
Compliance frameworks are defined as declarative YAML packs. The following packs are supported:
| Framework | Pack ID | Automated Controls |
|---|---|---|
| NIST SP 800-53 Rev 5 | nist-800-53 |
21 (Canonical baseline) |
| SOC 2 Type II | soc2 |
39 (Technical criteria) |
| ISO/IEC 27001:2022 | iso27001 |
7 |
| PCI-DSS v4.0 | pci-dss |
8 |
| FedRAMP Moderate | fedramp |
8 |
| HIPAA Security Rule | hipaa |
6 |
| GDPR | gdpr |
5 |
| Custom Org-Policy | org-policy |
Configurable |
📤 Output Artifacts
- PR Summary Comment: Posts one updatable summary comment containing control mappings and evidence for all diff-filtered findings in the PR.
- OSCAL Assessment Results: Machine-readable
oscal.jsoncompliant with NIST SP 800-53 GRC tooling workflows. - Coverage Reports: Beautiful
coverage.mdandcoverage.htmlcontaining an audit-ready compliance matrix. - Aggregated SARIF: A combined
audit-packs.sariffile containing all scanner findings mapped to controls.
📦 Ecosystem Architecture
audit-packs is built as a modular ecosystem consisting of five Python packages:
| Package | PyPI Link | Role | Standalone? |
|---|---|---|---|
audit-packs |
Main CLI & Action entrypoint | Yes | |
audit-packs-core |
Primitives, diff parsing, normalization | No | |
audit-packs-mapping |
Compliance pack loader & OSCAL exporter | No | |
audit-packs-evidence |
Evidence collectors & heuristic agents | No | |
audit-packs-ai |
LLM consensus & confidence scoring | No |
🤝 Contributing & Backtrack Links
- GitHub Repository: https://github.com/prakharsingh/audit-packs
- Issue Tracker: https://github.com/prakharsingh/audit-packs/issues
- Contributing Guidelines: Refer to the repository CONTRIBUTING.md.
📄 License
This project is licensed under the Apache-2.0 License. See the LICENSE file in the main repository for details.
Metadata
Release files for audit-packs 0.9.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| audit_packs-0.9.0.tar.gz | 46.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| audit_packs-0.9.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 95.5 kB
Release files / audit_packs-0.9.0.tar.gz
| Download URL | audit_packs-0.9.0.tar.gz |
|---|---|
| Size | 46.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ad4e0fc7dfe8984fc4d7499b2187939ac8861cbb7d77ce8624bbd79bb7d8dd0b
|
|
BLAKE2b-256 checksum How to use checksums |
ad53481958d1d21e447b93629be63895937748acef813b5047e357fca26dc82f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Release files / audit_packs-0.9.0-py3-none-any.whl
| Download URL | audit_packs-0.9.0-py3-none-any.whl |
|---|---|
| Size | 49.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
83a946ef9338acb2b93dd794c8acb716b9e29ecbf7ad6fc65f3cae11ed825a02
|
|
BLAKE2b-256 checksum How to use checksums |
2aabd278fccd747df6bfe4b5d7e4796b8bfbe422464089a38954998d454e2b54
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|