ski-audit-ledger 3.0.3

Audit ledger management tool for the SKI Framework.

audit-ledger

Install from PyPI: pip install ski-audit-ledger (publishing starts with the first release after June 2026).

⚠ STATUS: EARLY ALPHA (v0.1.0a0). Alpha-quality tooling. See the repo root README.md for the project-wide status.

Manage and verify the immutable audit ledger for SKI Framework compliance monitoring.

v2.1 highlights

• Real verify_integrity(). Recomputes every entry's hash from the documented canonical serialization. Catches in-place tampering, not just chain-linkage breakage.
• Real backup. Invokes pg_dump and verifies the dump with pg_restore --list. No more stub.
• Five-verdict taxonomy. --verdict-filter now accepts NULL_UNMAPPED and NULL_STALE (the pre-v2.1 NULL is gone).
• No confidence_level. Removed entirely from the data model.
• Append-only enforcement lives at the database layer (reference-implementation/src/ledger/append_only.sql). UPDATE, DELETE, and TRUNCATE on ledger_entries are refused by triggers.

Overview

The audit ledger is a hash-chained, immutable record of all compliance verdicts generated by the SKI Model runtime. This tool provides utilities for:

• Verification — Verify ledger integrity and detect tampering
• Backup — Create secure backups with cryptographic verification
• Export — Extract ledger data for analysis and reporting
• Reporting — Generate compliance reports and summaries

Features

Ledger Integrity Verification

Verify the complete hash chain of ledger entries to ensure no records have been tampered with:

audit-ledger verify --ledger-db postgresql://user:pass@localhost/ledger

Output:

Ledger Integrity Verification
├── Total entries: 4,327
├── Chain integrity: ✓ VERIFIED
├── Hash verification: ✓ 4,327/4,327 valid
├── Timestamp order: ✓ VALID
└── Last verified: 2026-05-21T15:30:00Z

Secure Backup

Create verified backups with integrity checking:

audit-ledger backup \
  --source postgresql://user:pass@localhost/ledger \
  --output ./ledger-backup-2026-05-21.sql \
  --verify

Data Export

Export ledger entries for analysis and reporting:

# Export to JSON
audit-ledger export \
  --source postgresql://user:pass@localhost/ledger \
  --output ledger-data.json \
  --format json

# Export to CSV
audit-ledger export \
  --source postgresql://user:pass@localhost/ledger \
  --output ledger-data.csv \
  --format csv \
  --fields "id,timestamp,verdict,rule_id,telemetry_id"

Compliance Reporting

Generate formatted compliance reports:

audit-ledger report \
  --source postgresql://user:pass@localhost/ledger \
  --start-date 2026-01-01 \
  --end-date 2026-05-31 \
  --output compliance-report.html

Installation

From Source

cd tools/audit-ledger
pip install -e .

Requirements

• Python 3.8+
• PostgreSQL 12+ (for ledger database)
• psycopg2 (PostgreSQL adapter)
• sqlalchemy (ORM)

Quick Start

1. Verify Ledger Integrity

After running compliance monitoring for a period, verify the ledger has not been tampered with:

audit-ledger verify \
  --ledger-db postgresql://compliance:securepass@localhost:5432/ski_ledger \
  --verbose

2. Create a Backup

Before major system changes, create a verified backup:

audit-ledger backup \
  --source postgresql://compliance:securepass@localhost:5432/ski_ledger \
  --output ./backups/ledger-$(date +%Y%m%d).sql.gz \
  --compress \
  --verify

3. Generate Monthly Report

At month-end, generate a compliance summary:

audit-ledger report \
  --source postgresql://compliance:securepass@localhost:5432/ski_ledger \
  --start-date 2026-05-01 \
  --end-date 2026-05-31 \
  --output monthly-report-2026-05.html \
  --include-verdicts \
  --include-violations

4. Export for Audit

Export ledger data for external audit:

audit-ledger export \
  --source postgresql://compliance:securepass@localhost:5432/ski_ledger \
  --output audit-export.json \
  --format json \
  --date-range 2026-01-01,2026-05-31

Command Reference

verify

Verify ledger integrity and hash chain validity.

audit-ledger verify [OPTIONS]

Options:
  --ledger-db TEXT          PostgreSQL connection string [required]
  --verbose                 Show detailed verification output
  --check-timestamps        Verify timestamp ordering
  --repair                  Attempt to repair minor issues (use with caution)
  --output TEXT             Write results to file
  --help                    Show help message

backup

Create a backup of the ledger database.

audit-ledger backup [OPTIONS]

Options:
  --source TEXT             PostgreSQL connection string [required]
  --output TEXT             Output file path [required]
  --compress                Gzip compress the backup
  --verify                  Verify backup integrity after creation
  --encryption-key TEXT     Encrypt backup (optional)
  --help                    Show help message

export

Export ledger entries for analysis.

audit-ledger export [OPTIONS]

Options:
  --source TEXT             PostgreSQL connection string [required]
  --output TEXT             Output file path [required]
  --format TEXT             json|csv|jsonl (default: json)
  --fields TEXT             Comma-separated field names to include
  --date-range TEXT         Start,end dates (YYYY-MM-DD,YYYY-MM-DD)
  --verdict-filter TEXT     Filter by verdict (CLEAR|FLAG|NULL|DISCRETIONARY)
  --rule-id TEXT            Filter by rule ID
  --limit INTEGER           Maximum entries to export
  --help                    Show help message

report

Generate a compliance report from ledger data.

audit-ledger report [OPTIONS]

Options:
  --source TEXT             PostgreSQL connection string [required]
  --output TEXT             Output HTML file path
  --start-date TEXT         Report start date (YYYY-MM-DD)
  --end-date TEXT           Report end date (YYYY-MM-DD)
  --include-verdicts        Include verdict summary statistics
  --include-violations      Include violation details
  --include-timeline        Include timeline visualization
  --include-audit-trail     Include audit trail verification
  --title TEXT              Report title
  --organization TEXT       Organization name for report
  --help                    Show help message

Ledger Structure

The audit ledger stores compliance verdicts in a hash-chained format:

CREATE TABLE ledger_entries (
    id BIGSERIAL PRIMARY KEY,
    sequence_number BIGINT UNIQUE NOT NULL,
    previous_hash CHAR(64) NOT NULL,
    entry_hash CHAR(64) NOT NULL UNIQUE,
    timestamp TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    verdict TEXT NOT NULL CHECK (verdict IN (
        'CLEAR','FLAG','NULL_UNMAPPED','NULL_STALE','DISCRETIONARY'
    )),
    telemetry_id TEXT NOT NULL,
    telemetry_hash CHAR(64) NOT NULL,
    rule_id TEXT,
    knowledge_graph_version TEXT,
    ski_model_version TEXT NOT NULL,
    reasoning TEXT,
    track TEXT CHECK (track IS NULL OR track IN ('symbolic','llm')),
    escalation_status TEXT,
    escalation_notes TEXT,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

v2.1 schema notes. milm_version is renamed to ski_model_version. confidence_level has been removed entirely — B3.1 prohibits confidence scores in the ledger. The verdict CHECK constraint enforces the five-verdict taxonomy.

Each entry's entry_hash is SHA-256 of the canonical serialization documented in src/audit_ledger/canonical.py. The canonical payload includes sequence_number, previous_hash, timestamp, verdict, telemetry_id, telemetry_hash, rule_id, kg_version, ski_model_version, reasoning, and track. Modifying any one of these fields invalidates that entry's entry_hash.

Append-only enforcement is in reference-implementation/src/ledger/append_only.sql — BEFORE UPDATE, BEFORE DELETE, and BEFORE TRUNCATE triggers refuse any modification at the database layer.

Integrity verification process

The verify command checks, for every row:

1. Chain linkage — each row's previous_hash equals the prior row's entry_hash.
2. Entry-hash recomputation — re-derive entry_hash from the canonical payload and compare with what's stored. Catches in-place tampering that chain-linkage-only checks miss.
3. Sequence continuity — sequence_number has no gaps.
4. Timestamp ordering — timestamps are monotonic.

Example verification output:

Ledger Integrity Verification Report
====================================

Database: postgresql://localhost/ski_ledger
Verification Date: 2026-05-21T15:30:00Z

Chain Analysis:
  Total Entries: 4,327
  Sequence Range: 1 - 4,327
  Time Range: 2026-01-01 to 2026-05-21

Integrity Checks:
  ✓ Chain continuity: All 4,327 entries form valid chain
  ✓ Hash verification: 4,327/4,327 entries (100%)
  ✓ Timestamp ordering: All timestamps chronologically valid
  ✓ Data consistency: All references valid

Verdict Distribution:
  CLEAR: 3,821 (88.3%)
  FLAG: 412 (9.5%)
  DISCRETIONARY: 94 (2.2%)
  NULL: 0 (0.0%)

Risk Assessment:
  ✓ No signs of tampering
  ✓ No missing entries
  ✓ No invalid references
  Status: VERIFIED

Recommendation: Ledger integrity confirmed. Safe for regulatory reporting.

Backup & Recovery

Creating a Backup

# Standard backup
audit-ledger backup \
  --source postgresql://compliance:pass@localhost/ski_ledger \
  --output ./ledger-backup-2026-05-21.sql

# Compressed with verification
audit-ledger backup \
  --source postgresql://compliance:pass@localhost/ski_ledger \
  --output ./ledger-backup-2026-05-21.sql.gz \
  --compress \
  --verify

Verifying a Backup

# Restore to test database
psql test_ledger < ledger-backup-2026-05-21.sql

# Verify the restored ledger
audit-ledger verify \
  --ledger-db postgresql://user:pass@localhost/test_ledger

Export Examples

Export All Violations

audit-ledger export \
  --source postgresql://user:pass@localhost/ski_ledger \
  --output violations.json \
  --format json \
  --verdict-filter FLAG

Output:

{
  "export_date": "2026-05-21T15:30:00Z",
  "entry_count": 412,
  "entries": [
    {
      "id": 1543,
      "timestamp": "2026-05-15T10:30:00Z",
      "verdict": "FLAG",
      "rule_id": "e001",
      "telemetry_id": "TEL_2026_05_001",
      "reasoning": "SO2 emissions 125 ppm exceeds limit of 100 ppm"
    },
    ...
  ]
}

Export Monthly Audit Trail

audit-ledger export \
  --source postgresql://user:pass@localhost/ski_ledger \
  --output may-2026-audit.csv \
  --format csv \
  --date-range 2026-05-01,2026-05-31 \
  --fields "timestamp,verdict,rule_id,reasoning,escalation_status"

Reporting

Generate Monthly Compliance Report

audit-ledger report \
  --source postgresql://user:pass@localhost/ski_ledger \
  --start-date 2026-05-01 \
  --end-date 2026-05-31 \
  --output may-2026-compliance.html \
  --include-verdicts \
  --include-violations \
  --include-timeline \
  --organization "Acme Energy Corp"

Report includes:

• Executive summary of compliance status
• Verdict distribution charts
• Violations identified and status
• Remediation actions taken
• Timeline of escalations
• Audit trail verification status
• Regulatory filing readiness

Security Considerations

Database Security

The PostgreSQL ledger database should be:

• Behind a firewall (not internet-accessible)
• Using strong authentication (not default credentials)
• Requiring SSL/TLS for all connections
• Backed up regularly
• Monitored for unauthorized access

Backup Security

Backups should be:

• Encrypted if stored off-site
• Stored securely (not in version control)
• Verified before relying on them
• Tested for recovery capability

Access Control

Ledger access should be restricted to:

• Compliance officers
• System administrators
• Authorized auditors

Troubleshooting

Ledger Verification Fails

If verification reports issues:

# Get detailed diagnostic output
audit-ledger verify \
  --ledger-db postgresql://user:pass@localhost/ski_ledger \
  --verbose

# Attempt automatic repair (use with caution)
audit-ledger verify \
  --ledger-db postgresql://user:pass@localhost/ski_ledger \
  --repair

Export Takes Too Long

For large ledgers, use filters:

# Export only recent violations
audit-ledger export \
  --source postgresql://user:pass@localhost/ski_ledger \
  --output recent-flags.json \
  --verdict-filter FLAG \
  --date-range 2026-05-01,2026-05-31

Backup Verification Fails

Restore to a test database and verify:

# Create test database
createdb test_ledger

# Restore backup
psql test_ledger < backup.sql

# Verify restoration
audit-ledger verify --ledger-db postgresql://user:pass@localhost/test_ledger

API Usage

Python API

from audit_ledger import Ledger

# Connect to ledger
ledger = Ledger("postgresql://user:pass@localhost/ski_ledger")

# Verify integrity
result = ledger.verify_integrity()
print(f"Ledger valid: {result.is_valid}")
print(f"Total entries: {result.total_entries}")

# Export entries
entries = ledger.export_entries(
    start_date="2026-05-01",
    end_date="2026-05-31",
    verdict_filter="FLAG"
)

# Generate report
report = ledger.generate_report(
    start_date="2026-05-01",
    end_date="2026-05-31",
    include_verdicts=True,
    include_violations=True
)
ledger.save_report(report, "compliance-report.html")

Production Checklist

• Configure PostgreSQL with strong authentication
• Enable SSL/TLS for database connections
• Set up automated backup schedule
• Verify backup recovery procedures
• Document ledger access controls
• Train staff on ledger commands
• Set up monitoring for ledger database
• Plan audit trail retention policy
• Test export/reporting workflows
• Schedule regular integrity verification
• Document escalation procedures for verification failures

Support

For issues or questions:

• Check individual command help: audit-ledger COMMAND --help
• Review ledger structure documentation
• Open an issue on GitHub
• Contact your compliance team

---

Note: The audit ledger is critical for regulatory compliance. All operations should be performed by authorized personnel only. Always verify backups and test recovery procedures before relying on them.