certisigma-census 0.4.0

Cryptographic file inventory and exfiltration detection — powered by CertiSigma

CertiSigma Census

Cryptographic file inventory and exfiltration detection — powered by CertiSigma.

Census scans directories, computes SHA-256 hashes, attests them via the CertiSigma API (three-layer cryptographic proof: ECDSA T0, qualified TSA T1, Bitcoin T2), and maintains a local manifest. When suspect files surface, Census compares their hashes against the registry to prove — with cryptographic certainty — whether they match inventoried assets.

Installation

pip install certisigma-census

# With watch mode (filesystem monitoring)
pip install certisigma-census[watch]

# With PDF report generation
pip install certisigma-census[report]

# Everything
pip install certisigma-census[watch,report]

Requires Python 3.10+.

Quick Start

1. Inventory scan

export CERTISIGMA_API_KEY=cs_...

# Scan a directory and attest all file hashes
census scan /path/to/sensitive-files --source inventory-hr

# Dry run — hash only, no attestation
census scan /path/to/files --dry-run

# Scan only PDFs and Word docs, skip files over 100 MB
census scan /data --include "*.pdf" --include "*.docx" --max-size 100M

# Resume an interrupted scan
census scan /data --source quarterly --manifest inventory.db --resume

This produces a .census-manifest.db (SQLite) mapping each hash to its file path, size, and attestation metadata.

2. Breach comparison

# Compare suspect files against the CertiSigma registry
census compare /path/to/suspect-files --manifest /path/to/.census-manifest.db

# Save report as JSON or CSV
census compare /suspect --output report.json
census compare /suspect --output report.csv

Exit code: 0 if no matches, 1 if matches found.

3. Manifest status and export

# Show summary
census status /path/to/.census-manifest.db

# Export manifest as CSV for compliance reporting
census export manifest.db --format csv --output inventory.csv

# Export as JSON
census export manifest.db --format json --output inventory.json

4. Evidence verification

# Verify a hash against the CertiSigma registry
census verify a1b2c3d4e5f67890...

# Verify a file (hash it first, then check)
census verify /path/to/document.pdf --file

# Save OpenTimestamps proof
census verify a1b2c3... --save-ots proof.ots

No API key required — all verification endpoints are public.

5. Integrity check

# Check files against manifest baseline
census integrity manifest.db

# Strict mode: exit 1 on any discrepancy
census integrity manifest.db --strict

100% local operation — no API calls, no network needed.

6. Forensic reports

# HTML report (always available, zero dependencies)
census report manifest.db -o report.html

# PDF report (requires: pip install certisigma-census[report])
census report manifest.db -o report.pdf --evidence --integrity

# Evidence bundle: ZIP with report + OTS proofs + checksums
census report manifest.db -o bundle.zip --bundle --evidence

7. Watch mode (continuous monitoring)

# Watch a directory for changes and attest new/modified files
census watch /path/to/files --source "production"

# Dry run — hash only, no attestation
census watch /data --dry-run

# Network mount — use polling
census watch /mnt/share --polling --poll-interval 10

Requires: pip install certisigma-census[watch]

How It Works

1. Scan — Census walks the directory, computes SHA-256 for each file (streamed, constant memory), and builds a local manifest.
2. Attest — Hashes are sent in batches (up to 100 per call) to the CertiSigma API. Each hash receives a three-layer cryptographic proof (T0 ECDSA signature, T1 qualified TSA timestamp, T2 Bitcoin anchor).
3. Compare — Suspect files are hashed and verified against the registry via POST /verify/batch. Matches prove the file was previously inventoried, regardless of filename or directory structure changes.

The original file content never leaves the client. Only SHA-256 hashes are transmitted.

Features

Feature	Description	Docs
File filters	--include, --exclude globs; --min-size, --max-size	scanning.md
Resume scans	--resume skips unchanged files, preserves attestation state	scanning.md
CSV/JSON export	Compare reports and manifest export in both formats	comparison.md
Retry with backoff	Automatic retry on 429/5xx with exponential backoff	retry-and-resilience.md
Structured logging	--log-format json for SIEM/ELK integration	logging.md
Progress bars	Visual feedback for scan, attest, and compare operations	scanning.md
SQLite manifest	WAL mode, indexed lookups, auto-migration from JSON	manifest.md
Watch mode	Continuous filesystem monitoring with batch attestation	watching.md
Evidence verification	Full T0/T1/T2 chain, OTS proof export	evidence.md
Integrity check	Tamper detection against manifest baseline	integrity.md
Forensic reports	HTML, PDF, evidence bundles (ZIP)	reporting.md

Full documentation: docs/features/

CLI Reference

Global options

Option	Description
-v / --verbose	Enable debug logging
--log-format text|json	Log output format (default: text)
--version	Show version

census scan

Option	Description
--source LABEL	Source label for attestations
--manifest PATH	Manifest output path (default: <dir>/.census-manifest.db)
--api-key KEY	API key (or set CERTISIGMA_API_KEY)
--base-url URL	Override API base URL
--dry-run	Hash only, no attestation
--resume	Resume interrupted scan
--include GLOB	Include files matching pattern (repeatable)
--exclude GLOB	Exclude files matching pattern (repeatable)
--min-size SIZE	Skip files smaller than SIZE (e.g. 1K, 10M)
--max-size SIZE	Skip files larger than SIZE (default: 5G)

census compare

Option	Description
--manifest PATH	Local manifest for cross-referencing
--output PATH	Save report (.json or .csv by extension)
--include/--exclude/--min-size/--max-size	Same filters as scan

census export

Option	Description
--format csv|json	Output format (default: csv)
--output PATH	Output file (default: stdout)

census verify

Option	Description
--file	Treat argument as a file path (hash it first)
--save-ots PATH	Save OTS proof to this path
--json	Machine-readable JSON output
--api-key KEY	API key (optional for verify)
--base-url URL	Override API base URL

census integrity

Option	Description
--json	Machine-readable JSON output
--output PATH	Save results (.csv or .json by extension)
--strict	Exit with code 1 on any discrepancy

census report

Option	Description
-o/--output PATH	Output file (.html, .pdf, or .zip) required
--evidence	Fetch T0/T1/T2 evidence chain for attested files
--integrity	Run integrity check and include results
--bundle	Generate evidence bundle (ZIP)
--api-key KEY	API key (needed only with --evidence)

census status

Takes a manifest path as argument. No additional options.

census watch

Option	Description
--debounce SECS	Quiet period before processing (default: 2.0s)
--batch-interval SECS	Max time between attestation batches (default: 30s)
--scan-on-start / --no-scan-on-start	Baseline scan before watching (default: on)
--on-delete ignore|mark|remove	Action on file deletion (default: ignore)
--polling	Use PollingObserver for NFS/CIFS mounts
--poll-interval SECS	Polling interval (default: 5s)
--source/--manifest/--api-key/--dry-run	Same as census scan
--include/--exclude/--min-size/--max-size	Same filters as scan

Requires: pip install certisigma-census[watch]

Dependencies

• certisigma — Official CertiSigma Python SDK
• click — CLI framework

Optional:

• watchdog — Filesystem monitoring (only for census watch)
• fpdf2 — PDF report generation (only for census report with .pdf output)

License

MIT — Ten Sigma Sagl