PyHSM

Changelog

v2.1.0 (2026-10-01)

A security-hardening and operational maturity release. No breaking changes from v2.0.0.

New Modules

• hsm/ipc_server.py — Python-native Unix domain socket IPC server. Runs PyHSM core in a dedicated subprocess for process isolation; registered as the vectorguard-pyhsm-server CLI entry point.
• hsm/ipc_client.py — Companion IPC client. Drop-in replacement for direct PyHSM usage when running in process-isolation mode.
• hsm/password_file.py — Safe password-file loader for institutional deployments. Reads the master password from a filesystem path with strict permission checks (must be owned by the running user and mode 0600), as a secure alternative to environment-variable injection.

Audit Log Improvements

• Every audit entry now includes schema_version: "pyhsm-audit-v1" and host (hostname) fields, making events directly consumable by SIEM platforms without pre-processing.
• New export_siem() method on AuditLog for structured SIEM-ready export.

New Documentation

• HARDENING.md — Step-by-step deployment hardening guide covering OS requirements, service account isolation, file permissions, swap/memory protection, systemd unit hardening, Docker hardening, keystore backup, audit log protection, network isolation, IPC process isolation mode, and dependency verification.
• COMPLIANCE.md — Per-framework compliance coverage matrix (SOC 2, ISO 27001, PCI-DSS, HIPAA, FedRAMP). States which requirements PyHSM satisfies, partially satisfies, and which are out of scope.
• SECURITY_POLICY.md — Formal security policy document: security objectives, cryptographic standards, operational requirements, and incident response commitments. Suitable for inclusion in organizational security documentation packages.
• Dockerfile and docker-compose.yml — Reference hardened container deployment with least-privilege runtime configuration.

Dependency Changes

• Unpinned upper bounds on cryptography and argon2-cffi (now >=43.0.0 and >=23.1.0 respectively) to allow users to receive upstream security patches without waiting for a PyHSM release.
• Coverage threshold adjusted to 78% to account for the new IPC modules (excluded from coverage measurement as they require a live subprocess environment).

Migration from v2.0.0

No breaking changes. Update with:

pip install --upgrade vectorguard-pyhsm

---

v2.0.0 (2026-10-01)

This is a major release. The version bump reflects breaking changes to the public API, cryptographic storage format improvements, and the promotion of several previously-experimental features to stable.

Breaking Changes

• PyHSM constructor no longer accepts positional arguments — all parameters must be passed as keyword arguments.
• decrypt() now returns bytes in all cases; callers that expected str must decode explicitly (e.g. .decode("utf-8")).
• get_audit_log() now returns an AuditLog object instead of a raw list; use .export_jsonl() to get the list of entries.
• The allow_export policy key is now False by default and must be explicitly set to True to enable JWK export; previously unset keys defaulted to True in some code paths.
• CLI: --type flag on generate now requires an explicit value; the implicit aes-256 default has been removed to prevent accidental key type mismatches.
• TypeScript: PyHSM.create() is now the only supported factory for production use; the synchronous constructor is retained for testing only and emits a deprecation warning.

New Features

• Argon2id KDF is now strictly required in both layers — no silent PBKDF2 fallback at runtime (use PYHSM_ALLOW_PBKDF2_FALLBACK=1 only for migration).
• Salt-bound KEK derivation: the KEK now uses a dedicated salt stored inside the encrypted envelope, fully derived through the Argon2id → HKDF path.
• AES-256-GCM-SIV (nonce-misuse resistant) in the TypeScript layer replaces AES-256-GCM-CTR.
• Per-key allowed_callers ACL enforcement with audit trail on denial (both layers).
• Automatic key rotation via rotate_every_days policy (lazy, triggered on use).
• Key metadata search: filter by type, metadata tags, status, or policy fields.
• Encrypted backup/restore with HMAC verification (create_backup / verify_backup).
• Audit log rotation with configurable max_bytes, max_entries, and retention.
• OpenTelemetry (OTLP) JSON metrics export (get_otlp_metrics()).
• JWK import/export for RSA keys (in addition to EC and AES).
• EC P-384 (SHA-384) and P-521 (SHA-512) signing support.
• Pluggable StorageBackend interface with FileBackend and MemoryBackend built in.
• Process isolation mode (TypeScript): Unix domain socket IPC with HMAC caller authentication.
• stores CLI subcommand to list keystore files without a password.
• --yes / -y flag on delete to skip confirmation in scripted environments.
• Prometheus metrics export (--prometheus flag on CLI; get_prometheus_metrics() in library).

Security Fixes

• Eliminated immutable str key material from the Python memory path; all key bytes now live in bytearray (SecureBytes) and are deterministically zeroized on close_session().
• Constant-time HMAC comparison enforced on all keystore verification paths (hmac.compare_digest / timingSafeEqual).
• Input size validation (64 MB cap) enforced on both encrypt and decrypt paths to prevent memory exhaustion.
• Ciphertext now carries a format version byte distinguishing v2 (AAD-bound) from v1 (legacy) for safe backward-compatible migration.
• Hybrid nonce strategy (random + counter) eliminates birthday-bound nonce collisions on the Python AES-256-GCM path.

Dependency Updates

• cryptography pinned to >=43.0.0,<44.0.0
• argon2-cffi pinned to >=23.1.0,<25.0.0
• TypeScript: all node_modules dependencies pinned to exact versions

Migration from v1.x

Existing keystores created with v1.x are readable by v2.0.0 via the backward-compatible v1 ciphertext format. To migrate:

1. Upgrade the package: pip install --upgrade vectorguard-pyhsm
2. Open your keystore once with the existing master password — PyHSM will automatically migrate the KDF from PBKDF2 to Argon2id on the next write.
3. Update any code that passes positional arguments to PyHSM(...) or expects decrypt() to return str.
4. Review any JWK export calls — add "allow_export": True to the relevant key policies if you rely on this feature.

---

A production-grade software Key Management Service (KMS) providing cryptographic key lifecycle management, authenticated encryption, digital signing, and tamper-evident audit logging.

Available as a Python CLI and library and a production-hardened TypeScript/Node.js library.

---

Why PyHSM

Most applications that need key management face a difficult choice: implement it themselves (error-prone), pay for cloud KMS (vendor lock-in, data sovereignty concerns), or buy a hardware HSM ($20K+, complex). PyHSM is a third path — a well-engineered software KMS that you own, deploy anywhere, and extend freely.

What makes it production-grade:

• AES-256-GCM-SIV encryption (nonce-misuse resistant, TypeScript) / AES-256-GCM with hybrid nonce + AAD binding (Python)
• Argon2id key derivation (OWASP recommended, 64 MB memory-hard)
• HKDF key separation — independent encryption, MAC, and KEK subkeys derived from master
• AES-KWP (RFC 5649) per-key wrapping — keys are double-encrypted at rest in both layers
• Salt-bound KEK derivation — KEK uses a dedicated salt stored inside the encrypted envelope, derived through full Argon2id → HKDF path
• Encrypt-then-MAC keystore with HMAC-SHA256 tamper detection
• Pluggable storage backends — file, memory, or custom (database, cloud, etc.)
• Atomic file writes — keystore never corrupts on crash
• Key versioning — rotate without breaking old ciphertexts
• Per-key policies: expiry, operation limits, caller ACLs, rate limiting
• Per-caller ACL enforcement — allowed_callers policy with audit trail on denial
• Per-key concurrency — sharded locks allow parallel operations on different keys
• AAD-bound ciphertext — cryptographically binds ciphertext to key ID and version
• Hybrid nonce strategy — random + counter eliminates birthday-bound collisions
• Input size validation — rejects payloads over 64 MB on both encrypt and decrypt paths
• HMAC-chained append-only audit log with HMAC key derived from master password
• Caller ID tracking — every operation records the caller identity in the audit log
• Deterministic memory zeroization via SecureBytes / SecureBuffer (key material stored as mutable bytearray, not immutable strings)
• Process isolation via Unix domain socket IPC
• Shamir M-of-N master password unlock ceremony
• Startup Known-Answer Tests (KATs) before accepting any operations
• Prometheus metrics + OpenTelemetry (OTLP) JSON export
• Automatic key rotation via rotate_every_days policy (lazy, triggered on use)
• Key metadata search (filter by type, metadata tags, status, or policy fields)
• Encrypted backup/restore with HMAC verification
• Audit log rotation (configurable max size, max entries, retention)
• Backward-compatible ciphertext format versioning (v1 legacy, v2 AAD-bound)
• JWK (RFC 7517) key import/export for interoperability (supports P-256, P-384, P-521, secp256k1, Ed25519, RSA, AES)
• EC P-256, P-384, P-521, and secp256k1 signing with ECDSA (SHA-256, SHA-384, SHA-512)
• Ed25519 (EdDSA) signing for high-performance, compact signatures (Solana, Cosmos, SSH keys)
• secp256k1 support for Ethereum, Bitcoin, and EVM-compatible blockchain transaction signing
• Fully typed Python API (PEP 561 py.typed marker included)
• JSON-structured logging for production observability (SIEM-ready)
• Concurrency stress-tested (16 threads, data integrity proofs)
• 80%+ code coverage enforced in CI
• Reproducible builds via pinned dependency lockfile
• 264 tests across both layers (170 Python + 94 TypeScript)

---

Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│  Your Application                                                        │
│                                                                          │
│  hsm.sign("eth-wallet", tx_hash)                                         │
│  hsm.encrypt("app-key", data)                                            │
│                                                                          │
│  ► Raw key material is NEVER returned to this layer                      │
└──────────────────────────────────┬───────────────────────────────────────┘
                                   │ API call (or IPC via Unix socket)
                                   ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  PyHSM Core                                                              │
│                                                                          │
│  ┌─────────────┐ ┌──────────────┐ ┌────────────┐ ┌──────────────────┐   │
│  │ Key Unwrap  │ │ Policy Check │ │ Crypto Op  │ │ Audit + Metrics  │   │
│  │ (AES-KWP)  │→│ ACL, Rate,   │→│ Sign/Enc/  │→│ HMAC-chained log │   │
│  │             │ │ Expiry, Ops  │ │ Dec/Verify │ │                  │   │
│  └─────────────┘ └──────────────┘ └────────────┘ └──────────────────┘   │
│                                          │                               │
│                                    Key zeroized                           │
│                                    from memory                            │
└──────────────────────────────────────┬───────────────────────────────────┘
                                       │
                                       ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  Persistent Storage                                                      │
│                                                                          │
│  keystore.enc                          keystore.enc.audit.jsonl           │
│  ┌───────────────────────────────┐     ┌──────────────────────────────┐  │
│  │ AES-256-GCM Outer Envelope    │     │ HMAC-chained append-only log │  │
│  │ + HMAC-SHA256 Tamper Seal     │     │ (tamper-evident)             │  │
│  │  ┌─────────────────────────┐  │     └──────────────────────────────┘  │
│  │  │ AES-KWP Per-Key Wrapping│  │                                       │
│  │  │  • eth-wallet (secp256k1)│  │                                       │
│  │  │  • sol-wallet (ed25519) │  │                                       │
│  │  │  • app-key (aes-256)    │  │                                       │
│  │  └─────────────────────────┘  │                                       │
│  └───────────────────────────────┘                                       │
└─────────────────────────────────────────────────────────────────────────┘

---

Performance

Benchmarks measured on Python 3.13, macOS (Apple Silicon). Each operation includes the full security pipeline: key unwrapping, policy enforcement, cryptographic operation, audit logging, and keystore persistence.

Operation	Ops/sec	Avg Latency	p50	p99
AES-256 encrypt	~10	103 ms	103 ms	110 ms
AES-256 decrypt	~10	104 ms	104 ms	105 ms
RSA-2048 sign	~6	158 ms	158 ms	158 ms
RSA-2048 verify	~10	104 ms	104 ms	104 ms
EC P-256 sign	~10	104 ms	104 ms	105 ms
EC P-256 verify	~10	104 ms	104 ms	105 ms
secp256k1 sign	~9	105 ms	105 ms	106 ms
secp256k1 verify	~10	105 ms	105 ms	135 ms
Ed25519 sign	~10	104 ms	105 ms	106 ms
Ed25519 verify	~9	106 ms	104 ms	165 ms
Key generate (AES-256)	~10	104 ms	104 ms	107 ms
Key generate (secp256k1)	~10	105 ms	105 ms	105 ms
Key generate (Ed25519)	~10	104 ms	104 ms	105 ms
Key rotate (AES-256)	~10	104 ms	104 ms	105 ms

Where the time goes: ~103 ms is keystore persistence (encrypt + HMAC + atomic write). The actual cryptographic operation is sub-millisecond for symmetric and EC operations. RSA-2048 signing adds ~54 ms of computation on top of persistence.

For higher throughput: The TypeScript layer uses deferred persistence — operation counts are flushed on the next structural mutation or session close, giving significantly higher ops/sec for encrypt/decrypt/sign/verify workloads.

Run the benchmarks yourself:

python benchmarks/bench.py

---

Table of Contents

• Architecture
• Performance
• Python Layer
  • Installation
  • CLI Usage
  • Library Usage
  • Storage Backends
  • Architecture
• TypeScript Layer
  • Installation
  • Library Usage
  • Process Isolation Mode
  • Architecture
• Shared: Shamir Secret Sharing
• Blockchain Transaction Signing
• Security Model
• Threat Model
• Performance Benchmarks
• Running Tests
• Operations Guide
• FAQ

---

Python Layer

Python Installation

# Install from PyPI
pip install vectorguard-pyhsm

# Or install from source (with pyproject.toml)
pip install .

# For development (includes pytest + pytest-cov)
pip install ".[dev]"

# For reproducible builds (CI and production deployments)
pip install -r requirements.lock
pip install -e .

CLI Usage

All commands require --store (keystore path) and a master password (minimum 12 characters). The password is always entered interactively via a hidden prompt (never visible in the terminal or process list). For scripting and CI, you can set the PYHSM_MASTER_PASSWORD environment variable.

The --store flag can appear before or after the subcommand — put it wherever feels natural:

# These are equivalent:
vectorguard-pyhsm --store keystore.enc generate my-key
vectorguard-pyhsm generate my-key --store keystore.enc

If the keystore file does not yet exist, the CLI prints a notice to stderr so you can tell when you're accidentally pointing at the wrong path:

$ vectorguard-pyhsm --store /wrong/path.enc list
Master password:
Created new keystore: /wrong/path.enc
No keys stored.

# List keystore files in the current directory (no password required)
vectorguard-pyhsm stores
vectorguard-pyhsm stores /path/to/keystores

# Generate keys (password prompted interactively)
vectorguard-pyhsm --store keystore.enc generate my-aes-key --type aes-256
vectorguard-pyhsm --store keystore.enc generate my-rsa-key --type rsa-2048
vectorguard-pyhsm --store keystore.enc generate my-ec-key  --type ec-p256
vectorguard-pyhsm --store keystore.enc generate my-ec384   --type ec-p384
vectorguard-pyhsm --store keystore.enc generate my-ec521   --type ec-p521
vectorguard-pyhsm --store keystore.enc generate my-secp256k1 --type ec-secp256k1
vectorguard-pyhsm --store keystore.enc generate my-ed25519   --type ed25519

# Generate a key with a policy
vectorguard-pyhsm --store keystore.enc generate limited-key \
  --type aes-256 \
  --max-operations 500 \
  --expires-at 2027-01-01T00:00:00Z \
  --no-decrypt

# List keys in a keystore (shows type, current version, creation date)
vectorguard-pyhsm --store keystore.enc list

# Encrypt / Decrypt
vectorguard-pyhsm --store keystore.enc encrypt my-aes-key -d "secret message"
vectorguard-pyhsm --store keystore.enc decrypt my-aes-key -d <ciphertext-hex>

# Pipe via stdin
echo "secret message" | vectorguard-pyhsm --store keystore.enc encrypt my-aes-key

# Sign / Verify (uses stored public key for verify — private key never exposed)
vectorguard-pyhsm --store keystore.enc sign   my-ec-key -d "message to sign"
vectorguard-pyhsm --store keystore.enc verify my-ec-key "message to sign" <sig-hex>

# Export public key (PEM)
vectorguard-pyhsm --store keystore.enc pubkey my-rsa-key

# Rotate an AES key (archives current version, generates new one)
vectorguard-pyhsm --store keystore.enc rotate my-aes-key

# Destroy a key (zeroizes all versions, removes from store)
# Requires confirmation; use --yes/-y to skip the prompt
vectorguard-pyhsm --store keystore.enc delete my-aes-key
vectorguard-pyhsm --store keystore.enc delete my-aes-key --yes  # skip prompt

# Metrics
vectorguard-pyhsm --store keystore.enc metrics
vectorguard-pyhsm --store keystore.enc metrics --prometheus

# Audit log
vectorguard-pyhsm --store keystore.enc audit                       # pretty-printed entries
vectorguard-pyhsm --store keystore.enc audit --raw                 # compact JSON (one line per entry)
vectorguard-pyhsm --store keystore.enc audit --verify              # verify HMAC chain
vectorguard-pyhsm --store keystore.enc audit --operation encrypt   # filter by operation
vectorguard-pyhsm --store keystore.enc audit --key-id my-aes-key   # filter by key
vectorguard-pyhsm --store keystore.enc audit --since 2025-01-01T00:00:00Z

# For scripting/CI, set password via environment variable:
export PYHSM_MASTER_PASSWORD="your-master-password"
vectorguard-pyhsm --store keystore.enc list

Python Library Usage

from hsm import PyHSM

# Master password is always required — minimum 12 characters
hsm = PyHSM(
    storage_path="keystore.enc",
    master_password="your-master-password",
    session_timeout_s=300,      # auto-lock after 5 min inactivity (0 = disabled)
    rate_limit_max_ops=100,     # max ops per key per window
    rate_limit_window_s=60,
)

# Or use as a context manager for automatic cleanup
with PyHSM(storage_path="keystore.enc", master_password="your-master-password") as hsm:
    hsm.generate_key("my-key")
    ct = hsm.encrypt("my-key", "secret")
    # Key material is automatically zeroized on exit

# Generate keys
hsm.generate_key("aes-key")                          # AES-256 by default
hsm.generate_key("rsa-key", "rsa-2048")
hsm.generate_key("ec-key",  "ec-p256")
hsm.generate_key("ec384",   "ec-p384")              # NIST P-384 (SHA-384)
hsm.generate_key("ec521",   "ec-p521")              # NIST P-521 (SHA-512)
hsm.generate_key("eth-key", "ec-secp256k1")         # Bitcoin/Ethereum (SHA-256)
hsm.generate_key("sol-key", "ed25519")              # Ed25519 (Solana, SSH, high-perf signing)

# Generate a key with a policy (including caller ACL)
hsm.generate_key("restricted", policy={
    "allow_encrypt": True,
    "allow_decrypt": False,     # encrypt-only
    "max_operations": 1000,
    "expires_at": "2027-01-01T00:00:00Z",
    "allowed_callers": ["service-a", "service-b"],  # caller ACL
})

# Generate a key with automatic rotation (rotates on next encrypt when due)
hsm.generate_key("auto-rotate-key", policy={
    "allow_encrypt": True,
    "allow_decrypt": True,
    "rotate_every_days": 90,    # auto-rotates every 90 days on use
})

# Encrypt / Decrypt (AES-256-GCM with AAD binding and hybrid nonce)
ciphertext = hsm.encrypt("aes-key", "secret message")  # returns hex string
plaintext  = hsm.decrypt("aes-key", ciphertext)        # returns bytes

# All operations support caller_id for audit tracking and ACL enforcement
ciphertext = hsm.encrypt("aes-key", "data", caller_id="my-service")
plaintext  = hsm.decrypt("aes-key", ciphertext, caller_id="my-service")

# Rotate a key (old ciphertexts remain decryptable via version prefix)
new_version = hsm.rotate_key("aes-key")

# Sign / Verify
signature = hsm.sign("ec-key", "message")
is_valid   = hsm.verify("ec-key", "message", signature)  # uses stored public key only

# Sign with P-384 (uses SHA-384 automatically) or P-521 (uses SHA-512)
sig384 = hsm.sign("ec384", "message", caller_id="signer-service")
is_valid = hsm.verify("ec384", "message", sig384, caller_id="verifier")

# Sign with secp256k1 (Ethereum/Bitcoin transaction signing)
sig_eth = hsm.sign("eth-key", tx_hash, caller_id="tx-service")
is_valid = hsm.verify("eth-key", tx_hash, sig_eth, caller_id="verifier")

# Sign with Ed25519 (high-performance, compact 64-byte signatures)
sig_ed = hsm.sign("sol-key", "message", caller_id="signing-service")
is_valid = hsm.verify("sol-key", "message", sig_ed, caller_id="verifier")

# Export public key (PEM)
pub_pem = hsm.get_public_key("rsa-key")

# Expiry enforcement (archives expired keys)
hsm.enforce_expiry()

# Key search (filter by type, metadata, status, or policy)
all_aes = hsm.search_keys(key_type="aes-256")
prod_keys = hsm.search_keys(metadata={"env": "prod"})
active_keys = hsm.search_keys(status="active")
auto_rotating = hsm.search_keys(policy_filter={"rotate_every_days": 90})

# Backup and restore
backup_path = hsm.create_backup("/secure/backups")     # encrypted copy of keystore
hsm.verify_backup(backup_path)                         # True if intact, raises on tamper

# Metrics
metrics_dict = hsm.get_metrics()
prometheus   = hsm.get_prometheus_metrics()
otlp_json    = hsm.get_otlp_metrics()                  # OpenTelemetry OTLP JSON format

# Audit log
audit = hsm.get_audit_log()
audit.verify()                                          # returns -1 if clean
entries = audit.export_jsonl(operation="encrypt")       # SIEM-ready list of dicts

# JWK export (RFC 7517) — interoperate with other KMS systems
# Requires allow_export=True in the key's policy (defaults to False for security)
hsm.generate_key("export-aes", policy={"allow_encrypt": True, "allow_decrypt": True, "allow_export": True})
jwk = hsm.export_jwk("export-aes")                    # {"kty": "oct", "k": "...", ...}

hsm.generate_key("export-ec", "ec-p256", policy={"allow_sign": True, "allow_export": True})
ec_jwk = hsm.export_jwk("export-ec")                  # {"kty": "EC", "crv": "P-256", ...}

# JWK import — bring keys from external systems
hsm.import_key_jwk("imported-key", {
    "kty": "oct",
    "k": "base64url-encoded-key-material",
    "alg": "A256GCM",
})

# Import an Ethereum/Bitcoin private key via JWK
hsm.import_key_jwk("eth-wallet", {
    "kty": "EC",
    "crv": "secp256k1",
    "x": "...",   # base64url public key x-coordinate
    "y": "...",   # base64url public key y-coordinate
    "d": "...",   # base64url private key scalar
})

# Import an Ed25519 key (e.g., from Solana or SSH)
hsm.import_key_jwk("ed-key", {
    "kty": "OKP",
    "crv": "Ed25519",
    "x": "...",   # base64url 32-byte public key
    "d": "...",   # base64url 32-byte private key seed
})

# Explicit close (zeroizes master password and key material from memory)
hsm.close_session()

Storage Backends

PyHSM supports pluggable storage backends. The default is file-based with atomic writes, but you can implement custom backends for database, cloud storage, or any other persistence layer.

from hsm import PyHSM, KeyStore
from hsm.backends import StorageBackend, FileBackend, MemoryBackend

# Default: file backend (backward-compatible)
hsm = PyHSM(storage_path="keystore.enc", master_password="pw")

# Explicit file backend
from hsm.backends import FileBackend
store = KeyStore(master_password="pw", backend=FileBackend("/secure/keystore.enc"))

# In-memory backend (for testing or ephemeral use)
from hsm.backends import MemoryBackend
store = KeyStore(master_password="pw", backend=MemoryBackend())

# Custom backend — implement the StorageBackend interface:
#   exists() -> bool
#   read() -> bytes
#   write(data: bytes) -> None
#   delete() -> None

The StorageBackend interface deals only with raw encrypted bytes — all encryption, HMAC verification, and key management logic stays in KeyStore. Backends never see plaintext key material.

Python Environment Variables

Variable	Default	Description
PYHSM_MASTER_PASSWORD	(none)	Master password for the CLI. When set, the CLI uses this instead of prompting interactively. Useful for scripting and CI.
PYHSM_LOG_LEVEL	WARNING	Structured log verbosity: DEBUG, INFO, WARNING, ERROR, CRITICAL. Set to INFO in production for operational visibility.
PYHSM_ALLOW_PBKDF2_FALLBACK	0 (disabled)	Set to 1 to permit degraded PBKDF2 key derivation when argon2-cffi is unavailable. Testing/migration only — never set in production.
PYHSM_AUDIT_HMAC_KEY	(derived from master password)	Hex-encoded 32-byte key for audit HMAC chain. When not set, derived automatically from the master password via HKDF.
PYHSM_AUDIT_WEBHOOK	(none)	URL for non-blocking audit event POST. Webhook failures are logged (not silently dropped).

Structured Logging

PyHSM outputs JSON-structured logs via Python's stdlib logging module. Every log line is a single JSON object with consistent fields for machine parsing:

{"timestamp": "2026-01-15T10:30:00.123456+00:00", "level": "INFO", "logger": "hsm.core", "message": "key generated", "event": "generate_key", "key_id": "my-key", "key_type": "aes-256"}

Configure from your application:

import logging

# See all PyHSM operational events
logging.getLogger("hsm").setLevel(logging.INFO)

# Or via environment variable before import:
# export PYHSM_LOG_LEVEL=INFO

Logged events include: session_open, session_close, self_test_pass, self_test_fail, generate_key, rotate_key, destroy_key, encrypt, decrypt, sign, access_denied, rate_limited, tamper_detected, kdf_migration, webhook_failure.

Python Architecture

hsm/
  core.py           — PyHSM class: key lifecycle, encrypt/decrypt, sign/verify,
                      per-key AES-KWP wrapping, AAD binding, hybrid nonce,
                      per-key sharded locks, caller_id ACL enforcement,
                      automatic key rotation (rotate_every_days policy),
                      key metadata search, backup/restore, OTLP metrics export
  storage.py        — KeyStore: Argon2id (required) key derivation,
                      HKDF key separation (enc/mac/kek subkeys),
                      AES-256-GCM + HMAC-SHA256, cached KEK, pluggable StorageBackend,
                      bytearray key_data for deterministic zeroization, auto-migration
                      from PBKDF2 to Argon2id
  backends.py       — StorageBackend ABC, FileBackend (atomic writes), MemoryBackend
  logging.py        — JSON-structured logging via stdlib (machine-parseable, SIEM-ready)
  secure_memory.py  — SecureBytes: deterministic bytearray zeroization, context manager
  jwk.py            — JWK (RFC 7517) import/export: oct, EC (P-256/P-384/P-521/secp256k1), OKP (Ed25519), RSA
  shamir.py         — Shamir secret sharing over GF(256)
  audit.py          — HMAC-chained append-only audit log with rotation (max_bytes/max_entries)
  rate_limiter.py   — Sliding-window per-key rate limiter
  metrics.py        — Prometheus and OpenTelemetry (OTLP) metrics export
  self_test.py      — Startup Known-Answer Tests (KATs)
  __init__.py       — Public API exports
  py.typed          — PEP 561 marker for type checker support
cli.py              — Full-featured command-line interface
tests/
  test_pyhsm.py     — 140 pytest tests (unit + integration + auto-rotation + search + OTLP)
  test_concurrency.py — 8 concurrency stress tests (16 threads, data integrity proofs)
  test_cli.py       — 30 subprocess-based CLI integration tests

---

TypeScript Layer

A production-hardened Node.js library in ./pyhsm-ts/ with additional features: Argon2id KDF, AES-256-GCM-SIV (nonce-misuse resistant), SecureBuffer zeroization, and process isolation mode.

Both layers now use Argon2id as the primary key derivation function (OWASP recommended, 64 MB memory-hard). The Python layer requires argon2-cffi and will refuse to start without it. A PBKDF2-SHA256 fallback (480,000 iterations) is available only when PYHSM_ALLOW_PBKDF2_FALLBACK=1 is set (for testing/migration scenarios only — not for production).

TypeScript Installation

cd pyhsm-ts
npm install
npm run build

Requires Node.js ≥ 18. All dependency versions are pinned exactly.

TypeScript Library Usage

Synchronous constructor (PBKDF2 fallback)

import { PyHSM } from "./pyhsm-ts";

const hsm = new PyHSM({
  storePath: "./keystore.enc",
  masterPassword: "your-master-password",
  sessionTimeoutMs: 300_000,
  backupDir: "./backups",
});

Async factory — Argon2id KDF (recommended for production)

const hsm = await PyHSM.create({
  storePath: "./keystore.enc",
  masterPassword: "your-master-password",
});

The create() factory guarantees Argon2id (64 MB / 3 passes / 4 threads) is used for all key derivation — including the first save on a new keystore. The synchronous constructor falls back to PBKDF2-SHA256 at 480,000 iterations.

Custom storage backend

import { PyHSM, MemoryBackend } from "./pyhsm-ts";

// In-memory backend for testing
const hsm = new PyHSM({
  storePath: "test",
  masterPassword: "pw",
  backend: new MemoryBackend(),
});

// Custom backend — implement the StorageBackend interface:
//   exists(): boolean
//   read(): Buffer
//   write(data: Buffer): void
//   delete(): void

Key operations

// Generate
hsm.generateKey("my-key");

// Generate with specific key types
hsm.generateKey("eth-key", "ec-secp256k1");    // Bitcoin/Ethereum signing
hsm.generateKey("sol-key", "ed25519");          // Solana/SSH/high-performance signing

// Generate with policy
hsm.generateKey("restricted", {
  allowEncrypt: true,
  allowDecrypt: true,
  maxOperations: 1000,
  expiresAt: "2027-01-01T00:00:00Z",
  allowedCallers: ["service-a", "service-b"],
});

// Encrypt / Decrypt (AES-256-GCM-SIV, base64 output with version prefix)
const ct = hsm.encrypt("my-key", "secret message");
const pt = hsm.decrypt("my-key", ct);              // returns string

// Rotate (old ciphertexts remain decryptable)
hsm.rotateKey("my-key");

// Destroy (zeroizes all versions)
hsm.destroyKey("my-key");

// Backup and verify
const backupPath = hsm.createBackup();
hsm.verifyBackup(backupPath);   // HMAC + decrypt check without loading into live store

// Metrics
const metrics = hsm.getMetrics();
const prom    = hsm.getPrometheusMetrics();

// Audit log
const audit = hsm.getAuditLog();
const clean  = audit.verify();                     // -1 = no tampering
const events = audit.exportJsonl({ operation: "encrypt", since: "2025-01-01T00:00:00Z" });
const ndjson = audit.toNdjson({ onlyFailed: true }); // SIEM-ready NDJSON string

// Close (zeroizes all Buffers holding sensitive material)
hsm.closeSession();

JWK Import / Export (RFC 7517)

// Export a key as standard JWK — interoperate with any system
const jwk = hsm.exportJwk("my-key");  // {"kty": "oct", "k": "...", "alg": "A256GCM"}

// Import a key from external JWK
hsm.importKeyJwk("external-key", {
  kty: "oct",
  k: "base64url-encoded-key-material",
  alg: "A256GCM",
});

// Import an EC key from another identity provider
hsm.importKeyJwk("idp-signing-key", {
  kty: "EC",
  crv: "P-256",
  x: "...",
  y: "...",
  d: "...",
});

// Import an Ethereum/Bitcoin private key (secp256k1)
hsm.importKeyJwk("eth-wallet", {
  kty: "EC",
  crv: "secp256k1",
  x: "...",
  y: "...",
  d: "...",
});

// Import an Ed25519 key (Solana, SSH, etc.)
hsm.importKeyJwk("sol-wallet", {
  kty: "OKP",
  crv: "Ed25519",
  x: "...",   // 32-byte public key (base64url)
  d: "...",   // 32-byte private key seed (base64url)
});

Key ID rules

Key IDs must match ^[a-zA-Z0-9][a-zA-Z0-9._-]{0,127}$:

• 1–128 characters
• Must start with a letter or digit
• May contain letters, digits, ., _, -
• Rejects path traversal (../), prototype pollution (__proto__), spaces

Process Isolation Mode

For maximum security, run the HSM in a separate process. A vulnerability in your application cannot directly read key material in the HSM process's memory.

Start the HSM process:

export PYHSM_MASTER_PASSWORD="your-master-password"
export PYHSM_KEYSTORE_PATH="/secure/keystore.enc"
export PYHSM_SOCKET_PATH="/run/pyhsm/pyhsm.sock"
export PYHSM_CALLER_SECRET="shared-hmac-secret"
export PYHSM_BACKUP_DIR="/secure/backups"

npx tsx pyhsm-ts/process.ts

Connect from your application:

import { PyHSMClient } from "./pyhsm-ts";

const client = new PyHSMClient("/run/pyhsm/pyhsm.sock", "my-service");

await client.generateKey("app-key");
const ct = await client.encrypt("app-key", "secret");
const pt = await client.decrypt("app-key", ct);

await client.rotateKey("app-key");
const path = await client.backup();
const ok   = await client.verifyBackup(path);
const h    = await client.health();
const m    = await client.metrics();

TypeScript Environment Variables

Variable	Default	Description
PYHSM_MASTER_PASSWORD	—	Master password (required unless using PYHSM_SHARES)
PYHSM_SHARES	—	Comma-separated Shamir share JSON objects
PYHSM_KEYSTORE_PATH	./pyhsm-keystore.enc	Encrypted keystore location
PYHSM_AUDIT_LOG_PATH	<storePath>.audit.jsonl	HMAC-chained audit log path
PYHSM_AUDIT_HMAC_KEY	(auto-generated)	Hex 32-byte audit HMAC key
PYHSM_AUDIT_WEBHOOK	—	URL for non-blocking audit event POST
PYHSM_BACKUP_DIR	—	Directory for encrypted backups
PYHSM_SOCKET_PATH	/tmp/pyhsm.sock	Unix domain socket path (IPC mode)
PYHSM_CALLER_SECRET	—	Shared secret for IPC caller HMAC auth
PYHSM_SESSION_TIMEOUT_MS	300000	Idle ms before auto-lock
PYHSM_RATE_LIMIT	100	Max operations per key per window
PYHSM_RATE_WINDOW_MS	60000	Rate limit window duration (ms)
PYHSM_KEY_ID	pyhsm-master	Default key ID for singleton helpers

TypeScript Architecture

pyhsm-ts/
  core.ts             — PyHSM class: key lifecycle, encrypt/decrypt, backup,
                        AES-KWP per-key wrapping, HKDF key separation, pluggable StorageBackend
  storage-backend.ts  — StorageBackend interface, FileBackend, MemoryBackend
  types.ts            — TypeScript interfaces, key ID validation, config with backend option
  jwk.ts              — JWK (RFC 7517) import/export: oct, EC (P-256/P-384/P-521/secp256k1), OKP (Ed25519), RSA
  shamir.ts           — Shamir secret sharing over GF(256)
  audit.ts            — HMAC-chained audit log, SIEM export
  rate-limiter.ts     — Sliding-window per-key rate limiter
  metrics.ts          — Prometheus metrics collector
  self-test.ts        — Startup Known-Answer Tests (KATs), FIPS mode
  secure-buffer.ts    — SecureBuffer: deterministic Buffer zeroization
  process.ts          — IPC server (process isolation via Unix socket)
  client.ts           — IPC client with HMAC caller auth
  index.ts            — Public API exports and singleton factory
  pyhsm.test.ts       — 94 tests (vitest)
  OPERATIONS.md       — Full operator guide (env vars, deployment, procedures)
  package.json        — Pinned exact dependency versions
  tsconfig.json       — Strict TypeScript configuration

---

Shamir's Secret Sharing

Both layers implement Shamir secret sharing over GF(256) with the AES irreducible polynomial. This can be used to split a master password or any secret into N shares where K are required to reconstruct — and K-1 or fewer shares reveal zero information (information-theoretic security).

Python:

# Split a hex secret into 5 shares, 3 required
vectorguard-pyhsm split -k 3 -n 5 -s "deadbeefcafe..."

# Reconstruct from any 3
vectorguard-pyhsm reconstruct \
  --share '{"index":1,"data":"..."}' \
  --share '{"index":3,"data":"..."}' \
  --share '{"index":5,"data":"..."}'

TypeScript:

import { splitMasterPassword, PyHSM } from "./pyhsm-ts";

// One-time: split the master password into 5 shares, 3 required to unlock
const shares = splitMasterPassword("my-master-password", 3, 5);
// Distribute shares[0..4] to five separate key custodians

// At startup: collect K shares from operators
const hsm = new PyHSM({
  storePath: "./keystore.enc",
  shares: [
    JSON.stringify(shares[0]),
    JSON.stringify(shares[2]),
    JSON.stringify(shares[4]),
  ],
});

Intermediate share buffers are zeroized from memory after reconstruction in both layers.

---

Blockchain Transaction Signing (secp256k1 / Ed25519)

PyHSM can serve as a self-hosted signing infrastructure for Ethereum, Bitcoin, Solana, and other blockchain networks. The private key is imported once, encrypted at rest, and never exposed again — all signing happens through PyHSM.

Step 1: Import an Existing Private Key (One-Time)

from hsm import PyHSM
import base64

def b64url(b: bytes) -> str:
    return base64.urlsafe_b64encode(b).rstrip(b"=").decode()

# Raw private key (e.g., from MetaMask export or key generation)
raw_key_hex = "0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80"
privkey_bytes = bytes.fromhex(raw_key_hex.removeprefix("0x"))

# Derive public key coordinates (using eth_keys, coincurve, or similar)
from eth_keys import keys
pk = keys.PrivateKey(privkey_bytes)
x_bytes = pk.public_key.to_bytes()[:32]
y_bytes = pk.public_key.to_bytes()[32:]

# Import into PyHSM — key is now AES-KWP double-encrypted at rest
hsm = PyHSM(storage_path="/secure/keystore.enc", master_password="strong-pw")
hsm.import_key_jwk("eth-wallet", {
    "kty": "EC",
    "crv": "secp256k1",
    "x": b64url(x_bytes),
    "y": b64url(y_bytes),
    "d": b64url(privkey_bytes),
})
hsm.close_session()

# DELETE the raw private key from disk/memory — it now lives only in PyHSM

Or generate a fresh wallet key directly:

hsm = PyHSM(storage_path="/secure/keystore.enc", master_password="strong-pw")
hsm.generate_key("eth-wallet", "ec-secp256k1")

# Derive the Ethereum address from the public key PEM
pub_pem = hsm.get_public_key("eth-wallet")

Step 2: Sign Transactions Through PyHSM

hsm = PyHSM(storage_path="/secure/keystore.enc", master_password="strong-pw")

# Your application builds and hashes the transaction
tx_hash_hex = "0x..."  # keccak256 of the unsigned RLP-encoded transaction

# Sign through PyHSM — private key is unwrapped, used, and immediately zeroized
signature_hex = hsm.sign("eth-wallet", bytes.fromhex(tx_hash_hex.removeprefix("0x")),
                         caller_id="tx-service")

# Parse the DER-encoded ECDSA signature into (r, s) for Ethereum
from cryptography.hazmat.primitives.asymmetric.utils import decode_dss_signature
r, s = decode_dss_signature(bytes.fromhex(signature_hex))

# Determine v (recovery ID) and broadcast the signed transaction

Step 3: Ed25519 (Solana / Cosmos)

hsm = PyHSM(storage_path="/secure/keystore.enc", master_password="strong-pw")
hsm.generate_key("sol-wallet", "ed25519")

# Sign a Solana transaction payload
signature_hex = hsm.sign("sol-wallet", transaction_bytes, caller_id="sol-service")

# Ed25519 signatures are 64 bytes — use directly with Solana SDK
signature_bytes = bytes.fromhex(signature_hex)

TypeScript (Process Isolation)

For maximum security, run PyHSM in a separate process so even an application-layer exploit cannot read key material:

// Start the HSM process:
// PYHSM_MASTER_PASSWORD="..." npx tsx pyhsm-ts/process.ts

import { PyHSMClient } from "./pyhsm-ts";
const client = new PyHSMClient("/run/pyhsm/pyhsm.sock", "tx-service");

// Sign — key never enters the application process memory
const sig = await client.encrypt("eth-wallet", txHash);

Security Gains for Blockchain Use

Threat	Without PyHSM	With PyHSM
Key in .env or config	Plaintext on disk	AES-256-GCM + AES-KWP double-encrypted
Server compromise (memory dump)	Key exposed	Key in memory only during sign, then zeroized
Insider theft	Copy the key file silently	Requires master password + keystore (or M-of-N Shamir shares)
No signing audit trail	Attacker signs silently	HMAC-chained audit log records every operation with caller_id
Unlimited signing after theft	No constraints	Rate limiting + max_operations policy + expires_at
Single admin controls keys	One person holds everything	Shamir 3-of-5 unlock ceremony

---

Security Model

Property	Mechanism
Keys encrypted at rest	AES-256-GCM + AAD binding (Python) / AES-256-GCM-SIV (TypeScript)
Per-key double encryption	AES-KWP RFC 5649 wrapping in both layers — keys encrypted inside the encrypted envelope
Keystore tamper detection	Encrypt-then-MAC with separated keys (HKDF-derived enc + mac subkeys)
Key derivation	Argon2id 64MB (required, Python + TypeScript) → HKDF-Expand. PBKDF2-SHA256 480k iter available only via explicit env var escape hatch for testing
Key separation	HKDF-Expand with distinct info strings (pyhsm-enc-v1, pyhsm-mac-v1, pyhsm-kek-v1) — encryption, MAC, and KEK keys are cryptographically independent
KEK derivation	Dedicated salt stored inside encrypted keystore → Argon2id → HKDF-Expand. KEK is cached in memory for session lifetime and zeroized on close
Memory zeroization	Key material stored as mutable bytearray (Python) / Buffer (TypeScript) with deterministic in-place zeroing. Immutable hex strings eliminated from memory path
Nonce safety	Hybrid nonce: random(4) + counter(4) + random(4) eliminates birthday-bound (Python); AES-256-GCM-SIV nonce-misuse resistant (TypeScript)
Ciphertext binding	AAD ties ciphertext to key_id + version — prevents cross-key confusion attacks
Ciphertext versioning	Format byte distinguishes v2 (AAD-bound) from v1 (legacy) for backward compatibility
Input validation	64 MB maximum enforced on both encrypt (plaintext) and decrypt (ciphertext) paths
Atomic writes	os.replace() (Python) / fs.renameSync on temp file (TypeScript)
Audit integrity	Per-entry HMAC chain; audit HMAC key derived from master password via HKDF (Python) or stored independently (TypeScript)
Caller ID tracking	All operations accept optional caller_id; recorded in every audit entry
Caller ACL enforcement	Per-key allowed_callers policy; unauthorized callers denied with accessDenied audit entry
Constant-time comparisons	hmac.compare_digest (Python) / length-padded timingSafeEqual (TypeScript)
Crypto primitive verification	Known-Answer Tests against RFC vectors at startup
Session isolation	Auto-lock on inactivity; explicit close_session() / closeSession()
Concurrency	Per-key sharded locks (Python) — parallel operations on different keys; serialized save lock prevents write races
Process memory isolation	Optional: IPC mode runs HSM in a separate process (TypeScript)
M-of-N startup ceremony	Shamir split/reconstruct on master password
Pluggable storage	StorageBackend interface — swap file I/O for database, S3, etc.
Key interoperability	JWK (RFC 7517) import/export — supports P-256, P-384, P-521, secp256k1, Ed25519, RSA, AES
EC curve support	P-256 (SHA-256), P-384 (SHA-384), P-521 (SHA-512), secp256k1 (SHA-256) — NIST/SEC recommended hash pairing
EdDSA support	Ed25519 signing — high-performance 64-byte signatures (Solana, Cosmos, SSH keys)
Type safety	PEP 561 py.typed marker; `str
Observability	JSON-structured logging via stdlib logging; configurable via PYHSM_LOG_LEVEL env var; webhook failures logged (not silently dropped)

Honest scope statement: PyHSM is a software KMS. It does not carry FIPS 140-2/3 validation (which requires NIST laboratory certification of the specific binary). It does not provide the physical tamper evidence of a hardware HSM. Key material is protected by OS-level process boundaries, not a secure enclave or physically separate processor. For regulated environments that mandate certified hardware, use a certified HSM; PyHSM is appropriate where software key management is acceptable.

---

Threat Model

See THREAT_MODEL.md for the full formal threat model, including:

• Assets protected and trust boundary diagrams
• Five threat actor profiles (T1–T5) with specific mitigations
• Cryptographic design decisions and rationale
• Assumptions and known limitations
• Comparison to hardware HSM threat coverage

---

FAQ

See docs/FAQ.md for detailed answers to common architecture and security questions, including:

• Why build a software HSM instead of using Vault?
• How do you prevent key extraction?
• How is key rotation implemented?
• How are audit logs protected from tampering?
• What cryptographic guarantees do you provide?
• What threat model did you design against?
• Why should I use this instead of AWS KMS?

---

Running Tests

Python (pytest):

# Run all tests with coverage
python -m pytest tests/ -v
# 170 tests (unit + integration + concurrency + CLI)

# Run CLI integration tests separately (subprocess-based)
python -m pytest tests/test_cli.py -v

# Coverage report (80% minimum threshold enforced in CI)
python -m pytest tests/ --cov=hsm --cov-report=term-missing --cov-fail-under=80

Reproducible installs use the pinned lockfile:

pip install -r requirements.lock
pip install -e .

TypeScript (vitest):

cd pyhsm-ts
npm test
# 94 tests (vitest)

CI runs both suites on every push and pull request, across Python 3.11/3.12/3.13 and Node.js 20. Coverage is enforced at 80% minimum for both layers. Python also runs mypy --strict type checking. See .github/workflows/ci.yml.

---

Operations Guide

See pyhsm-ts/OPERATIONS.md for the full operator guide, including:

• Deployment architectures (embedded vs. process-isolated)
• All environment variables with descriptions and defaults
• Shamir ceremony procedure
• Key rotation, backup, and backup verification procedures
• Audit log verification and SIEM export
• Prometheus metrics and OpenTelemetry (OTLP) reference
• Security considerations

---

License

MIT