Quick start · How it works · Self-host · Frameworks · Scope · Explore
Open-source, self-hosted compliance automation. Connect your stack, continuously test controls, collect evidence, and hand auditors proof — running in your own cloud or VPC, on your own data lake.
- Your cloud, your evidence. Evidence stays in storage you run. Data leaves only through the connectors, sinks, and model integrations you configure.
- Two modes. Ingest evidence into a lake you own, or connect read-only to the security lake you already run. How it works.
- Deterministic and API-first. Rules decide pass or fail, and every result links to its evidence; models may summarize or propose, never decide. The console, API, CLI, MCP server, and CI gates share one engine.
Quick start
Use Python 3.11+, uv, and Node 22+:
git clone https://github.com/msaad00/trustops-security-data-lake.git
cd trustops-security-data-lake
uv sync --frozen --extra dev --extra server
make demo-local
Open localhost:8787/console/dashboard/. This loads fixture data and disables authentication; for a shared environment, see Self-host.
Other setup paths — pip, CLI only, and MCP
Source install without uv:
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev,server]"
make web-install web-build
security-lakehouse fixtures load --company golden --out build/lakehouse --rebase-times
security-lakehouse db upgrade --lake build/lakehouse
security-lakehouse serve --lake build/lakehouse --server --allow-insecure-no-auth --port 8787
CLI and local lake only:
pip install trustops-security-data-lake
security-lakehouse fixtures load --company golden --out ./lake --rebase-times
security-lakehouse assessment status --lake ./lake
The same lake over MCP (stdio); see headless GRC for the trust boundary:
pip install 'trustops-security-data-lake[mcp]'
TRUSTOPS_LAKE=./lake trustops-mcp
How it works
| Step | What you do | What you get |
|---|---|---|
| Collect | Connect a source with read-only access. | Evidence with source, freshness, and provenance. |
| Evaluate | Apply deterministic control rules. | Results tied to evidence and the evaluated catalog. |
| Resolve | Assign findings, track fixes, and review exceptions. | Ownership and a record of follow-up decisions. |
| Export | Freeze an assessment and share reports. | Evidence and assessment history for reviewers. |
Evidence arrives in one of two modes; both feed the same rules and assessments.
| Mode | How | Sources |
|---|---|---|
| Ingest | Read-only connectors pull evidence into a lake you own. No existing lake needed. | AWS, Azure, GCP, GitHub, GitLab, Okta, Google Workspace, Jira, Intune, BambooHR, Rippling, Workday, Jamf*, CrowdStrike Falcon*, Kubernetes*, KnowBe4* |
| Existing lake | Read-only queries against the lake you already run; a lake mapping maps your existing tables. | Snowflake, ClickHouse, Databricks*, Iceberg/Parquet* (including Amazon Security Lake through OCSF presets), BigQuery*, S3 object evidence, SIEM exports |
* Preview: implemented and fixture-tested, not yet verified against a live tenant. Lake mappings are experimental. The connector catalog lists all 28 contracts, 25 of them executable.
Self-host
| Path | Use it for |
|---|---|
| Docker image | One host: docker run -p 8787:8787 -v $PWD/lake:/lake ghcr.io/msaad00/trustops:0.2 |
| Helm chart | Kubernetes, with a persistent /lake volume and the scheduler. |
| EKS Terraform | Reference infrastructure for the chart on Amazon EKS. |
| AWS · Azure · GCP | Read-only posture roles for each cloud; no static keys. |
| Snowflake · Databricks · ClickHouse | Schema and bootstrap SQL for each existing-lake reader. |
Server mode requires authentication (OIDC, SAML, or API keys). Start with the deployment guide.
Frameworks and common controls
17 framework packs · 78 reusable safeguards · 21 control families · 2,031 catalogued requirements.
1,182 requirements have safeguard mappings; 350 have reviewed mappings. Catalog coverage and evaluated customer posture are separate measures.
Control families: Identity and access · Data protection · Detection · Audit logging · Change management · Configuration management · Secure development · Secure architecture · Vulnerability management · Third-party risk · Risk management · Availability and recovery · Incident response · Governance · People security · Physical security · Network security · System maintenance · Processing integrity · Privacy · AI governance.
 SOC 2 |
 ISO 27001 · 27017 · 27701 · 42001 |
 NIST CSF 2.0 |
 NIST AI RMF |
 CIS Controls · CIS AWS |
 CMMC 2.0 |
 EU AI Act · GDPR |
NIST 800-53 · NIST RMF FedRAMP · HIPAA · PCI DSS |
A reviewed mapping has been confirmed by a person; a proposed one has not, and neither is a certification. Some packs are limited: PCI DSS v4.0.1 covers its 12 principal requirements, ISO/IEC 27701:2025 seeds 10 of its 78 Annex A controls, and NIST RMF mappings are all proposed. SOC 1 is planned. The coverage matrix has the exact boundary per framework.
How the Common Control Framework evaluates
| Layer | What it represents |
|---|---|
| Control families | Risk domains that organize reusable safeguards. |
| Safeguards | Evidence requirements, ownership, review frequency, and executable evaluation rules. |
| Framework mappings | Links from safeguards to framework requirements, reviewed or proposed. |
| Assessment results | Pass, fail, stale, or not evaluated, from the collected evidence. |
One safeguard can serve several frameworks. A requirement passes only when every mapped safeguard passes; an unmapped requirement stays unmapped. Details: Common Control Framework · executable catalog. From the CLI:
security-lakehouse frameworks safeguards --format table
Scope
| Area | Status |
|---|---|
| License | Apache-2.0; no per-seat license. You run the infrastructure. |
| Audit workflow | Audit room, readiness, frozen assessments, trust-center shares, access reviews, OSCAL export. |
| Policies and vendor risk | MVP: policy templates with attestation, vendor questionnaires. |
| Automation | Versioned API, CLI, MCP server, CI posture gate, webhooks. |
| Integrations | 25 executable connectors; add your own as a separately installed Python package. |
| Not offered | A managed service, or certification. Results are evidence for your auditor. |
Details: product status · roadmap.
Explore
01 · Product tour
Images show the bundled demo fixture, not live customer evidence.
Overview
Frameworks
Evidence
Graph
Triage
Walkthrough · Connections · Findings · Remediation · Audit room · Workflows · Trust center
02 · Connector credentials
In the console: Connections → choose a source → Test → Enable → Sync. For automation, use the headless setup playbook. Cloud connectors use short-lived or workload identity credentials; for GCP, Application Default Credentials (a service-account key file also works). SaaS connectors use scoped API tokens or an integration-user login. Settings keep a credential reference (an environment variable name or mounted secret file), not the secret itself.
- AWS uses STS AssumeRole, one External ID per deployed role, short-lived session credentials, and read-only IAM posture APIs. Temporary credentials expire after each session; TrustOps stores no long-lived access keys. Roll out with CloudFormation StackSets or Terraform workspaces; Bulk account import is planned. See the cloud setup guide and the credential lifecycle.
- Azure uses a customer-owned Entra application, managed identity, or federated workload identity with Reader scope.
- Snowflake uses a read-only service identity with a key-pair or OAuth token reference. TrustOps stores identifiers, not passwords or private-key contents. Snowflake is the existing security-data-lake path.
- GitHub uses a GitHub App installation token, which expires within an hour.
Ship your own connector as a Python package: adding connectors.
03 · Architecture and storage
Source → Raw evidence → Normalized facts → Control evaluation → Assessment
↓ ↓
Owned findings Review / export
| Layer | Boundary |
|---|---|
| Evidence and evaluation | Local JSONL and verified assessment generations; one writer per lake. |
| Analytics and state | SQLite mart (DuckDB optional) and an application database. |
| Portable evidence | Optional Parquet export and Iceberg REST publication. |
Existing-lake readers are evidence sources; they do not host TrustOps. Architecture · assessment generations · continuous ingestion
04 · API, agents, and CI
| Surface | Purpose |
|---|---|
| API | Versioned /api/v1 access for integrations. |
| MCP | Read assessments and propose actions through governed tools. |
| CI | Posture and control-test thresholds in delivery workflows. |
| OSCAL | Component-definition and assessment-results JSON. |
| Webhooks | Signed event delivery to your systems. |
Develop and verify
Checks and repository layout
make smoke # backend, contracts, docs, brand, pipeline, API
make web-ci # install, typecheck, production build
make security # dependency audits and pre-commit checks
| Directory | Contents |
|---|---|
src/security_lakehouse/ |
Assessment engine, API, auth, connectors, and MCP. |
app/web/ |
Next.js console. |
controls/, frameworks/, mappings/ |
Rules, framework catalogs, and mappings. |
deploy/ |
Deployment and infrastructure examples. |
docs/ |
Product, architecture, operations, and API guides. |
1 maintainer
Metadata
1 maintainer
Release files for trustops-security-data-lake 0.2.18
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| trustops_security_data_lake-0.2.18.tar.gz | 2.8 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| trustops_security_data_lake-0.2.18-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 5.7 MB
Release files / trustops_security_data_lake-0.2.18.tar.gz
| Download URL | trustops_security_data_lake-0.2.18.tar.gz |
|---|---|
| Size | 2.8 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
492f51bf10cc18746ac60b377404405859f9028bc4a728c3dcc8297e889e7036
|
|
BLAKE2b-256 checksum How to use checksums |
918a9c3df04927e09aeb1866d64a0af5cac58133b2af98aa32689268b0024aa6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 28, 2026.
Transparency logRelease files / trustops_security_data_lake-0.2.18-py3-none-any.whl
| Download URL | trustops_security_data_lake-0.2.18-py3-none-any.whl |
|---|---|
| Size | 2.9 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a8cece5d99b89d7786bd9d0a223e15b8dc963657c5c0bf21df88e4f5da614d75
|
|
BLAKE2b-256 checksum How to use checksums |
8ed71c178141fb02f13e6e69c6927a0a301ff6d907628c2addb948903d705c35
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 28, 2026.
Transparency log
