Truthlocks Delta Lake integration

Build structured, hash-addressed evidence from Delta Lake metadata for your application's audit workflow.

Package: delta-lake-maip · Python: 3.9+ · License: Apache-2.0

Developer documentation · Console

Install

python -m pip install delta-lake-maip

Use a virtual environment and lock compatible framework versions with your application. The local example below needs no hosted credentials and makes no external requests.

Quickstart: create local evidence

from delta_lake_maip import DeltaMAIPConfig, generate_write_receipt, hash_receipt_payload

receipt = generate_write_receipt(
    table_path="./warehouse/events", version=1, operation="WRITE",
    commit_info={"timestamp": 1700000000000},
    add_actions=[{"path": "part-0001.parquet", "size": 256,
                  "stats": {"numRecords": 2}}],
    remove_actions=[], schema_json='{"type":"struct","fields":[]}',
    config=DeltaMAIPConfig(auto_receipt=False),
)
assert receipt.payload_hash == hash_receipt_payload(receipt)
print(receipt.model_dump_json(indent=2))

Connect real data

Supply real commitInfo, add/remove actions, schema JSON and table version from your Delta transaction log. The example uses illustrative metadata; it neither creates a table nor reads Parquet data.

Available interfaces

generate_write_receipt, generate_merge_receipt and generate_schema_evolution_receipt construct local evidence. DeltaCommitListener and DeltaTableTracker support log observation; review storage access and submission behavior before attaching them to a live table.

Configuration

Evidence and delivery guarantees

The local result is a Pydantic model with a payload_hash. Save it using model_dump_json() and retain the metadata needed to reproduce it. Recomputing the hash checks consistency of that local representation; it does not validate a server signature, establish a trusted identity, or confirm transparency-log inclusion.

Table statistics and schema hashes describe supplied transaction metadata, not an independently audited copy of all table rows.

Submit and verify hosted evidence (1.1.0+)

Register a machine identity in your workspace before submission. Set MAIP_AGENT_ID to its identifier and MAIP_API_KEY to a workspace API key with access to machine identities and receipts. Provider keys are separate. A tl_test_ key selects the sandbox host unless MAIP_API_URL overrides it; production defaults to https://api.truthlocks.com/v1. Do not use a tenant header as authentication.

This example submits the local receipt created above only when credentials are set:

import asyncio
import os
from maip_ai_connectors.client import MAIPClient
from maip_ai_connectors.config import MAIPConfig

async def submit_and_verify(local_receipt):
    config = MAIPConfig(agent_id=os.environ["MAIP_AGENT_ID"],
                        api_key=os.environ["MAIP_API_KEY"])
    async with MAIPClient(config) as client:
        stored = await client.create_receipt(local_receipt)
        check = await client.verify_receipt(stored.receipt_id)
        if not check.valid or not check.signature_verified:
            raise RuntimeError(check.reason or check.verdict)
        print(stored.receipt_id, check.verdict)

if os.environ.get("MAIP_API_KEY") and os.environ.get("MAIP_AGENT_ID"):
    asyncio.run(submit_and_verify(receipt))

The server stores integration metadata in a signed MAIP action receipt. Its receipt_id differs from the local payload_hash. Verification checks the signature and current lifecycle state; it does not establish transparency-log inclusion. Old receipts whose signing key was lost return UNVERIFIABLE.

For automatic hooks, enable auto_receipt=True. In synchronous code delivery is confirmed before the hook returns. In asynchronous code, call await wrapper.flush() and await wrapper.aclose() before shutdown; inspect get_submitted_receipts() for server responses. These lifecycle methods apply to AI wrappers/callbacks; data clients use their async context manager or create_receipt_sync() instead. Read retries are bounded; writes are never automatically replayed. After a timeout, check the agent's receipts before retrying. Listing returns up to the latest 100 receipts for one agent, with integration filters applied to that page.

Troubleshooting

For support, share package and Python versions plus a sanitized example. Do not include keys, customer data or full authentication headers. No automatic compliance certification or fixed processing-latency guarantee is implied.