Skip to main content

CDK Constructs for S3 Tables

Project description

Amazon S3 Tables Construct Library

---

cdk-constructs: Experimental

The APIs of higher level constructs in this module are experimental and under active development. They are subject to non-backward compatible changes or removal in any future version. These are not subject to the Semantic Versioning model and breaking changes will be announced in the release notes. This means that while you may use them, you may need to update your source code when upgrading to a newer version of this package.


Amazon S3 Tables

Amazon S3 Tables deliver the first cloud object store with built-in Apache Iceberg support and streamline storing tabular data at scale.

Product Page | User Guide

Usage

Define an S3 Table Bucket

# Build a Table bucket
sample_table_bucket = TableBucket(scope, "ExampleTableBucket",
    table_bucket_name="example-bucket-1",
    # optional fields:
    unreferenced_file_removal=UnreferencedFileRemoval(
        status=UnreferencedFileRemovalStatus.ENABLED,
        noncurrent_days=20,
        unreferenced_days=20
    )
)

Define an S3 Tables Namespace

# Build a namespace
sample_namespace = Namespace(scope, "ExampleNamespace",
    namespace_name="example-namespace-1",
    table_bucket=table_bucket
)

Define an S3 Table

# Build a table
sample_table = Table(scope, "ExampleTable",
    table_name="example_table",
    namespace=namespace,
    open_table_format=OpenTableFormat.ICEBERG,
    without_metadata=True
)

# Build a table with an Iceberg Schema
sample_table_with_schema = Table(scope, "ExampleSchemaTable",
    table_name="example_table_with_schema",
    namespace=namespace,
    open_table_format=OpenTableFormat.ICEBERG,
    iceberg_metadata=IcebergMetadataProperty(
        iceberg_schema=IcebergSchemaProperty(
            schema_field_list=[SchemaFieldProperty(
                name="id",
                type="int",
                required=True
            ), SchemaFieldProperty(
                name="name",
                type="string"
            )
            ]
        )
    ),
    compaction=CompactionProperty(
        status=Status.ENABLED,
        target_file_size_mb=128
    ),
    snapshot_management=SnapshotManagementProperty(
        status=Status.ENABLED,
        max_snapshot_age_hours=48,
        min_snapshots_to_keep=5
    )
)

Learn more about table buckets maintenance operations and default behavior from the S3 Tables User Guide

Controlling Table Bucket Permissions

# Grant the principal read permissions to the bucket and all tables within
account_id = "123456789012"
table_bucket.grant_read(iam.AccountPrincipal(account_id), "*")

# Grant the role write permissions to the bucket and all tables within
role = iam.Role(stack, "MyRole", assumed_by=iam.ServicePrincipal("sample"))
table_bucket.grant_write(role, "*")

# Grant the user read and write permissions to the bucket and all tables within
table_bucket.grant_read_write(iam.User(stack, "MyUser"), "*")

# Grant permissions to the bucket and a particular table within it
table_id = "6ba046b2-26de-44cf-9144-0c7862593a7b"
table_bucket.grant_read_write(iam.AccountPrincipal(account_id), table_id)

# Add custom resource policy statements
permissions = iam.PolicyStatement(
    effect=iam.Effect.ALLOW,
    actions=["s3tables:*"],
    principals=[iam.ServicePrincipal("example.aws.internal")],
    resources=["*"]
)

table_bucket.add_to_resource_policy(permissions)

Controlling Table Bucket Encryption Settings

S3 TableBuckets have SSE (server-side encryption with AES-256) enabled by default with S3 managed keys. You can also bring your own KMS key for KMS-SSE or have S3 create a KMS key for you.

If a bucket is encrypted with KMS, grant functions on the bucket will also grant access to the TableBucket's associated KMS key.

# Provide a user defined KMS Key:
key = kms.Key(scope, "UserKey")
encrypted_bucket = TableBucket(scope, "EncryptedTableBucket",
    table_bucket_name="table-bucket-1",
    encryption=TableBucketEncryption.KMS,
    encryption_key=key
)
# This account principal will also receive kms:Decrypt access to the KMS key
encrypted_bucket.grant_read(iam.AccountPrincipal("123456789012"), "*")

# Use S3 managed server side encryption (default)
encrypted_bucket_default = TableBucket(scope, "EncryptedTableBucketDefault",
    table_bucket_name="table-bucket-3",
    encryption=TableBucketEncryption.S3_MANAGED
)

When using KMS encryption (TableBucketEncryption.KMS), if no encryption key is provided, CDK will automatically create a new KMS key for the table bucket with necessary permissions.

# If no key is provided, one will be created automatically
encrypted_bucket_auto = TableBucket(scope, "EncryptedTableBucketAuto",
    table_bucket_name="table-bucket-2",
    encryption=TableBucketEncryption.KMS
)

Controlling Table Permissions

# Grant the principal read permissions to the table
account_id = "123456789012"
table.grant_read(iam.AccountPrincipal(account_id))

# Grant the role write permissions to the table
role = iam.Role(stack, "MyRole", assumed_by=iam.ServicePrincipal("sample"))
table.grant_write(role)

# Grant the user read and write permissions to the table
table.grant_read_write(iam.User(stack, "MyUser"))

# Grant an account permissions to the table
table.grant_read_write(iam.AccountPrincipal(account_id))

# Add custom resource policy statements
permissions = iam.PolicyStatement(
    effect=iam.Effect.ALLOW,
    actions=["s3tables:*"],
    principals=[iam.ServicePrincipal("example.aws.internal")],
    resources=["*"]
)

table.add_to_resource_policy(permissions)

Coming Soon

L2 Construct support for:

  • KMS encryption support for Tables

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aws_cdk_aws_s3tables_alpha-2.232.2a0.tar.gz (127.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

aws_cdk_aws_s3tables_alpha-2.232.2a0-py3-none-any.whl (127.7 kB view details)

Uploaded Python 3

File details

Details for the file aws_cdk_aws_s3tables_alpha-2.232.2a0.tar.gz.

File metadata

File hashes

Hashes for aws_cdk_aws_s3tables_alpha-2.232.2a0.tar.gz
Algorithm Hash digest
SHA256 c2b9af6151b6b46984dd63605caa0e80bec8d1340f81c4e8448d4cb1f48390b4
MD5 ff2a4507f223c6e17f1733a6c69c3ff5
BLAKE2b-256 896da173e6f683933bb74b6624207db39db4a09a95ca3e56906062e7443d84bb

See more details on using hashes here.

File details

Details for the file aws_cdk_aws_s3tables_alpha-2.232.2a0-py3-none-any.whl.

File metadata

File hashes

Hashes for aws_cdk_aws_s3tables_alpha-2.232.2a0-py3-none-any.whl
Algorithm Hash digest
SHA256 0e7c9884f48b2158d78b56e4b65e49a52a07aa4835a7882345eafc7cec504768
MD5 a3a072250fc2830940491dc88710e873
BLAKE2b-256 2fecb5924a9121fc501ce7c009bbe1b6d873e579aec18e750a01c0f81460505d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page