cdklabs.cdk-construct-connect-datalake 0.0.10

Construct library for Amazon Connect Data Lake

Amazon Connect Data Lake CDK Construct

An AWS Cloud Development Kit (CDK) construct that enables access to Amazon Connect analytics data lake. This solution automates the complete Connect Data Lake setup process, eliminating the need for manual configuration or custom CloudFormation templates.

The construct uses a Lambda-backed custom resource to manage the deployment process. It handles associating Connect datasets, accepting RAM resource shares, granting Lake Formation permissions, and creating resource link tables in a centralized Glue database—with support for same-account and cross-account configurations.

Usage

Prerequisites

• Amazon Connect instance
• AWS CDK v2
• For cross-account setups: An IAM role in the target account. See Cross Account Setup documentation

Installation

Install the construct library in your CDK project directory:

TypeScript/JavaScript

npm install @cdklabs/cdk-construct-connect-datalake

Python

pip install cdklabs.cdk-construct-connect-datalake

Java

Add the following dependency to your pom.xml:

<dependency>
  <groupId>io.github.cdklabs</groupId>
  <artifactId>cdk-construct-connect-datalake</artifactId>
  <version>VERSION</version>
</dependency>

.NET

dotnet add package Cdklabs.CdkConstructConnectDatalake

Go

go get github.com/cdklabs/cdk-construct-connect-datalake-go/cdkconstructconnectdatalake

Basic Usage

Add the DataLakeAccess construct to a CDK stack deployed in the same AWS account and region as your Amazon Connect instance.

from cdklabs.cdk_construct_connect_datalake import DataLakeAccess, DataType


DataLakeAccess(self, "DataLakeAccess",
    instance_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",  # Your Connect instance ID
    dataset_ids=[DataType.CONTACT_RECORD, "contact_statistic_record"
    ]
)

Important: When deploying alongside a Connect instance in the same stack, add a dependency to the construct:

Example

from cdklabs.cdk_construct_connect_datalake import DataLakeAccess, DataType
from aws_cdk.aws_connect import CfnInstance


connect_instance = CfnInstance(self, "ConnectInstance",
    identity_management_type="CONNECT_MANAGED",
    instance_alias="my-instance",
    attributes=CfnInstance.AttributesProperty(
        inbound_calls=True,
        outbound_calls=True
    )
)

data_lake = DataLakeAccess(self, "DataLakeAccess",
    instance_id=connect_instance.attr_id,
    dataset_ids=[DataType.CONTACT_RECORD]
)

# Ensure data lake resources are deleted before the Connect instance
data_lake.node.add_dependency(connect_instance)

Cross-Account Configuration

Configure the construct to create data lake resources in a different AWS account by specifying targetAccountId and targetAccountRoleArn. The construct assumes the target role to accept the RAM resource share(s) and create Glue resources in that account.

from cdklabs.cdk_construct_connect_datalake import DataLakeAccess, DataType


DataLakeAccess(self, "DataLakeAccess",
    instance_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    dataset_ids=[DataType.CONTACT_RECORD, "contact_statistic_record"
    ],

    # Target account where the resources are created
    target_account_id="123456789012",

    # IAM role in the target account for cross-account role assumption
    target_account_role_arn="arn:aws:iam::123456789012:role/RoleName"
)

Multiple Instances

Enable data lake access for multiple Connect instances by creating a separate construct for each. A dependency should be added between them to ensure sequential deployment, preventing conflicts from concurrent operations.

from cdklabs.cdk_construct_connect_datalake import DataLakeAccess, DataType


# First Connect instance data lake setup
data_lake1 = DataLakeAccess(self, "DataLakeAccess1",
    instance_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    dataset_ids=[DataType.CONTACT_RECORD, DataType.AGENT_STATISTIC_RECORD
    ]
)

# Second Connect instance data lake setup
data_lake2 = DataLakeAccess(self, "DataLakeAccess2",
    instance_id="yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy",
    dataset_ids=[DataType.CONTACT_RECORD, DataType.CONTACT_FLOW_EVENTS
    ]
)

# Create dependency to ensure sequential deployment
data_lake2.node.add_dependency(data_lake1)

API Reference

DataLakeAccess

The main construct class for setting up Amazon Connect Data Lake integration.

Properties:

• instanceId (string): Amazon Connect instance ID
• datasetIds (Array<string | DataType>): Array of dataset IDs to associate. Use DataType enum values or string literals for datasets not yet in the enum.
• targetAccountId? (string): Target AWS account ID receiving resources (optional)
• targetAccountRoleArn? (string): IAM role ARN in the target account for cross-account role assumption (optional)

DataType Enum

For a list of supported dataset types, see the API Documentation.

Resources Created

This construct creates the following AWS resources:

Infrastructure Components

• CloudFormation Custom Resource Provider: Framework for managing custom resource lifecycle

• Lambda Function: Custom resource handler that orchestrates the data lake setup

• IAM Role: Execution role with permissions for Connect, RAM, Glue, and Lake Formation operations

  Show IAM permissions

  • connect:BatchAssociateAnalyticsDataSet
  • connect:AssociateAnalyticsDataSet
  • connect:BatchDisassociateAnalyticsDataSet
  • connect:DisassociateAnalyticsDataSet
  • connect:ListAnalyticsDataAssociations
  • connect:ListAnalyticsDataLakeDataSets
  • connect:ListInstances
  • ds:DescribeDirectories
  • ram:AcceptResourceShareInvitation
  • ram:GetResourceShareInvitations
  • ram:GetResourceShares
  • glue:CreateDatabase
  • glue:CreateTable
  • glue:DeleteDatabase
  • glue:DeleteTable
  • glue:GetDatabase
  • glue:GetTables
  • lakeformation:GetDataLakeSettings
  • lakeformation:PutDataLakeSettings
  • cloudformation:DescribeStacks
  • sts:AssumeRole (for cross-account setups only)

Deployment Workflow

The construct performs the following steps during deployment:

1. Dataset Association: Associates the specified datasets for an Amazon Connect instance with the target account
2. Database Creation: Creates the connect_datalake_database Glue database
3. Lake Formation Setup: Configures the Lambda execution role (or assumed role for cross-account) as a data lake administrator
4. Resource Share Acceptance: Accepts the RAM resource share invitation(s). Multiple dataset associations often consolidate into a single RAM resource share
5. Table Creation: Creates resource link tables for each dataset, enabling queries via Amazon Athena

When deploying to the same account as the Connect instance, all steps execute within that account. For cross-account configurations, steps 2-5 execute in the target account.

Limitations

• Table Naming: Resource link tables created by this construct are named using the format {datasetId}_{dataCatalogId}
• Region Support: The construct must be deployed in the same AWS region and account as the Amazon Connect instance. For cross-account configurations, resources are created in the target account within the same region
• Shared Database: The connect_datalake_database Glue database is shared across all deployments of this construct in an account

Troubleshooting

Partial failures during deployment

• If some workflow steps fail during create or update operations, the stack deployment will still show as successful. Error details for these partial failures are available in the CloudFormation stack outputs.

RAM resource share has expired

• Resource shares for new dataset associations can consolidate into existing AWS RAM shares, even if expired. Delete each construct that references the target account, confirm the associated resources are removed, then redeploy using the original construct definitions.

Failure to update Lake Formation permissions due to invalid principal

• IAM roles that have been deleted but not removed from Lake Formation principals will be considered invalid. Remove the principal causing this error from Lake Formation and redeploy the construct.

Resources are unable to be removed after a Connect instance has been deleted

• Constructs of this type must be deleted prior to deleting the instance, as cleanup after instance deletion is currently not supported. A GitHub issue can be raised if assistance removing these resources is required.

Support

For issues and questions:

• Reference the documentation for the analytics data lake in the Amazon Connect Administrator Guide
• Check the API Documentation
• Report bugs via GitHub Issues

Contributing

We welcome contributions! Please see our Contributing Guide for details.

License

This project is licensed under the Apache-2.0 License.