ccaf-avro-schema-helpers-python-lib 0.3.2.0

CCAF Avro Schema Helpers Python Library

Confluent Cloud for Apache Flink (CCAF) Avro Schema Helpers Python Library

Table of Contents

• 1.0 Overview
  • 1.1 generate_flink_sql_statements_for_fully_flatten_root_record module
  • 1.2 swap_camelcase_with_snakecase module
• 2.0 Resources
  • 2.1 Traversal Method Background
  • 2.2 Confluent Cloud for Apache Flink (CCAF) SQL Statements

1.0 Overview

1.1 generate_flink_sql_statements_for_fully_flatten_root_record module

This module's class constructs a pair of Flink SQL statements using a breadth-first traversal method from an Avro schema based on the provided outermost JSON object or JSON array column. These Flink SQL statements include the CREATE TABLE and INSERT INTO SELECT FROM statements. The CREATE TABLE statement creates the Sink Table, which subsequently establishes the backing sink Kafka topic. The INSERT INTO SELECT FROM statement generates a continuous, unbounded data stream that populates the sink table

1.2 swap_camelcase_with_snakecase module

This module's class uses a depth-first traversal method to traverse an entire Avro schema and converts every camelCase record or field name into a snake_case record or field name. This is useful for transforming Avro schema field names into Flink SQL style field names.

2.0 Resources

2.1 Traversal Method Background

• Breadth First Search Traversal
• Depth First Search Traversal

2.2 Confluent Cloud for Apache Flink (CCAF) SQL Statements

• CREATE TABLE Statement in Confluent Cloud for Apache Flink
• SELECT Statement in Confluent Cloud for Apache Flink
• WITH Clause in Confluent Cloud for Apache Flink
• INSERT INTO FROM SELECT Statement in Confluent Cloud for Apache Flink