recap-core 0.8.0

Read schemas from web service, databases, and schema registries in a standard format

What is Recap?

Recap is a Python library that reads and writes schemas from web services, databases, and schema registries in a standard format.

Use Cases

• Compare schemas
• Check schema compatibility
• Store schemas in a catalog or registry
• Transpile schemas
• Transform schemas

Supported Formats

Format	Read	Write
Avro	✅	✅
Protobuf	✅	✅
JSON Schema	✅	✅
Snowflake	✅
PostgreSQL	✅
MySQL	✅
BigQuery	✅
Confluent Schema Registry	✅
Hive Metastore	✅

Documentation

Recap's documentation is available at recap.build.

Supported Types

Recap borrows types from Apache Arrow's Schema.fbs and Apache Kafka's Schema.java.

• null
• bool
• int
• float
• string
• bytes
• list
• map
• struct
• enum
• union

Recap Format

Recap schemas can be stored in YAML, TOML, or JSON formats using Recap's type spec. Here’s a YAML example:

type: struct
fields:
  - name: id
    type: int
    bits: 64
    signed: false
  - name: email
    type: string
    bytes: 255

Usage

Install Recap:

pip install recap-core

Get a Recap schema from a Protobuf schema:

from recap.converters.protobuf import ProtobufConverter

protobuf_schema = """
message Person {
    string name = 1;
}
"""

recap_schema = ProtobufConverter().to_recap(protobuf_schema)

Or a Snowflake table:

import snowflake.connector
from recap.readers.snowflake import SnowflakeReader

with snowflake.connector.connect(...) as conn:
  recap_schema = SnowflakeReader(conn).to_recap("TABLE", "PUBLIC", "TESTDB")

Or Hive's Metastore:

from pymetastore import HMS
from recap.readers.hive_metastore import HiveMetastoreReader

with HMS.create(...) as conn:
  recap_schema = HiveMetastoreReader(conn).to_recap("testdb", "table")

And write the schema as an Avro schema:

from recap.converters.avro import AvroConverter
avro_schema = AvroConverter().from_recap(recap_schema)

Or as a Protobuf schema:

from recap.converters.protobuf import ProtobufConverter
protobuf_schema = ProtobufConverter().from_recap(recap_schema)

Warning

Recap is still a little baby application. It's going to wake up crying in the middle of the night. It's going to vomit on the floor once in a while. But if you give it some love and care, it'll be worth it. As time goes on, it'll grow up and be more mature. Bear with it.