recap-core 0.6.0

Read schemas from web service, databases, and schema registries in a standard format

What is Recap?

Recap is a Python library that reads and converts web service schemas, database schemas, and schema registry schemas in a standard way.

Your data passes through web services, databases, message brokers, and object stores. Recap describes these schemas in a single language, regardless of which system your data passes through.

Format

Recap schemas use Recap's type spec. Schemas can be serialized in YAML, TOML, JSON, XML, or any other compatible language. Here’s a YAML example:

type: struct
fields:
  - name: id
    type: int
    bits: 64
    signed: false
  - name: email
    type: string
    bytes: 255

Features

Get Recap schemas for...

• Serialization formats (Avro, Protobuf, and JSON Schema)
• Databases (Snowflake and PostgreSQL)
• Schema registries (Confluent Schema Registry and Hive Metastore)

Usage

Install Recap:

pip install recap-core

Get a Recap schema from a Protobuf message:

from recap.converters.protobuf import ProtobufConverter

protobuf_schema = """
message Person {
    string name = 1;
}
"""

recap_schema = ProtobufConverter().to_recap(protobuf_schema)

Or a Snowflake table:

import snowflake.connector
from recap.readers.snowflake import SnowflakeReader

with snowflake.connector.connect(...) as conn:
  recap_schema = SnowflakeReader(conn).to_recap("TABLE", "PUBLIC", "TESTDB")

Or Hive's Metastore:

from pymetastore import HMS
from recap.readers.hive_metastore import HiveMetastoreReader

with HMS.create(...) as conn:
  recap_schema = HiveMetastoreReader(conn).to_recap("testdb", "table")

Warning

Recap is still a little baby application. It's going to wake up crying in the middle of the night. It's going to vomit on the floor once in a while. But if you give it some love and care, it'll be worth it. As time goes on, it'll grow up and be more mature. Bear with it.