A library for converting avro schemas to python types.
Project description
avro-to-python-types
A library for converting avro schemas to python types.
Currently, it supports converting record
s to TypedDict
. If you would like to see more features added, please open up an issue.
Why would I want this?
This library is targeted to people writing code generation for python apps that are using avro.
Usage
This library does one thing, it converts Avro schemas to python types.
To get up and running quickly, you can use this to simply load schemas and print out the python code that is generated.
import glob
from avro_to_python_types import typed_dict_from_schema_file
schema_files = glob.glob("schemas/*.avsc")
for schema_file in schema_files:
types = typed_dict_from_schema_file(schema_file)
print(types)
For a real world example of syncing a directory of schemas into a directory of matching python typed dictionaries check out the example app here
To try it out, simply clone this repo and run:
poetry env use 3.9
- This must be done as this library only supports Python 3.9 and above for type generation. You can still use this library in apps that use a lower Python version, as long as Python 3.9 is the active version when the types are generated (either locally or in your CI system).
poetry install
poetry run sync-example
For some more advanced examples, like referencing other schema files by their full name take a look at the tests here
Referencing schemas
This library supports referencing schemas in different files by their fullname.
In order for this behaviour to work, all schemas must be in the same directory and use the following naming convention: namespace.name.avsc
. Note that is the same as fullname.avsc
For more on this checkout the docs for fastavro here.
An example of this can be found in the tests.
Example output
The following example shows the type generated for a given schema.
{
"namespace": "example",
"type": "record",
"name": "User",
"fields": [
{ "name": "name", "type": "string" },
{ "name": "favorite_number", "type": ["null", "int"] },
{ "name": "favorite_color", "type": ["null", "string"] },
{
"name": "address",
"type": {
"type": "record",
"name": "AddressUSRecord",
"fields": [
{ "name": "streetaddress", "type": "string" },
{ "name": "city", "type": "string" }
]
}
},
{
"name": "other_thing",
"type": {
"type": "record",
"name": "OtherThing",
"fields": [
{ "name": "thing1", "type": "string" },
{ "name": "thing2", "type": ["null", "int"] }
]
}
}
]
}
from typing import TypedDict, Optional
# total=False allows us to skip passing optional fields into the constructor
class ExampleAddressUSRecord(TypedDict, total=False):
streetaddress: str
city: str
class ExampleOtherThing(TypedDict, total=False):
thing1: str
thing2: Optional[int]
class ExampleUser(TypedDict, total=False):
name: str
favorite_number: Optional[int]
favorite_color: Optional[str]
address: AddressUSRecord
other_thing: OtherThing
Testing
To run unit tests, run poetry run pytest
.
Requirements
Python 3.9 or greater.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for avro-to-python-types-0.12.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | ea3913cf7aad9c74bd6520877f7796e3de2ad57eedc7f87a71d4377b4751c409 |
|
MD5 | e28b9a9c15a3e652a2b79fe6a4367001 |
|
BLAKE2b-256 | 90dcaf3d71376fcc7dadccb6ac11c76f1f5a7587337496923349b5e38b6ebb68 |
Hashes for avro_to_python_types-0.12.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e6c437cf2b5ba0206493cba58dfeb488c14768518614eeb9dd4730a9b16eae9c |
|
MD5 | e6c19dd2d53dce095714cfa7a1d5b3b0 |
|
BLAKE2b-256 | fb29c1d9d26d12c0f94a6b6b98ff58a383123f56c99770559ef0c9c1a7428f68 |