Kedro
Project description
pydantic-kedro
Advanced serialization for Pydantic models via Kedro and fsspec.
This package implements custom Kedro "datasets" for both "pure" and "arbitrary" Pydantic models. You can also use it stand-alone, using Kedro just for serializing other object types.
Please see the documentation for a tutorial and more examples.
Usage with Kedro
You can use the [PydanticAutoDataSet][pydantic_kedro.PydanticAutoDataSet]
or any other dataset from pydantic-kedro
within your
Kedro catalog
to save your Pydantic models:
# conf/base/catalog.yml
my_pydantic_model:
type: pydantic_kedro.PydanticAutoDataSet
filepath: folder/my_model
Direct Dataset Usage
This example works for "pure", JSON-safe Pydantic models via
PydanticJsonDataSet
:
from pydantic import BaseModel
from pydantic_kedro import PydanticJsonDataSet
class MyPureModel(BaseModel):
"""Your custom Pydantic model with JSON-safe fields."""
x: int
y: str
obj = MyPureModel(x=1, y="why?")
# Create an in-memory (temporary) file via `fsspec` and save it
ds = PydanticJsonDataSet("memory://temporary-file.json")
ds.save(obj)
# We can re-load it from the same file
read_obj = ds.load()
assert read_obj.x == 1
Standalone Usage
You can also use pydantic-kedro
as a generic saving and loading engine for
Pydantic models:
from tempfile import TemporaryDirectory
from pydantic import BaseModel
from pydantic_kedro import load_model, save_model
class MyModel(BaseModel):
"""My custom model."""
name: str
# We can use any fsspec URL, so we'll make a temporary folder
with TemporaryDirectory() as tmpdir:
save_model(MyModel(name="foo"), f"{tmpdir}/my_model")
obj = load_model(f"{tmpdir}/my_model")
assert obj.name == "foo"
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pydantic_kedro-0.5.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c8eb7b05d3dee355804072883a9ffccfca69173ac49d47d13d3a4fa554edeb6e |
|
MD5 | 3941e75d6c9eb6c78507d80c652f8fc9 |
|
BLAKE2b-256 | 9a43ed3a45f5e30bc4562344c500e8ae0ba99e2687f0e1a2ff09d21ccd3822e2 |