Kedro
Project description
pydantic-kedro
Advanced serialization for Pydantic models via Kedro and fsspec.
This package implements custom Kedro "datasets" for both "pure" and "arbitrary" Pydantic models. You can also use it stand-alone, using Kedro just for serializing other object types.
Please see the documentation for a tutorial and more examples.
Usage with Kedro
You can use the [PydanticAutoDataset][pydantic_kedro.PydanticAutoDataset]
or any other dataset from pydantic-kedro
within your
Kedro catalog
to save your Pydantic models:
# conf/base/catalog.yml
my_pydantic_model:
type: pydantic_kedro.PydanticAutoDataset
filepath: folder/my_model
Direct Dataset Usage
This example works for "pure", JSON-safe Pydantic models via
PydanticJsonDataset
:
from pydantic import BaseModel
# from pydantic.v1 import BaseModel # Pydantic V2
from pydantic_kedro import PydanticJsonDataset
class MyPureModel(BaseModel):
"""Your custom Pydantic model with JSON-safe fields."""
x: int
y: str
obj = MyPureModel(x=1, y="why?")
# Create an in-memory (temporary) file via `fsspec` and save it
ds = PydanticJsonDataset("memory://temporary-file.json")
ds.save(obj)
# We can re-load it from the same file
read_obj = ds.load()
assert read_obj.x == 1
Standalone Usage
You can also use pydantic-kedro
as a generic saving and loading engine for
Pydantic models:
from tempfile import TemporaryDirectory
from pydantic.v1 import BaseModel
from pydantic_kedro import load_model, save_model
class MyModel(BaseModel):
"""My custom model."""
name: str
# We can use any fsspec URL, so we'll make a temporary folder
with TemporaryDirectory() as tmpdir:
save_model(MyModel(name="foo"), f"{tmpdir}/my_model")
obj = load_model(f"{tmpdir}/my_model")
assert obj.name == "foo"
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pydantic_kedro-0.8.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2652a7639119754c73f935f52557138a2b26d722208b0a5875093dc0b562854b |
|
MD5 | 5302cf1d7cf5086ac7a0bfab079befe7 |
|
BLAKE2b-256 | a62dff942492a6061a8e5a70f69df6d8c25c6f90f7049339c3a9ef59a126bd0f |