dagster-serde 0.0.0

Dagster library for ser/de of data

dagster-serde

Type-aware ser/de library for Dagster.

@dataclass
class MyStruct:
    foo: str


my_struct = MyStruct(foo="bar")


@asset(
    io_manager_key="json_io_manager",
)
def upstream() -> MyStruct:
    return my_struct  # my_struct is serialized to a json file


@asset
def downstream(
    upstream: MyStruct,
):  # my_struct is loaded as MyStruct based on the type hint
    assert upstream == my_struct

Installation

pip install dagster-serde

IOManagers

BaseSerdeUPathIOManager

Base class for IOManagers that ser/de to/from filesystems.

JsonIOManager

Implements BaseSerdeUPathIOManager for JSON files.

Examples

See examples.

Development

Installation

poetry install
poetry run pre-commit install

Testing

poetry run pytest