A set of tools to ease Dagster usage
Project description
Dagster Toolbox
A set of tools to ease Dagster usage
Requirements
- Python 3.10+
- Dagster 1.6.13 / 0.22.13
Installation
pip install dagster-toolbox
Usage
New object storage resource
Declare resource
from dagster_toolbox.resources import ObjectStorage
class Analytics(ObjectStorage):
bucket = "analytics"
@resource
def analytics_objects(init_context):
return Analytics()
RESOURCE_DEFS = {
"analytics_objects": analytics_objects,
}
Use resource
@asset
def export_files(context):
partition_key = context.asset_partition_key_for_output()
file_path = f"analytics/data/{partition_key}"
file_resource = context.resource.analytics_objects.read(file_path)
...
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dagster_toolbox-0.0.15.tar.gz
(15.7 kB
view hashes)
Built Distribution
Close
Hashes for dagster_toolbox-0.0.15-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0ef839e6b4c90ff7f5aad5e4d1e5d368352ece56bd2ee080c70ba0f7fd017e49 |
|
MD5 | cf2fa1d4b2a1fe54ee3588f542f43374 |
|
BLAKE2b-256 | 73221e5f821981f0450241dcf200e129bdc8d4fa2334e1ce5bfd9741557ec5a6 |