Intake single YAML hierarchical catalog.
Project description
Welcome to Intake DAL (data access layer) plugin
This Intake plugin helps abstract a dataset over disparate storage systems (eg: bulk, streaming, serving, …). It also provides an easy way to specialize a hierarchical catalog to a default DAL storage system.
Sample Catalog source entry:
user_events: driver: dal args: default: 'local' storage: local: 'csv://{{ CATALOG_DIR }}/data/user_events.csv' serving: 'in-memory-kv://foo' batch: 'parquet://{{ CATALOG_DIR }}/data/user_events.parquet'
Example code using sample catalog:
# Specialize the catalog dal default storge mode datasources # to be "serving". cat = DalCatalog(path, storage_mode="serving") # reads from the serving storage system # using the in-memory-kv Intake plugin df = cat.user_events.read()
Project details
Release history Release notifications
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Filename, size | File type | Python version | Upload date | Hashes |
---|---|---|---|---|
Filename, size intake_dal-0.1.2-py3-none-any.whl (6.6 MB) | File type Wheel | Python version py3 | Upload date | Hashes View hashes |
Filename, size intake-dal-0.1.2.tar.gz (6.5 MB) | File type Source | Python version None | Upload date | Hashes View hashes |
Close
Hashes for intake_dal-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 096464674b68395ced63686806b8d21ea40be270a235367ac8900a40a97966be |
|
MD5 | 2ac4903ab58a81366808d6ea2d2a4118 |
|
BLAKE2-256 | 83c144fd61bd91f126333ea8bf8c19e40313d28aa319efdeb227666a11864813 |