Avro reader for Dask.
Project description
Dask-Avro
Avro reader for Dask.
Free software: MIT license
Documentation: https://dask-avro.readthedocs.org.
Python versions: 2.7, 3.5+
Features
This projects provides an Avro format reader for Dask. Provides a convenient function to read one or more Avro files and partition them arbitrarily.
Quickstart
Usage:
import dask.bag import dask_avro delayeds = dask_avro.read_avro("data-*.avro", blocksize=2**26) data = dask.bag.from_delayed(delayeds)
Credits
This package was created with Cookiecutter and the rmax/cookiecutter-pypackage project template.
History
0.2.0 (2018-02-12)
Added support for fastavro 0.16+.
0.1.2 (2018-02-12)
Fix compatibility with dask 0.17.0.
0.1.1 (2018-01-18)
Pin fastavro version to <0.16 as latest versions don’t allow to use internal C-based _iter_avro function.
0.1.0 (2017-02-02)
First release on PyPI.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dask-avro-0.2.0.tar.gz
(11.0 MB
view hashes)
Built Distribution
Close
Hashes for dask_avro-0.2.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1b58655c066dbc0d852531a2c6f2b6c0ff74b8c4db065237fb4394107828a7f3 |
|
MD5 | d1eb03ef1615617235c9c721e6ab75ac |
|
BLAKE2b-256 | 9669224d207104f19139d804914ae7aa4a1b7970b964474c4a101240e59e994b |