FireSpark data processing utility library
Project description
FireSpark
FireSpark aims to provide Magna ML/MAS team a flexible and standardized library supporting data processing, management, dataset curation, and ETL related activities.
A dataset created using FireSpark is stored in Apache Parquet format. On top of a Parquet schema, FireSpark takes advantage of open source Petastorm library to support multidimensional arrays.
This repo is at its early phase development stage. Please contact me if you have question, especially on contributing use case specification, requirements, suggestions. :innocent:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
FireSpark-0.0.23.tar.gz
(33.9 kB
view hashes)
Built Distribution
FireSpark-0.0.23-py3-none-any.whl
(43.5 kB
view hashes)
Close
Hashes for FireSpark-0.0.23-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e50bee3b2d83341fe80bfe9281dc136c864e1516548d922317a8f356e5102d44 |
|
MD5 | c6f1842e19306eb070e89bcf8cfd3910 |
|
BLAKE2b-256 | babe590152dfc244cd7e2fc390734549977179014790bdeaaeb96955cf860681 |