FireSpark data processing utility library
Project description
FireSpark
FireSpark aims to provide Magna ML/MAS team a flexible and standardized library supporting data processing, management, dataset curation, and ETL related activities.
A dataset created using FireSpark is stored in Apache Parquet format. On top of a Parquet schema, FireSpark takes advantage of open source Petastorm library to support multidimensional arrays.
This repo is at its early phase development stage. Please contact me if you have question, especially on contributing use case specification, requirements, suggestions. :innocent:
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file FireSpark-0.0.32.tar.gz.
File metadata
- Download URL: FireSpark-0.0.32.tar.gz
- Upload date:
- Size: 324.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.4
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 386d2a411c3c714b52a6a50428d4874867f965d0a181295a0b58c50b0b17c7f3 |
|
| MD5 | 69556479a0baa812110b679c830ef761 |
|
| BLAKE2b-256 | 2606b2466e8c01e47cdf9837506c60810cb6c325d07fcdf5e2d8e57840241b81 |
File details
Details for the file FireSpark-0.0.32-py3-none-any.whl.
File metadata
- Download URL: FireSpark-0.0.32-py3-none-any.whl
- Upload date:
- Size: 535.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.4
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | ed728ccbe9c4fcf7452d75894346c4241a7ac0302a2f0c802e69b5587a7ac97a |
|
| MD5 | df9de50d254c7ab5c3c3f12b723363b7 |
|
| BLAKE2b-256 | 673837e1d2fcdc0c357f478d3aaa95f4b408bc57ea8752a0aee63302890a18b6 |
