Cut memory footprint by half in just 2 lines of code. Compress Pandas DataFrame without losing information.
Project description
pandazip
Go minimal, go green, go pandazip.
Cut memory footprint by half in just 2 lines of code. Compress Pandas DataFrame without/with losing information. Swift parallel execution.
Install
Shap can be installed from PyPI:
pip install pandazip
Compressing Pandas DataFrame using Pandazip
from pandazip import Pandazip
compressed_dataframe = Pandazip().zip(raw_dataframe)
Lossless Compression
Compression level can be tuned with level parameter. Default is level="low", which is lossless. Every column is converted to the smallest datatype that can store column's data without losing information.
compressed_dataframe = Pandazip().zip(raw_dataframe, level="low")
Lossly Compression
When level parameter is set to "mid" or "high", Pazdazip limits numeric datatypes to 32 and 16 bits respectively. Also, string columns are converted to categoric datatype, if feasable.
compressed_dataframe = Pandazip().zip(raw_dataframe, level="high")
Results
Pandazip is tested on more than 100 Kaggle datasets and notebooks, feel free to share your results.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
