Cut memory footprint by half in just 2 lines of code. Compress Pandas DataFrame without losing information.
Project description
pandazip
Go minimal, go green, go pandazip.
Cut memory footprint by half in just 2 lines of code. Compress Pandas DataFrame without/with losing information. Swift parallel execution.
Install
Shap can be installed from PyPI:
pip install pandazip
Compressing Pandas DataFrame using Pandazip
from pandazip import Pandazip compressed_dataframe = Pandazip().zip(raw_dataframe)
Lossless Compression
Compression level can be tuned with level parameter. Default is level="low", which is lossless. Every column is converted to the smallest datatype that can store column's data without losing information.
compressed_dataframe = Pandazip().zip(raw_dataframe, level="low")
Lossly Compression
When level parameter is set to "mid" or "high", Pazdazip limits numeric datatypes to 32 and 16 bits respectively. Also, string columns are converted to categoric datatype, if feasable.
compressed_dataframe = Pandazip().zip(raw_dataframe, level="high")
Results
Pandazip is tested on more than 100 Kaggle datasets and notebooks, feel free to share your results.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
| Filename, size | File type | Python version | Upload date | Hashes |
|---|---|---|---|---|
| Filename, size pandazip-0.0.6-py3-none-any.whl (4.2 kB) | File type Wheel | Python version py3 | Upload date | Hashes View |
| Filename, size pandazip-0.0.6.tar.gz (3.2 kB) | File type Source | Python version None | Upload date | Hashes View |