Skip to main content

A small package to optimize memory usage of Pandas DataFrames

Project description

DF_Squeezer

DF_Squeezer is a Python package designed to help reduce the memory usage of pandas DataFrames. By analyzing the data types of DataFrame columns, DF-Squeezer suggests and can automatically apply data type conversions that are more memory-efficient without losing information.

Features

  • Analysis of DataFrame Columns: Identifies columns that can be converted to more memory-efficient data types.
  • Memory Usage Reporting: Calculates and reports the potential memory savings from suggested conversions.
  • Automatic Conversion Application: Optionally applies suggested conversions directly to the DataFrame.

Installation

To install DF-Squeezer, run the following command in your terminal:

pip install df_squeezer

Ensure you have pandas installed in your environment as it is a required dependency.

Usage

from df_squeezer import df_squeezer
import pandas as pd

# Sample DataFrame
df = pd.DataFrame(
    {
        "a": range(1000),
        "b": [float(i) for i in range(1000)],
        "c": ["category" + str(i % 3) for i in range(1000)],
    }
)

# Use df_squeeze to analyze and report potential dtype conversions
optimized_df = df_squeezer(df, report=True, edit=False)

# To directly apply the suggested conversions
optimized_df = df_squeezer(df, report=True, edit=True)

Parameters

df (pd.DataFrame): The input DataFrame to analyze and optimize. report (bool, optional): If True (default), prints suggested changes for data type conversions along with potential memory savings. edit (bool, optional): If False (default), the function only reports potential optimizations without applying them. If True, the function applies the suggested data type conversions directly to the DataFrame. Returns pd.DataFrame: The original DataFrame with applied changes if edit=True. Otherwise, returns the original DataFrame unmodified. Contributing Contributions to DF-Squeezer are welcome! Please refer to the contribution guidelines for more information.

License This project is licensed under the MIT License - see the LICENSE file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

df_squeezer-0.0.6.tar.gz (2.9 kB view details)

Uploaded Source

File details

Details for the file df_squeezer-0.0.6.tar.gz.

File metadata

  • Download URL: df_squeezer-0.0.6.tar.gz
  • Upload date:
  • Size: 2.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.11.7

File hashes

Hashes for df_squeezer-0.0.6.tar.gz
Algorithm Hash digest
SHA256 859d629eee8c3c940981baef146165aa34b1040613cc4765666cd9ba0e0dac99
MD5 b2102569790094f343a9c1994b6de504
BLAKE2b-256 507b8b9c9281a3325ed674a39fcff9a006b482dcf2d39ef260f009b7892054e9

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page