Skip to main content

A small package to optimize memory usage of Pandas DataFrames

Project description

DF_Squeezer

DF_Squeezer is a Python package designed to help reduce the memory usage of pandas DataFrames. By analyzing the data types of DataFrame columns, DF-Squeezer suggests and can automatically apply data type conversions that are more memory-efficient without losing information.

Features

  • Analysis of DataFrame Columns: Identifies columns that can be converted to more memory-efficient data types.
  • Memory Usage Reporting: Calculates and reports the potential memory savings from suggested conversions.
  • Automatic Conversion Application: Optionally applies suggested conversions directly to the DataFrame.

Installation

To install DF-Squeezer, run the following command in your terminal:

pip install df_squeezer

Ensure you have pandas installed in your environment as it is a required dependency.

Usage

from df_squeezer import df_squeezer
import pandas as pd

# Sample DataFrame
df = pd.DataFrame(
    {
        "a": range(1000),
        "b": [float(i) for i in range(1000)],
        "c": ["category" + str(i % 3) for i in range(1000)],
    }
)

# Use df_squeeze to analyze and report potential dtype conversions
optimized_df = df_squeezer(df, report=True, edit=False)

# To directly apply the suggested conversions
optimized_df = df_squeezer(df, report=True, edit=True)

Parameters

df (pd.DataFrame): The input DataFrame to analyze and optimize. report (bool, optional): If True (default), prints suggested changes for data type conversions along with potential memory savings. edit (bool, optional): If False (default), the function only reports potential optimizations without applying them. If True, the function applies the suggested data type conversions directly to the DataFrame. Returns pd.DataFrame: The original DataFrame with applied changes if edit=True. Otherwise, returns the original DataFrame unmodified. Contributing Contributions to DF-Squeezer are welcome! Please refer to the contribution guidelines for more information.

License This project is licensed under the MIT License - see the LICENSE file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

df_squeezer-0.0.12.tar.gz (3.1 kB view details)

Uploaded Source

File details

Details for the file df_squeezer-0.0.12.tar.gz.

File metadata

  • Download URL: df_squeezer-0.0.12.tar.gz
  • Upload date:
  • Size: 3.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.11.5

File hashes

Hashes for df_squeezer-0.0.12.tar.gz
Algorithm Hash digest
SHA256 9dcd97bb89562278f1539409d1c133340471d209d16c7ebc630f8c6a90bd6ad9
MD5 8dccdadf48f48346a6c54d2a2a11a225
BLAKE2b-256 5c569883c2e796f3fd191f529b0b65d2f9f856dfa6d3751d69e8b9680d2178f0

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page