Skip to main content

A small package to optimize memory usage of Pandas DataFrames

Project description

DF-Squeezer

DF-Squeezer is a Python package designed to help reduce the memory usage of pandas DataFrames. By analyzing the data types of DataFrame columns, DF-Squeezer suggests and can automatically apply data type conversions that are more memory-efficient without losing information.

Features

  • Analysis of DataFrame Columns: Identifies columns that can be converted to more memory-efficient data types.
  • Memory Usage Reporting: Calculates and reports the potential memory savings from suggested conversions.
  • Automatic Conversion Application: Optionally applies suggested conversions directly to the DataFrame.

Installation

To install DF-Squeezer, run the following command in your terminal:

pip install df-squeezer

Ensure you have pandas installed in your environment as it is a required dependency.

Usage

from df_squeezer import df_squeezer
import pandas as pd

# Sample DataFrame
df = pd.DataFrame({
    'a': range(1000),
    'b': [float(i) for i in range(1000)],
    'c': ['category' + str(i % 3) for i in range(1000)]
})

# Use df_squeeze to analyze and report potential dtype conversions
optimized_df = df_squeezer(df, report=True, edit=False)

# To directly apply the suggested conversions
optimized_df = df_squeezer(df, report=True, edit=True)


Parameters

df (pd.DataFrame): The input DataFrame to analyze and optimize. report (bool, optional): If True (default), prints suggested changes for data type conversions along with potential memory savings. edit (bool, optional): If False (default), the function only reports potential optimizations without applying them. If True, the function applies the suggested data type conversions directly to the DataFrame. Returns pd.DataFrame: The original DataFrame with applied changes if edit=True. Otherwise, returns the original DataFrame unmodified. Contributing Contributions to DF-Squeezer are welcome! Please refer to the contribution guidelines for more information.

License This project is licensed under the MIT License - see the LICENSE file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

df-squeezer-0.0.5.tar.gz (2.9 kB view details)

Uploaded Source

File details

Details for the file df-squeezer-0.0.5.tar.gz.

File metadata

  • Download URL: df-squeezer-0.0.5.tar.gz
  • Upload date:
  • Size: 2.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.11.7

File hashes

Hashes for df-squeezer-0.0.5.tar.gz
Algorithm Hash digest
SHA256 c67c09c9e6816c57d26e51614116ef5b0929b48bb9c375610daa32e130e9af58
MD5 8d3ac9b21fd90793751c7acd197f223c
BLAKE2b-256 e38481f55a1467cea42357fd74c3e1906dd40465621e56508b144105164c28c6

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page