iparq 0.1.1

Display version and compression information about a parquet file

iparq

After reading this blog, I began to wonder which Parquet version and compression methods the everyday tools we rely on actually use, only to find that there’s no straightforward way to determine this. That curiosity and the difficulty of quickly discovering such details motivated me to create ipq (Information Parquet). My goal with ipq is to help users easily identify the specifics of the Parquet files generated by different engines, making it clear which features—like newer encodings or certain compression algorithms—the creator of the parquet is using.

Installation

Using pip

1. Install the package using pip:

   pip install iparq
   

2. Verify the installation by running:

   iparq --help
   

Using uv

1. Clone the repo:

   git clone https://github.com/MiguelElGallo/iparq.git
   cd iparq
   

2. Make sure to have Astral’s UV installed by following the steps here:

   https://docs.astral.sh/uv/getting-started/installation/

3. Execute the following command:

   uv run iparq <filename>
   

Replace <filename> with the path to your .parquet file. The utility will read the metadata of the file and print the compression codecs used in the parquet file.

Example output

ParquetMetaModel(
    created_by='parquet-cpp-arrow version 14.0.2',
    num_columns=3,
    num_rows=3,
    num_row_groups=1,
    format_version='2.6',
    serialized_size=2223
)
Compression codecs: {'SNAPPY'}