Skip to main content

A tool to convert CSVs to Parquet files

Project description

csv2parquet

Build Status codecov

Convert a CSV to a parquet file. You may also find sqlite-parquet-vtable useful.

Installing

If you just want to use the tool, install it and its dependencies via pip:

sudo pip install pyarrow csv2parquet

If you want to clone the repo and work on the tool, install its dependencies via pipenv:

pipenv install

Usage

Next, create some Parquet files. The tool supports CSV and TSV files.

csv2parquet file.csv [--rows NNN] [--row-group-size NNN] [--output output.parquet] [--codec CODEC]

where CODEC is one of snappy, gzip, brotli or none

Testing

pylint csv2parquet
pytest

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

csv2parquet-0.0.3.tar.gz (3.3 kB view hashes)

Uploaded Source

Built Distribution

csv2parquet-0.0.3-py3-none-any.whl (4.7 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page