parquet-tools
This is a pip installable parquet-tools. In other words, parquet-tools is a CLI tools of Apache Arrow. You can show parquet file content/schema on local disk or on Amazon S3. It is incompatible with original parquet-tools.
Features
- Read Parquet data (local file or file on S3)
- Read Parquet metadata/schema (local file or file on S3)
Installation
$ pip install parquet-tools
Usage
$ parquet-tools --help
usage: parquet-tools [-h] {show,csv,inspect} ...
parquet CLI tools
positional arguments:
{show,csv,inspect}
show Show human readble format. see `show -h`
csv Cat csv style. see `csv -h`
inspect Inspect parquet file. see `inspect -h`
optional arguments:
-h, --help show this help message and exit
Usage Examples
Show local parquet file
$ parquet-tools show test.parquet
+-------+-------+---------+
| one | two | three |
|-------+-------+---------|
| -1 | foo | True |
| nan | bar | False |
| 2.5 | baz | True |
+-------+-------+---------+
Show parquet file on S3
$ parquet-tools show s3://bucket-name/prefix/*
+-------+-------+---------+
| one | two | three |
|-------+-------+---------|
| -1 | foo | True |
| nan | bar | False |
| 2.5 | baz | True |
+-------+-------+---------+
Inspect parquet file schema
$ parquet-tools inspect /path/to/parquet
Inspect output
############ file meta data ############
created_by: parquet-cpp version 1.5.1-SNAPSHOT
num_columns: 3
num_rows: 3
num_row_groups: 1
format_version: 1.0
serialized_size: 2226
############ Columns ############
one
two
three
############ Column(one) ############
name: one
path: one
max_definition_level: 1
max_repetition_level: 0
physical_type: DOUBLE
logical_type: None
converted_type (legacy): NONE
############ Column(two) ############
name: two
path: two
max_definition_level: 1
max_repetition_level: 0
physical_type: BYTE_ARRAY
logical_type: String
converted_type (legacy): UTF8
############ Column(three) ############
name: three
path: three
max_definition_level: 1
max_repetition_level: 0
physical_type: BOOLEAN
logical_type: None
converted_type (legacy): NONE
Cat CSV parquet and transform csvq
$ parquet-tools csv s3://bucket-name/test.parquet |csvq "select one, three where three"
+-------+-------+
| one | three |
+-------+-------+
| -1.0 | True |
| 2.5 | True |
+-------+-------+
Metadata
Release files for parquet-tools 0.2.16
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| parquet_tools-0.2.16.tar.gz | 28.2 kB | Without provenance | Details for parquet_tools-0.2.16.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| parquet_tools-0.2.16-py3-none-any.whl | Python 3 | none | any | Details for parquet_tools-0.2.16-py3-none-any.whl |
Total release size: 60.1 kB
Release files / parquet_tools-0.2.16.tar.gz
| Download URL | parquet_tools-0.2.16.tar.gz |
|---|---|
| Size | 28.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7bad2ad38f2381afde9d109577199d953946a904b0d74b2f074e47040673f60a
|
|
BLAKE2b-256 checksum How to use checksums |
7155ce056a1be7cb3cf90165240e81cce45848c37a2aa7453156e3229cb79d12
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.18
|
Release files / parquet_tools-0.2.16-py3-none-any.whl
| Download URL | parquet_tools-0.2.16-py3-none-any.whl |
|---|---|
| Size | 31.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a360eec233123c2c66f38867ec6dd5c79d48b8f757af8162bf8825d0190dd3d5
|
|
BLAKE2b-256 checksum How to use checksums |
60acbcfe4e1a19b382be1cc3106ef64611ef85e8683bb07fcc7eab376170dffe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.18
|
Security for parquet-tools 0.2.16
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.