Skip to main content

Tools for indexing gzip files to support random-like access.

Installing

To install library, run the following coomand:

$ python setup.py install

Testing

$ make test
$ make lint

Indexing a file

Indexing a file from stdin:

$ cat profiles.json.gz | gzipi index --format json --field id > index.json.gz

Indexing a local file:

$ cat profiles.json.bz -i profiles.json.gz -o index.json.gz --format csv --column 0 --delimiter ','

Retrieving data

Retrieving all data for specific domains:

$ zcat domains.json.gz | gzipi retrieve -f profiles.json.gz -i index.json.gz --format json > data.json --field domain

Using local and S3 paths:

$ gzipi retrieve -k domains.json.gz -f s3://domain-data/reports/Q1.json.gz -i index.json.gz --format json -o data.json --field domain

Repacking existing archives

$ gzipi repack -f profiles.json.gz -i new_index.json.gz -o new_profiles.json.gz --format json --field domain

Help

To get more information, run the following command:

$ gzipi --help

Release files for gzipi 0.1.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for gzipi 0.1.3
File Size Uploaded
gzipi-0.1.3.tar.gz 12.8 kB Details

Release files / gzipi-0.1.3.tar.gz

Download URL gzipi-0.1.3.tar.gz
Size 12.8 kB
Tags Source
SHA-256 checksum
How to use checksums
56baff05d9848de31ca0e956e3e5931146e34ce1ee595be937d246f5ad956d6d
BLAKE2b-256 checksum
How to use checksums
775ef8d509d8b98a9d1e212d67586582d9973f0a52b2a7dc6d6215f6183a36d4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.34.0 CPython/3.7.4

Release history Release notifications | RSS feed

This release

0.1.3 This release

1 release file

0.1.2

1 release file

0.1.1

1 release file

0.1.0

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page