S3 Concat is used to concatenate many small files in an s3 bucket into fewer larger files.
Install
Comming soon: :sub:`pip install s3-concat`
Usage
This installs a cli called s3-concat. Find out how to use it by
running s3-concat -h.
It can also be imported and used in any other python scripts by foing
import s3_concat.
More docs to follow on that.
Limitations
This uses the multipart upload of s3 and its limits are
https://docs.aws.amazon.com/AmazonS3/latest/dev/qfacts.html
TODO: give details on how it deals with files smaller then 5MB because
it is handled differently.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
s3-concat-0.0.3.tar.gz
(4.1 kB
view details)
File details
Details for the file s3-concat-0.0.3.tar.gz.
File metadata
- Download URL: s3-concat-0.0.3.tar.gz
- Upload date:
- Size: 4.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/1.9.1 pkginfo/1.4.1 requests/2.18.4 setuptools/38.4.0 requests-toolbelt/0.8.0 tqdm/4.19.5 CPython/3.6.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
397cdd4ddccd27f676900516202efadf35e8485352228e9784227ea74a4dcdc8
|
|
| MD5 |
493c520f3e32df541541e6de78f4d5aa
|
|
| BLAKE2b-256 |
8c8571d5dbfd798839a8ca3a06401036a39b665bb0005e2f410872406633d850
|
