Skip to main content

AWS scripts for geoadmin

Project description

tool-aws

Build Status

Macro CMDs for managing AWS resources

Installation

$ pip install tool_aws

Usage

Batch delete files in S3 by listing content iteratively with a prefix.

$ s3rm --bucket-name ${BUCKET_NAME} --prefix ${PATH}

Batch delete tiles in S3 using a bbox in LV95:

$ s3rm --bucket-name ${BUCKET_NAME} --prefix /1.0.0/ch.swisstopo.fixpunkte-agnes/default/current/2056/* --bbox 2671000,1139000,2712250,1158500 --image-format png

You can always use the help function:

$ s3rm --help
usage: s3rm [-h] -b BUCKETNAME -p PREFIX [--profile PROFILENAME] [--bbox BBOX]
            [-n NBTHREADS] [-s CHUNKSIZE] [-i IMAGEFORMAT] [-lr LOWRES]
            [-hr HIGHRES] [-f]

Purpose:
    This script is intended for efficient and MASSIVE RECURSIVE
    DELETION of S3 'folders'. This mean that any resource matching with
    the PREFIX variable, will be DELETED IRREVERSIVELY.

    Use this script CAREFULLY.

optional arguments:
  -h, --help            show this help message and exit

Mandatory arguments:
  -b BUCKETNAME, --bucket-name BUCKETNAME
                        bucket name
  -p PREFIX, --prefix PREFIX
                        Prefix (string) relative to the bucket base path.

Program options:
  --profile PROFILENAME
                        AWS profile
  --bbox BBOX           a bounding box in lv95
  -n NBTHREADS, --threads-number NBTHREADS
                        Number of threads (subprocess), default: machine
                        number of CPUs
  -s CHUNKSIZE, --chunk-size CHUNKSIZE
                        Chunk size for S3 batch deletion, default is set to
                        1000 (maximal value for S3)
  -i IMAGEFORMAT, --image-format IMAGEFORMAT
                        The image format
  -lr LOWRES, --lowest-resolution LOWRES
                        The lowest resolution in meters
  -hr HIGHRES, --highest-resolution HIGHRES
                        The highest resolution in meters
  -f, --force           force the removal, i.e. no prompt for confirmation.

Disclaimer:
    This software is provided "as is" and
    is not granted to work in particular cases or without bugs.
    The author disclaims any responsability in case of data loss,
    computer damage or any other bad issue that could arise
    using this software.

Setup in dev mode

$ python3 -m env .venv $ source .venv/bin/activate $ pip install -e . $ pip install -r dev-requirements.txt

To launch the tests:

$ nosetests tests/

Style

Control styling:

$ flake8 tool_aws/ tests/

Autofix mistakes:

$ find tool_aws/* tests/* -type f -name '*.py' -print | xargs autopep8 --in-place --aggressive --aggressive --verbose

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tool_aws-0.2.6.tar.gz (10.7 kB view details)

Uploaded Source

Built Distribution

tool_aws-0.2.6-py3-none-any.whl (10.2 kB view details)

Uploaded Python 3

File details

Details for the file tool_aws-0.2.6.tar.gz.

File metadata

  • Download URL: tool_aws-0.2.6.tar.gz
  • Upload date:
  • Size: 10.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.8.10

File hashes

Hashes for tool_aws-0.2.6.tar.gz
Algorithm Hash digest
SHA256 0619360fedbbd91a60a824c27e73c3b5b88e2dbd6fe38c97ac98c9215efb8ec2
MD5 40d20177e9c6f6e5d279c72df48bd734
BLAKE2b-256 e8229bb00e912128730d1fe5c085b27aeed9bf7e684f8ad8b139d2a219ec8d4f

See more details on using hashes here.

File details

Details for the file tool_aws-0.2.6-py3-none-any.whl.

File metadata

  • Download URL: tool_aws-0.2.6-py3-none-any.whl
  • Upload date:
  • Size: 10.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.8.10

File hashes

Hashes for tool_aws-0.2.6-py3-none-any.whl
Algorithm Hash digest
SHA256 54378265d89b71ff01694b14fbe4cf3c4f2d581979e3c3e79d5d07850bc3fb80
MD5 be1f1e5cfebf9dabc4361e4527b4fcde
BLAKE2b-256 3b2f7c128766a1c6b0562b631d4c90ce68981f2dd68f579845465fc3e7326b8d

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page