claws 0.0.21

The Crossref Labs AWS tooling system.

Claws: Crossref Labs AWS Tools

Claws is a Python module that provides a simple and convenient way to interact with AWS services, such as S3 and CloudWatch. It utilizes the boto3 library and aiohttp for asynchronous operations, making it ideal for applications that require high-performance and efficient resource utilization.

This is a prototype Crossref Labs system. It is not guaranteed to be stable and the metadata schema and behaviour may be subject to change at any time.

Features

• S3 client and resource management with automatic deferred initialization
• Presigned URL generation for S3 objects
• Asynchronous downloading of multiple S3 objects
• Retrieval of S3 objects as strings
• Custom exception handling for S3 object retrieval errors
• Easy integration with monitoring and instrumentation tools
• Streaming upload to S3 to handle large JSON files

Requirements

• Python 3.6 or higher
• boto3
• aiohttp
• smart-open

Installation

To install the Claws module, simply use the following pip command:

pip install claws

Usage

Here is a basic example of how to use the Claws module to interact with AWS S3:

from claws.aws_utils import AWSConnector


# Initialize the AWSConnector with a specific S3 bucket
connector = AWSConnector(bucket="my-bucket", unsigned=False)

# Download an S3 object as a string
result = connector.s3_obj_to_str(bucket="my-bucket", s3_path="path/to/object.txt")

# Download multiple S3 objects asynchronously in parallel
s3_objects = ["path/to/object1.txt", "path/to/object2.txt", "path/to/object3.txt"]
results = connector.get_multiple_s3_objs(bucket="my-bucket", s3_objs=s3_objects)

for result in results:
    print(result)

Contributing

Contributions to the Claws module are welcome. Please feel free to submit pull requests or report issues on the GitHub repository.

License

Claws is released under the MIT License.

Credits

• .gitignore from Github.
• AWS Lambda Powertools by Amazon.

© Crossref 2023