clearskies-aws
clearskies bindings for working in AWS, which means additional:
- backends (DynamoDB, SQS)
- Secret/environment integrations (parameter store/secret manager)
- DB connectivity via IAM auth
- Contexts (ALB, HTTP API Gateway, Rest API Gateway, direct Lambda invocation, lambda+SQS)
Installation, Documentation, and Usage
To install:
pip3 install clear-skies-aws
Usage
Anytime you use a context from clearskies-aws, the default dependencies are adjust to:
- Provide
dynamo_db_backendas an allowed backend - Configure clearskies to use SSM Parameter Store as the default secrets "manager".
In both cases you must provide the AWS region for your resources, which you do by setting the AWS_REGION environment variable (either in an actual environment variable or in your .env file).
Parameter Store
To use the SSM parameter store you just inject the secrets variable into your callables:
import clearskies_aws
def parameter_store_demo(secrets):
return secrets.get("/path/to/parameter")
execute_demo_in_elb = clearskies_aws.contexts.lambda_alb(parameter_store_demo)
def lambda_handler(event, context):
return execute_demo_in_elb(event, context)
Also, per default behavior, clearskies can fetch things from your secret manager if specified in your environment/.env file. For instance, if your database password is stored in parameter store, then you can reference it from your .env file with a standard cursor backend:
db_host = "path-to-aws.rds"
db_username = "sql_username"
db_password = "secret://path/to/password/in/parameter/store"
db_database = "sql_database_name"
Secret Manager
If desired, you can swap out the parameter store integration for secret manager. Just remember that you can configure parameter store to fetch secrets from secret manager, so you might be best off doing that and sticking with the default parameter store integration. Still, if you want to use secret manager, you just configure it in your application or context:
import clearskies_aws
def secret_manager_demo(secrets):
return secrets.get('SecretARNFromSecretManager')
execute_demo_in_elb = clearskies_aws.contexts.lambda_elb(
parameter_store_demo,
bindings={'secrets': clearskies_aws.secrets.SecretManager},
)
def lambda_handler(event, context):
return execute_demo_in_elb(event, context)
Contexts
clearskies_aws adds the following contexts:
| name/import | Usage |
|---|---|
clearskies_aws.contexts.lambda_api_gateway |
Lambdas behind a Rest API Gateway |
clearskies_aws.contexts.lambda_elb |
Lambdas behind a simple ELB/ALB |
clearskies_aws.contexts.lambda_http_gateway |
Lambdas behind an HTTP Gateway |
clearskies_aws.contexts.lambda_inocation |
Lambdas invoked directly |
clearskies_aws.contexts.lambda_sqs_standard_partial_batch |
Lambdas attached to an SQS queue |
Lambdas+SQS
Here's a simple example of using the Lambda+SQS context:
import clearskies_aws
import clearskies
def process_record(request_data):
print(request_data)
process_sqs = clearskies_aws.contexts.lambda_sqs_standard_partial_batch(process_record)
def lambda_handler(event, context):
process_sqs(event, context)
See the AWS docs.
Note that, unlike other contexts, the Lambda+SQS context really only works with a simple callable. Routing and other handlers don't make much sense here. Keep in mind that, before invoking the Lambda, AWS may batch up records together in arbitrary ways. The context will take care of this and will invoke your callable once for each record in the AWS event - not once for the event. request_data will be populated with the actual message for the event. In addition, it assumes that a JSON message was sent to the queue, so request_data will be an object/list/etc, rather than a string. This is intended to be used with partial batching. Therefore, if your function raises an error, the context will catch it, return a failure response for the cooresponding message, and then continue processing any other messages in the batch.
SQS Backend
To use the SQS backend just declare it for your model, set the table name to return the queue url, and execute a "create" operation to send data to the queue. Note that the SQS backend is write-only: you can "create" records (resulting in a message being sent to the queue), but you can't read data back out. The way the queue system in SQS works is just too different than a standard database for that to make sense in the context of a clearskies model.
#!/usr/bin/env python3
import clearskies_aws
import clearskies
from collections import OrderedDict
class Product(clearskies.Model):
def __init__(self, sqs_backend, columns):
super().__init__(sqs_backend, columns)
@classmethod
def table_name(cls):
return 'https://sqs.us-east-1.amazonaws.com/123456789012/test-queue'
def columns_configuration(self):
return OrderedDict([
clearskies.column_types.string('name'),
clearskies.column_types.string('description'),
])
def send_to_queue(products):
for i in range(10):
products.create({
'name': 'test',
'description': str(i),
})
cli = clearskies_aws.contexts.cli(
send_to_queue,
binding_classes=[Product],
)
if __name__ == '__main__':
cli()
IAM DB Auth
For non-serverless RDS databases, AWS supports login via IAM. You have to provide a few additional details in your environment to make this work:
| name | value |
|---|---|
AWS_REGION |
The region your database is in (e.g. us-east-1) |
db_endpoint |
The endpoint from your database (available in RDS) |
db_username |
The username to use to connect |
db_database |
The name of the database |
ssl_ca_bundle_filename |
Path to the appropriate SSL bundle (see https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html) |
and then you have to enable it in your application/context configuration:
import clearskies_aws
def cursor_via_iamdb_auth(cursor):
print('I connected successfully!')
execute_demo_in_elb = clearskies_aws.contexts.lambda_elb(
cursor_via_iamdb_auth,
additional_configs=[clearskies_aws.di.IAMDBAuth]
)
def lambda_handler(event, context):
return execute_demo_in_elb(event, context)
Of course normally you wouldn't want to interact with it directly. Adding IAMDBAuth to your additional_configs and setting up the necessary environemnt variables will be sufficient that any models that use the cursor_backend will connect via IAM DB Auth, rather than using hard-coded passwords.
IAM DB Auth with SSM Bastion
Coming shortly
1 maintainer
Metadata
1 maintainer
Release files for clear-skies-aws 2.1.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| clear_skies_aws-2.1.3.tar.gz | 147.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| clear_skies_aws-2.1.3-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 255.5 kB
Release files / clear_skies_aws-2.1.3.tar.gz
| Download URL | clear_skies_aws-2.1.3.tar.gz |
|---|---|
| Size | 147.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0d6a4d00348789b71fbe9728ca46ea808af79e0cff5e71dfe87644af9c8d0bbe
|
|
BLAKE2b-256 checksum How to use checksums |
cb0197d3d7b2b77ee11ce6ab296e9931e9210e8ed037ed84117c484bcb9c4f55
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.0 {"installer":{"name":"uv","version":"0.13.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / clear_skies_aws-2.1.3-py3-none-any.whl
| Download URL | clear_skies_aws-2.1.3-py3-none-any.whl |
|---|---|
| Size | 108.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
360b0057e31582d659f18cccbce7275ccb68a8103cbd625ba65906d4255a0d0b
|
|
BLAKE2b-256 checksum How to use checksums |
44d616ef342a347f19f55c6bb704d222d7a336d33eccd1dd9331e19f77876dc7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.0 {"installer":{"name":"uv","version":"0.13.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
