Tools For S3 Used By Saritasa
Project description
saritasa-s3-tools
Extension for boto3 to ease work with s3
Table of contents
Installation
pip install saritasa-s3-tools
or if you are using poetry
poetry add saritasa-s3-tools
Features
S3Client
andAsyncS3Client
for integrations with s3 buckets. This clients are extension to boto3 clients with proper typing, support for async and method to generate signed urls for file upload.S3FileTypeConfig
for defining configuration parameters for direct upload to s3. Check out moreS3Key
for generating unique keys for s3 upload, used forS3FileTypeConfig
S3FileField
andS3ImageFileField
- factory-boy fields for generating files and saving it ins3
pytest
plugin with fixtures forboto3
,S3Client
andAsyncS3Client
and etcDjango
plugin for setting up models and api
Django
saritasa-s3-tools
comes with django plugin which helps with models
config and api(Django Rest Framework
and drf_spectacular
).
You can try it out in example
folder.
Configuration
Setup model
First you need to add file/image fields to you model, like below
import saritasa_s3_tools.django
from django.db import models
class ModelWithFiles(models.Model):
"""Test model with different files configs."""
file = saritasa_s3_tools.django.S3FileField(
blank=True,
null=True,
# S3FileTypeConfig is needed so that we could understand where to save
# file, who can save file, what files can be save and what are size
# constraints
s3_config=saritasa_s3_tools.S3FileTypeConfig(
name="django-files",
key=saritasa_s3_tools.keys.S3KeyWithPrefix("django-files"),
allowed=("text/plain",),
auth=lambda user: bool(user and user.is_authenticated),
content_length_range=(1000, 20000000),
),
)
image = saritasa_s3_tools.django.S3ImageField(
blank=True,
null=True,
s3_config=saritasa_s3_tools.S3FileTypeConfig(
name="django-images",
key=saritasa_s3_tools.keys.S3KeyWithPrefix("django-images"),
allowed=("image/png",),
content_length_range=(5000, 20000000),
),
)
Then add S3FieldsConfigMixin
mixin to your serializer, like this
from rest_framework import serializers
import saritasa_s3_tools.django
from .. import models
class ModelWithFilesSerializer(
saritasa_s3_tools.django.S3FieldsConfigMixin,
serializers.ModelSerializer,
):
"""Serializer to show info model with files."""
class Meta:
model = models.ModelWithFiles
fields = "__all__"
Then just add S3GetParamsView
view to your project urls like that.
from django.urls import path
path(
"s3/",
include("saritasa_s3_tools.django.urls"),
name="saritasa-s3-tools",
),
Optional dependencies
[async]
- Add this to enable async support[factory]
- Add this to enable factory-boy fieldS3FileField
andS3ImageFileField
fromsaritasa_s3_tools.factory
[testing]
- Add this to enable testing helping functions fromsaritasa_s3_tools.testing.shortcuts
[django]
- Add this to enable django support[django-openapi]
- Add this to enable drf-spectacular support
To install all optional dependencies add [all]
Direct upload example
import saritasa_s3_tools
import pathlib
import xml.etree.ElementTree
s3_client = saritasa_s3_tools.S3Client(
boto3_client=boto3_client,
default_bucket=s3_bucket,
)
s3_params = s3_client.generate_params(
filename=pathlib.Path(__file__).name,
config=saritasa_s3_tools.S3FileTypeConfig.configs["files"],
content_type="application/x-python-code",
extra_metadata={
"test": "123",
},
)
with (
httpx.Client() as client,
pathlib.Path(__file__).open("rb") as upload_file,
):
upload_response = client.post(
url=s3_params.url,
data={
key: value
for key, value in s3_params.params.items()
if value is not None
},
files={"file": upload_file.read()},
)
parsed_response = xml.etree.ElementTree.fromstring( # noqa: S314
upload_response.content.decode(),
)
file_key = parsed_response[2].text
file_url = parsed_response[0].text
pytest plugin
saritasa-s3-tools
comes with pytest plugin which can setup boto3 instances
and create/clean up buckets for testing. Supports pytest-xdist
.
Fixtures
access_key_getter
,s3_endpoint_url_getter
,s3_region
- are used to configure boto3 session and clients/resources that are used in tests. You can override them or set values in ini file for pytest. Plugin will tell what you are missing.aws_session
- Returnsboto3.Session
aws_config
- Returnsbotocore.config.Config
, override if you need customization, None by default.boto3_resource
- Returns s3 resource or in typingmypy_boto3_s3.S3ServiceResource
boto3_client
- Returns s3 client or in typingmypy_boto3_s3.S3Client
s3_bucket_name
- Name of bucket for testing, default:saritasa-s3-tools
ors3_bucket_name
from ini file.s3_bucket_cleaner
- Returns function which cleans all files from buckets3_bucket_factory
- Returns manager which creates bucket, and when it's no longer needed deletes its3_bucket
- Creates bucket vias3_bucket_factory
and return it's names3_client
- Returnssaritasa_s3_tools.S3Client
async_s3_client
- Returnssaritasa_s3_tools.AsyncS3Client
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for saritasa_s3_tools-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f25f6c3dd30e380eeaf6f216882ffddd32f0240519012be995816c918f08e7b5 |
|
MD5 | 9e438fc4627ff2af7b11f55117ce89f5 |
|
BLAKE2b-256 | b67f923e14bafb112c5e5aeb5dcef956a9b7a711ccf0b74d6a32ecdf256ddddb |