Skip to main content

Lightweight library to presign URLs compatible with boto3

Project description

SignURLarity

Fast, lightweight S3 client focused on presigned URL generation. SignURLarity provides a boto3-compatible API with significantly better performance by avoiding the boto3 dependency overhead.

Features

  • Fast presigned URL generation (faster than boto3)
  • Async support with AsyncClient
  • Connection pooling for better performance
  • S3-compatible services (AWS S3, MinIO, etc.)

Performance

When creating a Pull Request, performance tests with respects to boto are ran and added as comment. Obviously, due to the nature of github Actions, the numbers may vary (a lot). Just as an example, here are the numbers of one of the later PR (the bigger the better). See below to run it yourself

Test 3.11 3.12 3.13 3.14
create_bucket_aio 0.888 0.953 0.979 1.071
create_bucket_aio_cm 0.939 0.935 0.946 0.976
create_bucket_sync 1.145 1.194 1.142 1.198
create_bucket_sync_cm 1.141 1.144 1.171 1.173
generate_presigned_post_aio 8.516 8.722 8.606 8.804
generate_presigned_post_aio_cm 8.504 8.742 8.465 8.890
generate_presigned_post_sync 6.845 6.468 6.875 6.995
generate_presigned_post_sync_cm 6.713 6.465 6.638 6.768
generate_presigned_url_aio 15.597 15.490 15.075 17.026
generate_presigned_url_aio_cm 15.599 15.137 14.784 17.059
generate_presigned_url_sync 15.361 14.835 13.976 16.008
generate_presigned_url_sync_cm 14.739 14.806 14.014 16.203
head_bucket_aio 0.996 1.080 1.001 1.009
head_bucket_aio_cm 0.996 1.265 0.997 1.071
head_bucket_sync 1.468 1.522 1.456 1.522
head_bucket_sync_cm 1.211 1.147 1.505 1.507
head_object_aio 0.877 0.916 0.834 0.913
head_object_aio_cm 0.966 0.895 0.885 0.900
head_object_sync 1.516 1.443 0.898 1.584
head_object_sync_cm 1.493 1.547 1.511 1.470

Installation

pip install signurlarity

Quick Start

Synchronous Client

from signurlarity import Client

with Client(
    endpoint_url="https://s3.us-west-2.amazonaws.com",
    aws_access_key_id="your-access-key",
    aws_secret_access_key="your-secret-key",
) as client:
    url = client.generate_presigned_url(
        "get_object",
        Params={"Bucket": "mybucket", "Key": "myfile.txt"},
        ExpiresIn=3600,
    )

Async Client

from signurlarity.aio import AsyncClient

async with AsyncClient(
    endpoint_url="https://s3.us-west-2.amazonaws.com",
    aws_access_key_id="your-access-key",
    aws_secret_access_key="your-secret-key",
) as client:
    url = await client.generate_presigned_url(
        "get_object",
        Params={"Bucket": "mybucket", "Key": "myfile.txt"},
        ExpiresIn=3600,
    )

Documentation

For detailed documentation including:

  • Complete API reference
  • Advanced usage examples
  • Error handling
  • Additional methods (presigned POST, head operations, etc.)

Please refer to the docstrings in the source code:

Development

Run tests

installation pixi

This will run functionnal tests. It will spawn docker container to test against rustfs, minio and moto

pixi run unit-test # add any pytest option you want

Any pytest argument can be added

pre-commit

SignURLarity uses pre-commit to format code and check for issues. The easiest way to use pre-commit is to run the following after cloning:

pixi run pre-commit install

This will result in pre-commit being ran automatically each time you run git commit. If you want to explicitly run pre-commit you can use:

pixi run pre-commit # (1)!
pixi run pre-commit --all-files # (2)!
  1. Runs pre-commit only for files which are uncommitted or which have been changed.
  2. Runs pre-commit for all files even if you haven't changed them.

Benchmark

For a full performance comparison, run

pixi run full-benchmark /whatever/outputdir

This will compare the results of boto and signurlarity against rustfs for python version 3.11, 3.12, 3.13 and 3.14, and generate json files in the output directoty

If you want to run it for a specific version only:

pixi run -e py314 benchmark --test-results-dir=/whatever/you/want

you can then display it with

pixi run -e py314 display-benchmark-comparison --test-results-dir=/whatever/you/want

Profiling tests

A few profiling tests are available

pixi run -e py314 profile-test -s --test-results-dir=/whatever/you/want

This will generate prof files

You can convert it in svg and open it in your web browser like so

pixi shell -e py314
flameprof --format=log profile_generate_presigned_post/presigned_post.prof | flamegraph > profile_generate_presigned_post/presigned_post.svg

:

Example results

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

signurlarity-0.2.2.tar.gz (62.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

signurlarity-0.2.2-py3-none-any.whl (37.7 kB view details)

Uploaded Python 3

File details

Details for the file signurlarity-0.2.2.tar.gz.

File metadata

  • Download URL: signurlarity-0.2.2.tar.gz
  • Upload date:
  • Size: 62.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for signurlarity-0.2.2.tar.gz
Algorithm Hash digest
SHA256 6212860f849edcf565ff002a55114779d2df20847a2828548ab3c8b8beb0e9d2
MD5 56b04883e522ab2ecb377f21f89b3bec
BLAKE2b-256 db2acd08fbff130b2871f75bce96ac0db90cf1ac3df0bddba481340465c22f0f

See more details on using hashes here.

Provenance

The following attestation bundles were made for signurlarity-0.2.2.tar.gz:

Publisher: deployment.yml on DIRACGrid/signurlarity

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file signurlarity-0.2.2-py3-none-any.whl.

File metadata

  • Download URL: signurlarity-0.2.2-py3-none-any.whl
  • Upload date:
  • Size: 37.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for signurlarity-0.2.2-py3-none-any.whl
Algorithm Hash digest
SHA256 0c8089ecac04ce105e525b60749d134b171410beefc962e167e0e64141d6e7a7
MD5 c999153a53a0f574316d4f5fe6faf069
BLAKE2b-256 592568f81c50aeb3f30e01c20da0f56d14c1a779b57d9308c50e19c63dc8413f

See more details on using hashes here.

Provenance

The following attestation bundles were made for signurlarity-0.2.2-py3-none-any.whl:

Publisher: deployment.yml on DIRACGrid/signurlarity

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page