Streamlit component that allows you to upload files to Google Cloud Storage via signed url
Project description
Streamlit Signed URL Uploader
Overview
This Streamlit component provides a solution to overcome the 32MB request size limit in App Engine and Cloud Run by leveraging cloud storage signed URLs. It allows users to upload large files directly to Google Cloud Storage (GCS) and seamlessly integrate them into web applications deployed on App Engine or Cloud Run.
Considering the schema below, the Signed Url Uploader abstracts away steps 2 and 3. Check this article for more details.
Installation
You can install the latest release using:
pip install st_signedUrl_uploader
Usage
This component serves as an efficient uploader for applications that are deployed on App Engine or Cloud Run, specifically designed to handle large file uploads by utilizing Google Cloud Storage (GCS) and signed URLs. To facilitate ease of integration and testing, the component can be used in two ways: directly in cloud deployments or locally using a service account for development and testing purposes. Below, you will find detailed instructions for both scenarios, including sample code and a list of infrastructure prerequisites necessary to ensure smooth operation and integration of the uploader in your projects.
Usage with App Engine or Cloud Run
Requirements :
- Google Cloud Storage Bucket: Set up a bucket to store uploaded files.
- CORS Configuration: Set your bucket's CORS settings using:
gcloud storage buckets update gs://BUCKET_NAME --cors-file=CORS_CONFIG_FILE
Example CORS configuration file can be found in CORS.json or create your own based on Google Cloud Documentation
Implementation :
- Service Account Permissions: Ensure the App Engine or Cloud Run service account has the
roles/iam.serviceAccountTokenCreator
role for generating signed URLs.
Sample Code :
from st_signedUrl_uploader import signedUrl_uploader
from google import auth
from google.cloud import storage
def main():
bucket_name = "your-bucket-name" # Specify your GCS bucket name here
# Authenticate and create a storage client
credentials, project = auth.default()
credentials.refresh(auth.transport.requests.Request())
storage_client = storage.Client()
# Use the uploader function to handle file uploads
signedUrl_uploader(storage_client, credentials, bucket_name)
if __name__ == "__main__":
main()
After deploying your application on App Engine or Cloud run, the default service account credentials are retrieved and used for signed url generation.
Testing locally with a service account
Requirements :
- Google Cloud Storage Bucket: Set up a bucket to store uploaded files.
- CORS Configuration: Set your bucket's CORS settings using:
gcloud storage buckets update gs://BUCKET_NAME --cors-file=CORS_CONFIG_FILE
Example CORS configuration file can be found in CORS.json or create your own based on Google Cloud Documentation
Implementation :
- Create a service account and give it the role
roles/iam.serviceAccountTokenCreator
Sample Code :
from st_signedUrl_uploader import signedUrl_uploader
from google import auth
from google.cloud import storage
import os
def main():
bucket_name = "your-bucket-name" # Specify your GCS bucket name here
# Authenticate and create a storage client
parent_dir = os.path.dirname(os.path.abspath(__file__))
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = os.path.join(parent_dir, 'path/to/key_file.json')
credentials, project = auth.default()
storage_client = storage.Client()
# Use the uploader function to handle file uploads
signedUrl_uploader(storage_client, credentials, bucket_name)
if __name__ == "__main__":
main()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for st_signedurl_uploader-1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | aa06d2fa0c49e5c684a7755a3e82b724fd6b9ecaade940d4cde30d0ab2c95454 |
|
MD5 | 2c8bc8bc2ca1c445d731eeaed1f5e14c |
|
BLAKE2b-256 | 49dba4afb1d91d7b1759a0e8f892e5b18481cbdf804852a37f26273892c4060d |
Hashes for st_signedUrl_uploader-1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2bad197ec7dd53f5e49037816876129ad210166422fcc5e61bee5c889fe858a6 |
|
MD5 | c4e481556fe6b6fc6ee6d45deed45244 |
|
BLAKE2b-256 | 89473d1277a7691d118ed39f9341fd9ce641e532d16d49f65c845f90657938ac |