A set of helper functions for CSV to Salesforce procedures, with reporting in AWS S3, based in a Django project
Project description
Overview
A set of helper functions for CSV to Salesforce procedures, with reporting in AWS S3, based in a Django project. The use case is extremely specific, but the helpers should be modular so they can be cherry-picked.
Typical use case:
- Receive an S3 event
- Download the S3 object
- Serialize the file into JSON
- Bulk upsert the JSON data to Salesforce
- Parse the results of the upsert for errors
- Construct a CSV error report
- Move the triggering S3 object to an archive folder
- Push the error report to an error folder in the same bucket
- Push an object to Salesforce that details information about the above execution
High-level Example
Using the Orchestrator
class, you can skip manually setting up a lot of the above
steps. This class is intended to be subclassed, and should provide plenty of options
for overriding methods to better suit your use-case.
Inheriting the Orchestrator
# orchestrator.py
from django.conf import settings
from django_s3_csv_2_sfdc.classes import Orchestrator as BaseOrchestrator
class Orchestrator(BaseOrchestrator):
def __init__(self, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
# This must be defined in the child class because your Salesforce object could be named anything
self.execution_object_name = "Integration_Execution__c"
@property
def execution_sfdc_hash(self):
# And it could have any number of fields
return {
"Number_of_Errors__c": self.error_count,
"Error_Report__c": self.error_report_link,
"Data_File__c": self.s3_object_key,
}
@property
def error_report_link(self):
return f"https://{self.bucket_name}.{settings.AWS_REGION}.amazonaws.com/{self.error_file_s3_key}"
Using the Orchestrator
# biz_logic.py
from django.conf import settings
from django_s3_csv_2_sfdc.classes import SfClient
# import the custom Orchestrator defined above
from .orchestrator import Orchestrator
salesforce = SfClient()
orchestrator = Orchestrator("some/s3/key/file.csv", settings.S3_BUCKET, sf_client=salesforce)
upsert_key = "My_External_ID__c"
accounts_data = [{"Name": "A name", upsert_key: "123"}]
results = salesforce.bulk.Account.upsert(results, upsert_key)
# You'll call log_batch for each batch you upload. This method
# will parse the results in search of errors
orchestrator.log_batch(results, accounts_data, "Account", upsert_key)
# This will create the error report, archive the source s3 file, and push
# the integration object to Salesforce. You'll definitely want to customize
# this by overriding this method or the methods it invokes
orchestrator.automagically_finish_up()
Low-level Example
from django_s3_csv_2_sfdc.csv_helpers import create_error_report
from django_s3_csv_2_sfdc.s3_helpers import download_file, respond_to_s3_event, upload_file
from django_s3_csv_2_sfdc.sfdc_helpers import extract_errors_from_results
# handler for listening to s3 events
def handler(event, context):
respond_to_s3_event(event, download_and_process)
def download_and_process(s3_object_key, bucket_name):
download_path = download_file(s3_object_key, bucket_name)
# This function contains your own biz logic; does not come from this library
results = serialize_and_push_to_sfdc(download_path)
sucesses, errors = parse_bulk_upsert_results(results)
report_path, errors_count = create_error_report([errors])
upload_file(report_path, bucket_name)
Just take what'cha need!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for django-s3-csv-2-sfdc-0.4.5.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | dd77e7bcaea58ed26209ec1e5c3ced72920023e8492784b754630f494a499ed7 |
|
MD5 | 73e04dab764943060d69329b6d4e4c95 |
|
BLAKE2b-256 | f3c630cd5da43e271c0ce47b2d8b831b469140b729b352470af9133d8923133e |
Close
Hashes for django_s3_csv_2_sfdc-0.4.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cd6587e152d5697000ea4b88146e62a5d24af558cd5b03a14e659614e51f4d12 |
|
MD5 | c721bd47b95058875d6f5860a16021af |
|
BLAKE2b-256 | 40fbe338c8dbea0b06ce89b297ce9aee88b67e7a6e20d231e724a2542bf042c4 |