A CLI which automatically uploads pip packages and directories to aws efs to be used in aws lambda
Project description
efsync
efsync is an CLI tool/ sdk to automatically upload files and dependencies to AWS EFS. The CLI is easy to use, you only need access to an AWS Account, an AWS EFS-filesystem up and running, and Docker installed. I wrote an article with an complete walkthrough. you can check this one out here or simply start with the Quick Start.
Documentation
Quick Start
- Install via pip3
pip3 install efsync
- sync your pip packages or files to AWS EFS
efsync -cf efsync.yaml
Examples
CLI Example with efsync.yaml
sync your pip packages or files to AWS EFS
efsync -cf efsync.yaml
CLI Example with parameter
sync your pip packages or files to AWS EFS
efsync -r requirements.txt -py 3.8 -epd lib -fd tmp -ap schueler -ar eu-central-1 -sbd <subnet_id> -ekn <ec2-key-name> -efi <efs_filesystem_id>
SDK Example with efsync.yaml
create an configuration file efsync.yaml
# lambda ci python version for pip installation
python_version: 3.8
# pip directory
efs_pip_dir: lib
# extra directory for file upload like ML models
file_dir: dir
# requirements file
requirements: requirements.txt
# Defines if the efs should be cleaned up before uploading
clean_efs: True
# aws profile configuration
aws_profile: efsync
aws_region: eu-central-1
#aws vpc and ec2 shit
efs_filesystem_id: fs-2adfas123
subnet_Id: subnet-xxx
ec2_key_name: efsync-asd913fjgq3
- sync your pip packages or files to AWS EFS
from efsync import efsync
efsync('efsync.yaml')
CLI
cli_short | cli_long | default | description |
---|---|---|---|
-h | --help | - | displays all commands |
-r | --requirements | requirements.txt | path of your requirements.txt |
-cf | --config_file | - | path of your efsync.yaml |
-py | --python_version | 3.8 | Python version used to install dependencies |
-epd | --efs_pip_dir | lib | directory where the pip packages will be installed on efs |
-efi | --efs_filesystem_id | - | File System ID from the EFS filesystem |
-ce | --clean_efs | True | Defines if the EFS should be cleaned up before uploading |
-fd | --file_dir | tmp | directory where all other files will be placed |
-ap | --aws_profile | efsync | name of the used AWS profile |
-ar | --aws_region | eu-central-1 | aws region where the efs is running |
-sbd | --subnet_Id | - | subnet id of the efs |
-ekn | --ec2_key_name | - | temporary key name for the ec2 instance |
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
efsync-0.2.4.tar.gz
(11.1 kB
view hashes)
Built Distribution
efsync-0.2.4-py3-none-any.whl
(13.7 kB
view hashes)