No project description provided
Project description
Bokchoi simplifies running Python batch jobs on AWS spot instances. Bokchoi handles requesting spot instances, deploying your code and ensures the spot requests are cancelled when all jobs are finished.
Getting Started
Installing
To install bokchoi:
pip install bokchoi
Settings
Say you have a project folder with a single python script:
YourProjectFolder └─ deep_nn.py
In your project folder create a settings file named bokchoi_settings.json:
YourProjectFolder ├─ deep_nn.py └─ bokchoi_settings.json
This file should contain the following:
{
"<project_name>": {
"EntryPoint": "deep_nn.main",
"Region": "us-east-1",
"Platform": "EC2",
"Requirements": [
"numpy==1.13.0",
"boto3==1.4.4"
],
"EC2": {
"InstanceCount": 1,
"SpotPrice": "0.1",
"LaunchSpecification": {
"ImageId": "ami-123456",
"InstanceType": "c4.large",
"SubnetId": "subnet-123456"
}
}
}
}
Deploying
Deploying your job to AWS is now as simple as running:
bokchoi project_name deploy
bokchoi project_name run
Undeploying
To undeploy your job, removing all resources from your AWS environment:
bokchoi project_name undeploy
EMR
Bokchoi now also supports running python applications on Amazon EMR. To run your app on an EMR cluster use the following settings:
{
"<project_name>": {
"EntryPoint": "deep_nn.py",
"Region": "us-east-1",
"Platform": "EMR",
"Requirements": [
"numpy==1.13.0",
"boto3==1.4.4"
],
"EMR": {
"InstanceCount": 2,
"Version": "emr-5.8.0",
"SpotPrice": "0.10",
"LaunchSpecification": {
"InstanceType": "m1.medium",
"SubnetId": "subnet-123456",
"AdditionalSecurityGroups": ["sg-12ab34"]
}
}
}
}
Acknowledgements
Shamelessly inspired by Zappa (https://github.com/Miserlou/Zappa)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.