The SageMaker TensorFlow Training Toolkit is an open source library for making the TensorFlow framework run on Amazon SageMaker.
This repository also contains Dockerfiles which install this library, TensorFlow, and dependencies for building SageMaker TensorFlow images.
For information on running TensorFlow jobs on SageMaker:
Table of Contents
Getting Started
Prerequisites
Make sure you have installed all of the following prerequisites on your development machine:
For Testing on GPU
Recommended
A Python environment management tool. (e.g. PyEnv, VirtualEnv)
Building your Image
Amazon SageMaker utilizes Docker containers to run all training jobs & inference endpoints.
The Docker images are built from the Dockerfiles specified in docker/.
The Dockerfiles are grouped based on TensorFlow version and separated based on Python version and processor type.
The Dockerfiles for TensorFlow 2.0+ are available in the tf-2 branch.
To build the images, first copy the files under docker/build_artifacts/ to the folder container the Dockerfile you wish to build.
# Example for building a TF 2.1 image with Python 3 cp docker/build_artifacts/* docker/2.1.0/py3/.
After that, go to the directory containing the Dockerfile you wish to build, and run docker build to build the image.
# Example for building a TF 2.1 image for CPU with Python 3 cd docker/2.1.0/py3 docker build -t tensorflow-training:2.1.0-cpu-py3 -f Dockerfile.cpu .
Don’t forget the period at the end of the docker build command!
Running the tests
Running the tests requires installation of the SageMaker TensorFlow Training Toolkit code and its test dependencies.
git clone https://github.com/aws/sagemaker-tensorflow-container.git cd sagemaker-tensorflow-container pip install -e .[test]
Tests are defined in test/ and include unit, integration and functional tests.
Unit Tests
If you want to run unit tests, then use:
# All test instructions should be run from the top level directory pytest test/unit
Integration Tests
Running integration tests require Docker and AWS credentials, as the integration tests make calls to a couple AWS services. The integration and functional tests require configurations specified within their respective conftest.py.Make sure to update the account-id and region at a minimum.
Integration tests on GPU require Nvidia-Docker.
Before running integration tests:
Build your Docker image.
Pass in the correct pytest arguments to run tests against your Docker image.
If you want to run local integration tests, then use:
# Required arguments for integration tests are found in test/integ/conftest.py
pytest test/integration --docker-base-name <your_docker_image> \
--tag <your_docker_image_tag> \
--framework-version <tensorflow_version> \
--processor <cpu_or_gpu>
# Example
pytest test/integration --docker-base-name preprod-tensorflow \
--tag 1.0 \
--framework-version 1.4.1 \
--processor cpu
Functional Tests
Functional tests are removed from the current branch, please see them in older branch r1.0.
Contributing
Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.
License
SageMaker TensorFlow Containers is licensed under the Apache 2.0 License. It is copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. The license is available at: http://aws.amazon.com/apache2.0/
1 maintainer
Release files for sagemaker-tensorflow-training 20.0.1.post2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| sagemaker_tensorflow_training-20.0.1.post2.tar.gz | 12.0 kB | Details |
Release files / sagemaker_tensorflow_training-20.0.1.post2.tar.gz
| Download URL | sagemaker_tensorflow_training-20.0.1.post2.tar.gz |
|---|---|
| Size | 12.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d154bba3ae0c1725cabacc49ef06c27812959e6432622d9f9905202b99c6d30a
|
|
BLAKE2b-256 checksum How to use checksums |
611a83fa82e79d0c924526a430505f64e72f15e1ee5b65c59f7c00020d988b95
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.24.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.46.1 CPython/3.6.10
|
