Skip to main content

Opinionated machine learning organization and configuration

Project description

microcosm-sagemaker

Opinionated machine learning with SageMaker

Usage

For best practices, see cookiecutter-microcosm-sagemaker.

Profiling

Make sure pyinstrument is installed, either using pip install pyinstrument or by installing microcosm-sagemaker with profiling extra dependencies:

pip install -e '.[profiling]'

To enable profiling of the app, use the --profile flag with runserver:

runserver --profile

The service will log that it is in profiling mode and announce the directory to which it is exporting. Each call to the endpoint will be profiled and its results with be stored in a time-tagged html file in the profiling directory.

Experiment Tracking

To use Weights and Biases, install microcosm-sagemaker with wandb extra depdency:

pip install -e '.[wandb]'

To enable experiment tracking in an ML repository:

  • Choose the experiment tracking stores for your ML model. Currently, we only support wandb. To do so, add wandb to graph.use() in app_hooks/train/app.py and app_hooks/evaluate/app.py.

  • Add the API key for wandb to the environment variables injected by Circle CI into the docker instance, by visiting https://circleci.com/gh/globality-corp/<MODEL-NAME>/edit#env-vars and adding WANDB_API_KEY as an environment variable.

  • Microcosm-sagemaker automatically adds the config for the active bundle and its dependents to the wandb's run config.

  • To report a static metric:

class MyClassifier(Bundle):
    ...

    def fit(self, input_data):
        ...
        self.experiment_metrics.log_static(<metric_name>=<metric_value>)
  • To report a time-series metric:
class MyClassifier(Bundle):
    ...

    def fit(self, input_data):
        ...
        self.experiment_metrics.log_timeseries(
            <metric_name>=<metric_value>,
            step=<step_number>
        )

Note that the step keyword argument must be provided for logging time-series.

Artifact Tests

If you want to report your artifact tests to wandb, add the following line to the top of your conftest.py. For more information on using plugins in pytest, see here.

pytest_plugins = 'pytest_sagemaker'

It should be generated by globality-build, but in case it is not, also make sure to run artifact tests with --capture=tee-sys. This will allow to both capture and show stdout.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for microcosm-sagemaker, version 0.9.4
Filename, size File type Python version Upload date Hashes
Filename, size microcosm-sagemaker-0.9.4.tar.gz (26.1 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page