Skip to main content
Help the Python Software Foundation raise $60,000 USD by December 31st!  Building the PSF Q4 Fundraiser

A library for helping to deploy a compute node

Project description


CircleCI PyPI

This is a CLI for setting up a Numerai compute node and deploying your models to it. This sets up a compute cluster in AWS (Amazon Web Services), and is architected to cost a minimal amount of money to run (on average, you will spend less than $1 per month).

Questions or feedback? Join us at RocketChat

Architecture Diagram


All you need is:

  1. AWS (Amazon Web Services) account setup with an API key
  2. A Numerai API key
  3. Docker setup on your machine
  4. Python3 (your model code doesn't have to use Python3, but this CLI tool needs it)

See the Prerequisites Help section if you need help getting these setup.

This project has been tested and found working on OSX, Windows 10, and Ubuntu 18.04, but should theoretically work anywhere that docker and Python are available.


Install this library with:

pip3 install numerai-cli


The following instructions will get you setup with a compute node in a matter of minutes:

mkdir example-numerai
cd example-numerai

numerai setup
numerai docker copy-example
numerai docker deploy

numerai setup will prompt your for AWS and Numerai API keys. Please refer to the AWS and Numerai API Key sections for instructions on how to obtain those.

Your compute node is now setup and ready to run. Look in the .numerai/submission_url.txt file to see your submission url that you will provide to Numerai as your webhook url. Go to your Numerai account and select the "Compute" section to enter it there.

Account Settings

The default example does not stake, so you will still have to manually do that every week. Alternatively, check out the bottom of for example code on how to stake automatically.


You can test the webhook url directly like so:

numerai compute test-webhook

If the command succeeds, it will return quickly with a status of "pending". This means that your container has been scheduled to run but hasn't actually started yet.

You can check that your job has been scheduled by running:

numerai compute status

And once it's in the RUNNING state, you can look at the logs:

numerai compute logs

You can also check for the raw AWS logs at or logs from your container at

NOTE: the container takes a little time to schedule. The first time it runs also tends to take longer (2-3min), with subsequent runs starting a lot faster.

Local Testing

You can test your container locally by running:

numerai docker run

This will run the container exactly like it runs in the cloud. Keep in mind that this will be limited by your machine's RAM, or in the case of Windows/OSX, you can increase how much memory is allocated to docker: Windows or OSX

Common Problems

numerai not in PATH

numerai: command not found

Try and run ~/.local/bin/numerai on osx/linux or %LOCALAPPDATA%\Programs\Python\Python37-32\Scripts\numerai.exe on Windows.

Alternatively, exit your terminal/command prompt and re-open it. By default, pip will try to add itself to your PATH for subsequent runs, but it requires you to restart the terminal.

Docker not installed

subprocess.CalledProcessError: Command 'docker run --rm -it -v /home/jason/tmp/.numerai:/opt/plan -w /opt/plan hashicorp/terraform:light init' returned non-zero exit status 127.

If you're certain that docker is installed, make sure that your user can execute docker, ie. try to run docker ps. If that's the issue, then depending on your system, you can do the following:


Make sure the docker application is running and finished booting up. It can take a few minutes to be completely ready. When clicking on the docker tray icon, it should say "Docker Desktop is Running".

If you're using Docker Toolbox on Windows, then make sure you've opened the "Docker Quickstart Terminal".

sudo usermod -aG docker $USER

Then reboot or logout/login for this to take effect.

Wrong AWS API key

Error: error validating provider credentials: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.
        status code: 403, request id: 32500359-7d9e-11e9-b0ed-596aba1b72c5
subprocess.CalledProcessError: Command 'docker run -e "AWS_ACCESS_KEY_ID=..." -e "AWS_SECRET_ACCESS_KEY=..." --rm -it -v /home/jason/tmp/.numerai:/opt/plan -w /opt/plan hashicorp/terraform:light apply -auto-approve' returned non-zero exit status 1.

Drive not shared

If you get:

docker: Error response from daemon: Drive has not been shared

Then you need to share your drive. See for details.

Compute Node Architecture

numerai setup under the hood uses Terraform to setup an AWS environment with the following:

  • An ECR (Elastic Container Repository) for storing docker containers
  • A Fargate task to run your compute job in the ECS (Elastic Container Service)
  • A lambda endpoint that schedules your compute job to run

There's actually a bunch of other bits of glue in AWS that are setup to run this, but these 3 are the most important. The lambda endpoint corresponds to the submission url that your provide back to Numerai. The ECR is where numerai docker deploy will push your image to. Fargate is where your task actually runs in the ECS, and it's where you'll want to look if things don't appear to be actually submitting.

Submission Webhook URL

This is the url that you provide back to Numerai. It's the thing that triggers the lambda and schedules your job to run. Once you've setup the webhook in your Numerai account, it will be called Saturday morning right after a new round opens, and if your job fails (determined by not having submitted all submissions successfully) then it will be triggered again around 24 hours later.

Docker example

Lets look at the docker example's files:

▶ tree
├── Dockerfile
├── requirements.txt


And then lets look at the Dockerfile:

FROM python:3

ADD requirements.txt .
RUN pip install -r requirements.txt

ADD . .



CMD [ "python", "./" ]

So breaking this down line by line:

FROM python:3

This Dockerfile inherits from python:3, which provides us a working Python environment.

ADD requirements.txt .
RUN pip install -r requirements.txt

We then add the requirements.txt file, and pip install every requirement from it. The ADD keyword will take a file from your current directory and copy it over to the Docker container.

ADD . .

After that, we add everything in the current directory. This will include all of your code, as well as any other files such as serialized models that you've saved to the current directory.



These are docker arguments that numerai train/run/deploy will always pass into docker. They are then set in your environment, so that you can access them from your script like so:

import os
public_id = os.environ["NUMERAI_PUBLIC_ID"]
secret_key = os.environ["NUMERAI_SECRET_KEY"]
CMD [ "python", "./" ]

This sets the default command to run your docker container. This is overridden in the numerai docker train command, but otherwise this will be the command that is always run when using numerai docker run and numerai docker deploy

The model code lives in here. We're using numerox, but realistically this file could host any kind of model.

The code that gets run when running numerai docker train

The code that gets run when running numerai docker run and is deployed to run in the numerai compute node after executing numerai docker deploy


numerai setup

The following command will setup a full Numerai compute cluster in AWS:

numerai setup

If this is your first time running, it will also ask for your AWS and Numerai API keys. These keys are stored in $HOME/.numerai for future runs.

There will be a bunch of output about how it's setting up the AWS cluster, but the only important part is at the end:


docker_repo =
submission_url =
  • submission_url is your webhook url that you will provide to Numerai. Save this for later. If you forget it, a copy is stored in .numerai/submission_url.txt.
  • docker_repo will be used in the next step but you don't need to worry about it since it's all automated for you

This command is idempotent and safe to run multiple times.

numerai docker copy-example

If you don't have a model already setup, then you should copy over the docker example.

numerai docker copy-example

WARNING: this will overwrite the following files if they exist: Dockerfile,,,, and requirements.txt

If you manage multiple tournament models, you should copy the python3-multimodel example

numerai docker copy-example --python3-multimodel

As well as using R:

numerai docker copy-example --rlang

numerai docker train (optional, but highly recommended)

Trains your model by running This assumes a file called exists and serializes your model to this directory. See the example if you want inspiration for how to do this.

numerai docker train

numerai docker run (optional)

To test your docker container locally, you can run it with this command. This will run the default CMD for the Dockerfile, which for the default example is

numerai docker run

numerai docker deploy

Builds and pushes your docker image to the AWS docker repo

numerai docker deploy

numerai destroy

If you ever want to delete the AWS environment to save costs or start from scratch, you can run the following:

numerai destroy

This will delete everything, including the lambda url, the docker container and associated task, as well as all the logs

This command is idempotent and safe to run multiple times.


Container uses up too much memory and gets killed

By default, Numerai compute nodes are limited to 8GB of RAM. If you need more, you can open up .numerai/ and update the fargate_cpu and fargate_memory settings to the following:

variable "fargate_cpu" {
  description = "Fargate instance CPU units to provision (1 vCPU = 1024 CPU units)"
  default     = "4096"

variable "fargate_memory" {
  description = "Fargate instance memory to provision (in MiB)"
  default     = "30720"

30720MB=30GB and is the maximum that Amazon can support.

After you've done this, re-run numerai setup.

Note: this will raise the costs of running your compute node, see for estimated costs. You only pay for the time it's running, rounded to the nearest minute.

Billing Alerts

Unfortunately, there's no automated way to setup billing alerts. If you wish to be alerted when costs pass some threshold, you should follow the instructions at to setup one.

We estimate costs to be less than $5 per month unless your compute takes more than 12 hours a week. Also keep in mind that increasing the RAM/CPU as described in the previous section will increase your costs.

Prerequisites Help


You need to signup for AWS and create an administrative IAM user

  1. Sign up for an AWS account:
  2. Create an IAM user with Administrative access:$new
    1. Give user a name and select "Programmatic access"
    2. For permissions, click "Attach existing policies directly" and click the check box next to "AdministratorAccess"
    3. Save the "Access key ID" and "Secret access key" from the last step. You will need them later

Numerai API Key

  1. You will need to create an API key by going to and clicking "Add" under the "Your API keys" section.
  2. Select the following permissions for the key: "Upload submissions", "Make stakes", "View historical submission info", "View user info"
  3. Your secret key will pop up in the bottom left of the page. Copy this somewhere safe.
  4. You public ID will be listed when you click "View" under "Your API keys". Copy this somewhere safe as well.


If you don't already have Python3, you can get it from or install it from your system's package manager.

conda is also a good option to get a working Python environment out of the box



If you have homebrew installed:

brew cask install docker

Otherwise you can install manually at

You should also increase the RAM allocated to the VM by changing "Memory" in the following:


Install docker desktop at

After you've installed docker, you must enable drive sharing:

You should also increase the RAM allocated to the VM by changing "Memory" in the following:

Docker Toolbox

If your machine doesn't have Hyper-V enabled, then you will have to install docker toolbox:

After it's installed, open the "Docker QuickStart Terminal" and run the following to increase its RAM:

docker-machine rm default
docker-machine create -d virtualbox --virtualbox-cpu-count=2 --virtualbox-memory=4096 --virtualbox-disk-size=50000 default

Also note, your code must live somewhere under your User directory (ie. C:\Users\USER_NAME\ANY_FOLDER). This is a restriction of docker toolbox not sharing paths correctly otherwise.


Install docker through your distribution.


sudo apt install

Also make sure to add your user to the docker group:

sudo groupadd docker
sudo usermod -aG docker $USER

Then reboot or logout/login for this to take effect.

For other Linux distros, check out and find your distro on the sidebar.


Destroy the AWS environment

numerai destroy

And then uninstall the package:

pip3 uninstall numerai-cli


Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for numerai-cli, version 0.1.23
Filename, size File type Python version Upload date Hashes
Filename, size numerai_cli-0.1.23-py3-none-any.whl (5.4 MB) File type Wheel Python version py3 Upload date Hashes View
Filename, size numerai-cli-0.1.23.tar.gz (5.4 MB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page