Skip to main content

AWS Metadata Proxy

Project description

PyPi Travis PyUp

Metadata proxy

currently undergoing some refactoring

Very simple project to present an AWS compatible metadata service to servers on-premise funneling all of the credential requests through a central server which either has AWS credentials or is running in AWS and has a host iam role.


Metadata Server

TODO - make docker container

Run the following docker container terrycain/SOMECONTAINER It listens on HTTP 8000 so put it behind a HTTPS loadbalancer / reverse proxy,

TODO - Create the following Dynamo Tables metadata-proxy-hosts metadata-proxy-containers metadata-proxy-users


Here are some environment variables used to configure the metadata server, defaults are in parenthesis at the beginning.

  • REG_KEY - Random string used for initial client registration.
  • HOSTNAME_PREFIX - (ip) Hostnames will be generated like so: -> HOSTNAME_PREFIX-192-168-0-1 .
  • IAM_HOST_ROLE_CACHE_TTL - (120) Time in seconds to cache the list of IAM roles used for a dropdown list in the UI. This can take a while at times..

Currently Redis is a dependency but that can be a standalone redis ran alongside this container, will look to make it o ptional in the future.

  • REDIS_HOST - (localhost) Redis hostname.
  • REDIS_PORT - (6379) Redis port.
  • REDIS_DB - (0) Redis DB.

STS credentials are cached (currently in redis) and they are encrypted with an AES key. This allows for multiple servers to be ran in a HA manner and also reduces STS.assume_role requests.

  • STS_SEED - This should be a random secret that is long.

User authentication is either based on users in a DynamoDB table or via OpenID Connect.

  • USER_AUTH_METHOD - (oidc) Either oidc or dynamo
  • OIDC_CLIENT_ID - OpenID Connect client ID.
  • OIDC_CLIENT_SECRET - Client secret.
  • OIDC_BASE_URL - Base url of the OpenID server, at startup the server will hit the wellknown metadata url to get OpenID config.

DynamoDB configuration

  • DYNAMODB_REGION - (eu-west-1) Region the DynamoDB tables are located
  • DYNAMODB_HOSTS_TABLE - (metadata-proxy-hosts) Table that stores registered hosts and what role they have been assigned
  • DYNAMODB_CONTAINER_TABLE - (metadata-proxy-containers) Table that contains
  • DYNAMODB_USERS_TABLE - (metadata-proxy-users)

Metadata proxy

Todo - make pip package


Install the Python package (requires Python 3.5.3+)

sudo pip3 install terrycain-metadata-proxy[uvloop]

Create the following systemd unit file and associated users

sudo groupadd --system metadata-proxy
sudo useradd --system --gid metadata-proxy --home-dir /var/lib/metadata-proxy --shell /sbin/nologin metadata-proxy
sudo mkdir /var/lib/metadata-proxy
sudo chown metadata-proxy:metadata-proxy /var/lib/metadata-proxy
sudo chown 0700 /var/lib/metadata-proxy
cat << EOF | sudo tee /etc/systemd/system/metadata-proxy.service
Description=Metadata proxy

ExecStartPre=/sbin/iptables -t nat -I OUTPUT -p tcp -d --dport 80 -j DNAT --to-destination
ExecStopPost=/sbin/iptables -t nat -D OUTPUT -p tcp -d --dport 80 -j DNAT --to-destination


sudo systemctl daemon-reload
sudo systemctl enable --now metadata-proxy

Test that the proxy is now running and the iptables rules are working

curl ; echo
# Should output:
# c5.16xlarge

Register the metadata service against the master server with the curl command from the UI

curl --noproxy '*' -XPOST -H "Content-Type: application/json" --data '{"server_url": "", "key": "0753e6f9-5884-462d-bb46-d376f27047a1"}' 


Installing on Windows (Requires python 3.5.3+)

pip3 install terrycain-metadata-proxy

Create a service on Windows TODO

Register the metadata service against the master server with the curl command from the UI


  • Better logging
  • Add support for cert.pem, key.pem, chain.pem ENV vars so that it can do HTTPS
  • Finish documentation

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

terrycain-metadata-proxy-1.5.0.tar.gz (9.7 kB view hashes)

Uploaded source

Built Distribution

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page