Skip to main content

AWS Metadata Proxy

Project description

PyPi Travis PyUp

Metadata proxy

currently undergoing some refactoring

Very simple project to present an AWS compatible metadata service to servers on-premise funneling all of the credential requests through a central server which either has AWS credentials or is running in AWS and has a host iam role.

Installation

Metadata Server

TODO - make docker container

Run the following docker container terrycain/SOMECONTAINER It listens on HTTP 8000 so put it behind a HTTPS loadbalancer / reverse proxy,

TODO - Create the following Dynamo Tables metadata-proxy-hosts metadata-proxy-containers metadata-proxy-users

Configuration

Here are some environment variables used to configure the metadata server, defaults are in parenthesis at the beginning.

  • REG_KEY - Random string used for initial client registration.
  • HOSTNAME_PREFIX - (ip) Hostnames will be generated like so: 192.168.0.1 -> HOSTNAME_PREFIX-192-168-0-1 .
  • IAM_HOST_ROLE_CACHE_TTL - (120) Time in seconds to cache the list of IAM roles used for a dropdown list in the UI. This can take a while at times..

Currently Redis is a dependency but that can be a standalone redis ran alongside this container, will look to make it o ptional in the future.

  • REDIS_HOST - (localhost) Redis hostname.
  • REDIS_PORT - (6379) Redis port.
  • REDIS_DB - (0) Redis DB.

STS credentials are cached (currently in redis) and they are encrypted with an AES key. This allows for multiple servers to be ran in a HA manner and also reduces STS.assume_role requests.

  • STS_SEED - This should be a random secret that is long.

User authentication is either based on users in a DynamoDB table or via OpenID Connect.

  • USER_AUTH_METHOD - (oidc) Either oidc or dynamo
  • OIDC_CLIENT_ID - OpenID Connect client ID.
  • OIDC_CLIENT_SECRET - Client secret.
  • OIDC_BASE_URL - Base url of the OpenID server, at startup the server will hit the wellknown metadata url to get OpenID config.

DynamoDB configuration

  • DYNAMODB_REGION - (eu-west-1) Region the DynamoDB tables are located
  • DYNAMODB_HOSTS_TABLE - (metadata-proxy-hosts) Table that stores registered hosts and what role they have been assigned
  • DYNAMODB_CONTAINER_TABLE - (metadata-proxy-containers) Table that contains
  • DYNAMODB_USERS_TABLE - (metadata-proxy-users)

Metadata proxy

Todo - make pip package

Linux

Install the Python package (requires Python 3.5.3+)

sudo pip3 install terrycain-metadata-proxy[uvloop]

Create the following systemd unit file and associated users

sudo groupadd --system metadata-proxy
sudo useradd --system --gid metadata-proxy --home-dir /var/lib/metadata-proxy --shell /sbin/nologin metadata-proxy
sudo mkdir /var/lib/metadata-proxy
sudo chown metadata-proxy:metadata-proxy /var/lib/metadata-proxy
sudo chown 0700 /var/lib/metadata-proxy
cat << EOF | sudo tee /etc/systemd/system/metadata-proxy.service
[Unit]
Description=Metadata proxy
Requires=network.target
After=network.target

[Service]
Environment=PYTHONUNBUFFERED=1
TimeoutStartSec=0
Restart=always
User=metadata-proxy
Group=metadata-proxy
WorkingDirectory=/var/lib/metadata-proxy
PermissionsStartOnly=true
ExecStartPre=/sbin/iptables -t nat -I OUTPUT -p tcp -d 169.254.169.254 --dport 80 -j DNAT --to-destination 127.0.0.1:8000
ExecStart=/usr/local/bin/metadata-proxy
ExecStopPost=/sbin/iptables -t nat -D OUTPUT -p tcp -d 169.254.169.254 --dport 80 -j DNAT --to-destination 127.0.0.1:8000

[Install]
WantedBy=default.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now metadata-proxy

Test that the proxy is now running and the iptables rules are working

curl http://169.254.169.254/latest/meta-data/instance-type ; echo
# Should output:
# c5.16xlarge

Register the metadata service against the master server with the curl command from the UI

curl --noproxy '*' -XPOST http://169.254.169.254/register -H "Content-Type: application/json" --data '{"server_url": "https://metadata-eu.ficoccs-prod.net/api/v1/register", "key": "0753e6f9-5884-462d-bb46-d376f27047a1"}' 

Windows

Installing on Windows (Requires python 3.5.3+)

pip3 install terrycain-metadata-proxy

Create a service on Windows TODO

Register the metadata service against the master server with the curl command from the UI

TODO List

  • Better logging
  • Add support for cert.pem, key.pem, chain.pem ENV vars so that it can do HTTPS
  • Finish documentation

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for terrycain-metadata-proxy, version 1.5.0
Filename, size File type Python version Upload date Hashes
Filename, size terrycain_metadata_proxy-1.5.0-py3-none-any.whl (10.0 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size terrycain-metadata-proxy-1.5.0.tar.gz (9.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page