klempner 0.0.3

Construct service request URLs

Constructs URLs that targeting other services.

This library makes building URLs for inter-service communication safer and easier to build.

URL building

url = klempner.url.build_url('account', 'path', 'with spaces',
                             query='arg',  multi=['arg', 'support'])
print(url)
# http://account/path/with%20spaces?query=arg&multi=arg&multi=support

build_url takes care of formatting the path and query parameters correctly in addition to discovering the service name. In this example, the service name is used as-is (see Unconfigured usage below). The real power in build_url is its ability to discover the scheme, host name, and port number based on the operating environment.

build_url uses the http scheme by default. If the port is determined by the discovery mechanism, then the scheme is set using a simple global mapping from port number to scheme.

Discovery examples

Unconfigured usage

url = klempner.url.build_url('account')
print(url)  # http://account/

This isn’t very useful but if you do not configure the discovery mechanism, then build_url assumes that the requested service is accessible directly by name.

Consul service discovery

The basic form of using consul is not discovery at all. It is simply URL construction that follows the naming convention that Consul’s DNS interface exposes.

os.environ['KLEMPNER_DISCOVERY'] = 'consul'
os.environ['CONSUL_DATACENTER'] = 'production'
url = klempner.url.build_url('account')
print(url)  # http://account.service.production.consul/

If you append +agent to the discovery method, then build_url will connect to a Consul agent and retrieve the port number for services. If the port has a registered service associated with it, then the service name will be used as the scheme.

Assuming that the account service is registered in consul with a service port of 8000:

os.environ['KLEMPNER_DISCOVERY'] = 'consul+agent'
url = klempner.url.build_url('account')
print(url)  # http://account.service.production.consul:8000/

Now let’s look at what happens for a RabbitMQ connection:

url = klempner.url.build_url('rabbit')
print(url)  # amqp://rabbit.service.production.consul:5432/

The scheme is derived by looking up the port in the klempner.config.URL_SCHEME_MAP and using the result if the lookup succeeds.

The library will connect to the agent specified by the CONSUL_HTTP_ADDR environment variable. If the environment variable is not specified, then the agent listening on the localhost will be used.

Kubernetes service discovery

os.environ['KLEMPNER_DISCOVERY'] = 'kubernetes'
url = klempner.url.build_url('account')
print(url)  # http://account.default.svc.cluster.local/

os.environ['KLEMPNER_DISCOVERY'] = 'kubernetes'
os.environ['KUBERNETES_NAMESPACE'] = 'my-team'
url = klempner.url.build_url('account')
print(url)  # http://account.my-team.svc.cluster.local/

Docker-compose service discovery

os.environ['KLEMPNER_DISCOVERY'] = 'docker-compose'
os.environ['COMPOSE_PROJECT_NAME'] = 'foo'
url = klempner.url.build_url('account')
print(url)  # http://127.0.0.1:32867/

This discovery mechanism discovers IP and port numbers for services using the Docker API. build_url retrieves the list of services from the docker host, filters the list using the “com.docker.compose.project” label, and selects the service using the “com.docker.compose.service” label.

Environment variable discovery

This form of discovery uses environment variables with the service name encoded into them:

os.environ['KLEMPNER_DISCOVERY'] = 'environment'
os.environ['ACCOUNT_HOST'] = '10.2.12.23'
os.environ['ACCOUNT_PORT'] = '11223'
url = klempner.url.build_url('account')
print(url)  # http://10.2.12.23:11223/

For a service named adder, the following environment variables are used if they are set.

Name	URL component	Default
ADDER_HOST	host portion of the authority	none
ADDER_PORT	port portion of the authority	omitted
ADDER_SCHEME	scheme	see below

The URL scheme defaults to looking up the port number in the klempner.config.URL_SCHEME_MAP dictionary. If the port number is not in the dictionary, then http is used as a default.

os.environ['KLEMPNER_DISCOVERY'] = 'environment'
os.environ['ACCOUNT_HOST'] = '10.2.12.23'
os.environ['ACCOUNT_PORT'] = '443'
url = klempner.url.build_url('account')
print(url)  # https://10.2.12.23:443/

Want to contribute?

Thank you. See docs/contributing.rst or https://klempner.readthedocs.io/en/latest/contributing.html for what you need to do.