Skip to main content
Join the official 2020 Python Developers SurveyStart the survey!

Construct service request URLs

Project description

Constructs URLs that targeting other services.

Version Python Source Coverage Quality Docs CI

This library makes building URLs for inter-service communication safer and easier to build.

URL building

url = klempner.url.build_url('account', 'path', 'with spaces',
                             query='arg',  multi=['arg', 'support'])
# http://account/path/with%20spaces?query=arg&multi=arg&multi=support

build_url takes care of formatting the path and query parameters correctly in addition to discovering the service name. In this example, the service name is used as-is (see Unconfigured usage below). The real power in build_url is its ability to discover the scheme, host name, and port number based on the operating environment.

build_url uses the http scheme by default. If the port is determined by the discovery mechanism, then the scheme is set using a simple global mapping from port number to scheme.

Discovery examples

Unconfigured usage

url = klempner.url.build_url('account')
print(url)  # http://account/

This isn’t very useful but if you do not configure the discovery mechanism, then build_url assumes that the requested service is accessible directly by name.

Consul service discovery

The basic form of using consul is not discovery at all. It is simply URL construction that follows the naming convention that Consul’s DNS interface exposes.

os.environ['KLEMPNER_DISCOVERY'] = 'consul'
os.environ['CONSUL_DATACENTER'] = 'production'
url = klempner.url.build_url('account')
print(url)  # http://account.service.production.consul/

If you append +agent to the discovery method, then build_url will connect to a Consul agent and retrieve the port number for services. If the port has a registered service associated with it, then the service name will be used as the scheme.

Assuming that the account service is registered in consul with a service port of 8000:

os.environ['KLEMPNER_DISCOVERY'] = 'consul+agent'
url = klempner.url.build_url('account')
print(url)  # http://account.service.production.consul:8000/

Now let’s look at what happens for a RabbitMQ connection:

url = klempner.url.build_url('rabbit')
print(url)  # amqp://rabbit.service.production.consul:5432/

The scheme is derived by looking up the port in the klempner.config.URL_SCHEME_MAP and using the result if the lookup succeeds.

The library will connect to the agent specified by the CONSUL_HTTP_ADDR environment variable. If the environment variable is not specified, then the agent listening on the localhost will be used.

Kubernetes service discovery

os.environ['KLEMPNER_DISCOVERY'] = 'kubernetes'
url = klempner.url.build_url('account')
print(url)  # http://account.default.svc.cluster.local/
os.environ['KLEMPNER_DISCOVERY'] = 'kubernetes'
os.environ['KUBERNETES_NAMESPACE'] = 'my-team'
url = klempner.url.build_url('account')
print(url)  #

Docker-compose service discovery

os.environ['KLEMPNER_DISCOVERY'] = 'docker-compose'
os.environ['COMPOSE_PROJECT_NAME'] = 'foo'
url = klempner.url.build_url('account')
print(url)  #

This discovery mechanism discovers IP and port numbers for services using the Docker API. build_url retrieves the list of services from the docker host, filters the list using the “com.docker.compose.project” label, and selects the service using the “com.docker.compose.service” label.

Environment variable discovery

This form of discovery uses environment variables with the service name encoded into them:

os.environ['KLEMPNER_DISCOVERY'] = 'environment'
os.environ['ACCOUNT_HOST'] = ''
os.environ['ACCOUNT_PORT'] = '11223'
url = klempner.url.build_url('account')
print(url)  #

For a service named adder, the following environment variables are used if they are set.

Name URL component Default
ADDER_HOST host portion of the authority none
ADDER_PORT port portion of the authority omitted
ADDER_SCHEME scheme see below

The URL scheme defaults to looking up the port number in the klempner.config.URL_SCHEME_MAP dictionary. If the port number is not in the dictionary, then http is used as a default.

os.environ['KLEMPNER_DISCOVERY'] = 'environment'
os.environ['ACCOUNT_HOST'] = ''
os.environ['ACCOUNT_PORT'] = '443'
url = klempner.url.build_url('account')
print(url)  #

Want to contribute?

Thank you. See docs/contributing.rst or for what you need to do.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for klempner, version 0.0.3
Filename, size File type Python version Upload date Hashes
Filename, size klempner-0.0.3.tar.gz (13.7 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page