Python Microservices Library

PyMS, Python MicroService, is a Microservice chassis pattern like Spring Boot (Java) or Gizmo (Golang). PyMS is a collection of libraries, best practices and recommended ways to build microservices with Python which handles cross-cutting concerns:

• Externalized configuration
• Logging
• Health checks
• Metrics
• Distributed tracing

PyMS is powered by Flask, Connexion and Opentracing.

Get started with Installation and then get an overview with the Quickstart.

Documentation

To know how use, install or build a project see the docs.

Motivation

When we started to create microservice with no idea, we were looking for tutorials, guides, best practices, but we found nothing to create professional projects. Most articles say:

• "Install flask"
• "Create routes"
• (Sometimes) "Create a swagger specs"
• "TA-DA! you have a microservice"

But... what happens with our configuration out of code like Kubernetes configmap? what happens with transactionality? If we have many microservices, what happens with traces?.

There are many problems around Python and microservices and we can`t find anyone to give a solution.

We start creating these projects to try to solve all the problems we have found in our professional lives about microservices architecture.

Nowadays, is not perfect and we have a looong roadmap, but we hope this library could help other fellas and friends ;)

Installation

pip install py-ms

Quickstart

You need to create 2 files: main.py and config.yml:

• main.py

from flask import jsonify

from pyms.flask.app import Microservice

ms = Microservice(path=__file__) # 1.1
app = ms.create_app() # 2.1


@app.route("/") # 3.1
def example():
  return jsonify({"main": "hello world"})


if __name__ == '__main__':
  app.run()

• config.yml

pyms:
  services: # 1.2
  requests:
  data: {}
  config: # 1.3
  DEBUG: true
  APP_NAME: business-glossary
  APPLICATION_ROOT : ""
  SECRET_KEY: "gjr39dkjn344_!67#"

So what did that code do?

1. Create a instance of PyMS Microservice class (#1.1). This initialization inject the configuration defined in the 1.3 block and could be accessed through current_app.config like typical Flask config. Then, initialize the service defined in the 1.2 block. See Services for more details.

2. Initialize Flask instance, Connexion if it was defined in the pyms configuration block, create a tracer, add health-check blueprint, initialize libs and set the PyMS Microservice in ms attribute and you can access to it with current_app.ms.
   This steps has their each functions and you can easy override it.

3. create_app return the flask instance and you can interact with it as a typical flask app

See Documentation to learn more.

Create a project from scaffold

PyMS has a command line option to create a project template like Microservices Scaffold. This command use cookiecutter to download and install this template

[Warning] You must run first pip install cookiecutter==1.7.0

pyms startproject

this output a lot of options step by step:

project_repo_url [https://github.com/python-microservices/microservices-scaffold]: 
project_name [Python Microservices Boilerplate]: example project
project_folder [example_project]: 
project_short_description [Python Boilerplate contains all the boilerplate you need to create a Python package.]: 
create_model_class [y]: 
microservice_with_swagger_and_connexion [y]: 
microservice_with_traces [y]: 
microservice_with_metrics [y]: 
application_root [/example_project]: 
Select open_source_license:
1 - MIT license
2 - BSD license
3 - ISC license
4 - Apache Software License 2.0
5 - GNU General Public License v3
6 - Not open source
Choose from 1, 2, 3, 4, 5, 6 [1]: 

When you finish to introduce the options, a project will be created in [project_folder] folder

How To Contrib

We appreciate opening issues and pull requests to make PyMS even more stable & useful! See This doc for more details.

Features

• Create project from scaffold #90 PyMS has a command line option to create a project template like Microservices Scaffold. This command use cookiecutter to download and install this template [Warning] You must run first pip install cookiecutter==1.7.0

  pyms startproject
  

  this output a lot of options step by step:

  project_repo_url [https://github.com/python-microservices/microservices-scaffold]: 
  project_name [Python Microservices Boilerplate]: example project
  project_folder [example_project]: 
  project_short_description [Python Boilerplate contains all the boilerplate you need to create a Python package.]: 
  create_model_class [y]: 
  microservice_with_swagger_and_connexion [y]: 
  microservice_with_traces [y]: 
  microservice_with_metrics [y]: 
  application_root [/example_project]: 
  Select open_source_license:
  1 - MIT license
  2 - BSD license
  3 - ISC license
  4 - Apache Software License 2.0
  5 - GNU General Public License v3
  6 - Not open source
  Choose from 1, 2, 3, 4, 5, 6 [1]: 
  

  When you finish to introduce the options, a project will be created in [project_folder] folder

• Encrypt/decrypt config (#86)

  • New encryption and decryption feature with AES
  • Create a key to encrypt config with new command line pyms create-key
  • Encrypt a string with the new command line pyms encrypt [STRING]
  • Set the key file with new environment variable KEY_FILE
  • If you set in your config a var with the prefix "ENC_" or "enc_" PyMS search for the key file and decrypt the string. Now you can encrypt your database url for production environments. I.E.:

  pyms:
  [...]
  config:
  ENC_DATABASE: gAAAAABeSZ714r99iRIxhoH77vTdRJ0iqSymShfqgGN9PJveqhQWmshRDuV2a8sATey8_lHkln0TwezczucH-aJHGP_LyEiPxwM-88clNa7FB1u4g7Iaw3A=
  

  you can access it in your code with current_app.config["DATABASE"] == "http://database-url"

• Human readable config errors #85 Show more help information if the config file not have a good structure with error messages like

  Config file must start with `pyms` keyword, for example:
  pyms:
  services:
  metrics: true
  requests:
  data: data
  swagger:
  path: ""
  file: "swagger.yaml"
  tracer:
  client: "jaeger"
  host: "localhost"
  component_name: "Python Microservice"
  config:
  DEBUG: true
  TESTING: true
  

• Add service name label by default #84 - @alex It adds a label to all jaeger's generated metrics that can be used to identify every microservice. See jaegertracing/jaeger-client-python#269. I messed up the previous PR, it's the same as this. https://github.com/python-microservices/pyms/pull/72 Fixes

• Send path to ConfFile from Microservice class to set path from code if you want to not use CONFIGMAP_FILE from env

Refactors

• Refactor config. Memoize of files moved to new class