Skip to main content

Celery platform to create microservices

Project description

# Micro
A platform to create microservices available through Celery API.

## Micro API
Micro uses a very simple API to run, list and get information about plugins:

* `plugins()`: list all available plugins.
* `info(plugin_name)`: show information about a specific plugin.
* `help(plugin_name)`: show the plugin help.
* `run(plugin_name, params)`: execute the given plugin.

To use this API you can use the [Micro-dev](https://github.com/humu1us/micro-dev) package who provide two important libraries, the access to the Celery API and the PluginBase class who allow writing Micro plugins.

### API example

```python
>>> from micro.api.endpoints import plugins, run
>>>
>>> plugins.delay().wait()
{'Example plugin': 'A very simple example plugin'}
>>>
>>> run.delay("Example plugin", name="Micro").wait()
'Hello Micro!!!'
```


## Micro plugins

Write a plugin to Micro is very simple all that you need is create a file named `interface.py` this file defines the plugin as follow:

```python
from micro.plugin.pluginbase import PluginBase
from micro.plugin.pluginbase import PluginDescription


class ExamplePlugin(PluginBase):
def __init__(self):
print("This is an example plugin")

# This is the method executed by Micro
def run(self, **kwargs):
return "Hello " + kwargs["name"] + "!!!"

# This description is required by Micro
plugin = PluginDescription(
name="Example Plugin",
author="Jhon Doe",
short_desc="A very simple example plugin",
long_desc="This plugin is a very simple example, "
"for that reason, we don't have a long description"
help_str="Params: name type string; A name to greet",
instance=ExamplePlugin
)
```
Each plugin needs to have its own folder inside of the plugins directory (check the section ["Configuring Micro"](https://github.com/humu1us/micro#configuring-micro) for details)

The plugin directory should look like this:

```
my_plugindir/
example-plugin/
interface.py
other-plugin/
inteface.py
```


## Installation
Development version:
```
$ git clone git@github.com:humu1us/micro.git
$ cd micro
$ pip install .
```
or direct from repo
```
$ pip install git+ssh://git@github.com/humu1us/micro.git
```

## Configuration
### Parameters priority
Micro can be configurated through CLI, environment variables, config file and/or default values (in that order).

### Command line (CLI)
These arguments are the highest priority for Micro. So, these overwrite any other parameters set by any other method. The CLI arguments that can be used are:

```
$ micro -h
usage: micro [-h] [-b BROKER_URL] [-q QUEUE_NAME] [-H HOSTNAME]
[-w NUM_WORKERS] [-lp LOG_PATH] [-pp PID_PATH]
[--default-params]

optional arguments:
-h, --help show this help message and exit
-b BROKER_URL, --broker-url BROKER_URL
Set the broker url
-q QUEUE_NAME, --queue-name QUEUE_NAME
Set the Celery queue name
-H HOSTNAME, --hostname HOSTNAME
Set the hostname for the workers
-w NUM_WORKERS, --num-workers NUM_WORKERS
Set the Celery worker number
-lp LOG_PATH, --log-path LOG_PATH
Set the log file path
-pp PID_PATH, --pid-path PID_PATH
Set the pid file path
--default-params Show default parameters
```

### Environment variables
The next priority in parameters for Micro are environment variables. The list of environment variables used are:

```
MICRO_CONFIG # config file location: /path/to/config/config.json
MICRO_PLUGIN_PATH # path to plugin folder: /path/to/plugin/folder
MICRO_LOG_PATH # path to log folder: /path/to/plugin/folder
MICRO_LOG_FROM # minimun log level to write: DEBUG, INFO, WARNING, ERROR, CRITICAL or FATAL
MICRO_BROKER_URL # broker url: ampq://user:pass@host:port//
MICRO_QUEUE_NAME # queue name used
MICRO_HOSTNAME # workers hostname
MICRO_NUM_WORKERS # number of workers to create (integer number)
MICRO_CELERY_LOG_PATH # path to Celery log folder: /path/to/celery/log/folder
MICRO_CELERY_PID_PATH # path to Celery pid folder: /path/to/celery/pid/folder
```

**IMPORTANT:** `MICRO_CONFIG`, `MICRO_PLUGIN_PATH`, `MICRO_LOG_PATH` and `MICRO_LOG_FROM` variables provide the only way to set config file, the plugin folder path, the logger file path and the logger level.

### Config file
The lowest priority is the use of a JSON config file. The path to this config file must be set using `MICRO_CONFIG` environment variable.

Config file example:

```js
{
"broker_url": "ampq://user:pass@host:port//",
"queue_name": "",
"hostname": "",
"num_workers": ,
"log_path": "/path/to/log/folder",
"pid_path": "/path/to/pid/folder"
}
```

A config file skeleton can be created using the following command:
`$ micro --default-params > config.json`

### Default values
The default values are:

```
$ micro --default-params
{
"broker_url": "",
"queue_name": "micro_queue",
"hostname": "micro",
"num_workers": 1,
"log_path": "/var/log",
"pid_path": "/var/run"
}
```

## Docker

### Build

To build the container first move to the branch/tag to use and then use the following command

```
$ docker build -t micro:0.0.1 .
```

At this moment `0.0.1` is the recommended version.

### Run
Run Micro as container is pretty easy and only needs to define `MICRO_BROKER_URL` to set the amqp host. All Micro environment variables are available with `-e` flag, for example:

```
$ docker run -e MICRO_BROKER_URL="amqp://guest:guest@my_host:5672//" -e MICRO_NUM_WORKERS=5 micro:0.0.1
```

The `MICRO_BROKER_URL` is the only mandatory environment variable to use


## Tests

Run all unit tests with:

```
$ python setup.py test
```


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Micro-1.0.0.tar.gz (10.4 kB view hashes)

Uploaded Source

Built Distribution

Micro-1.0.0-py3-none-any.whl (17.6 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page