Server to process the incoming facts from the Orion Context Broker
Project description
Introduction
This is the code repository for FIWARE Policy Manager GE - Facts, a server to process the incoming facts from the Orion Context Broker and publish the result into a RabbitMQ queue to be analysed by Fiware-Cloto. The facts are the result of the server resources consumption.
This project is part of FIWARE. Check also the FIWARE Catalogue entry for Policy Manager
Any feedback on this documentation is highly welcome, including bugs, typos or things you think should be included but aren’t. You can use github issues to provide feedback.
Top.
GEi overall description
Bosun GEri is the reference implementation of Policy Manager GE.
Bosun offers decision-making ability, independently of the type of resource (physical/virtual resources, network, service, etc.) being able to solve complex problems within the Cloud field by reasoning about the knowledge base, represented by facts and rules. Bosun GEri provides the basic management of cloud resources based on rules, as well as management of the corresponding resources within FIWARE Cloud instances based on infrastructure physical monitoring, resources and services security monitoring or whatever that could be defined by facts, actions and rules.
The baseline for the Bosun GEri is PyCLIPS, which is a module to interact with CLIPS expert system implemented in Python language. The reason to take PyCLIPS is to extend the OpenStack ecosystem with an expert system, written in the same language as the rest of the OpenStack services. Besides, It provides notification service to your own HTTP server where you can define your own actions based on the notifications launched by Policy Manager. Last but not least, Bosun is integrated with the Monitoring GEri in order to recover the information of the (virtual) system and calculate any possible change on it based on the knowledge database defined for it.
Top.
Components
- Fiware-Cloto
Fiware-Cloto is part of FIWARE Policy Manager. It provides a REST API to create rules associated to servers, subscribe servers to Context Broker to get information about resources consumption of that servers and launch actions described in rules when conditions are met.
- Fiware-Facts
Server to process the incoming facts from Orion Context Broker and publish the result into a RabbitMQ queue to be analysed by Fiware-Cloto. The facts are the result of the server resources consumption.
For more information, please refer to the documentation
Top.
Build and Install
Requirements
Operating systems: CentOS (RedHat) and Ubuntu (Debian), being CentOS 6.3 the reference operating system.
To install this module you have to install some components:
Python 2.7
Fiware-Cloto module (https://github.com/telefonicaid/fiware-cloto)
Redis 2.9.1 or above
RabbitMQ Server 3.3.0 or above (http://www.rabbitmq.com/download.html)
MySQL 5.6.14 or above (http://dev.mysql.com/downloads/mysql/)
Please, be sure you have installed mysql-devel package for development of MySQL applications. You should be able to install it from yum or apt-get package managers.
Examples:
centos$ sudo yum install mysql-devel
ubuntu$ sudo apt-get install mysql-develTop.
Installation
Using pip Install the component by executing the following instruction:
$ sudo pip install fiware-factsThis operation will install the component in your python site-packages folder.
Top.
Configuration file
The configuration used by the fiware-facts component is read from the file located at /etc/fiware.d/fiware-facts.cfg.
MySQL cloto configuration must be filled before starting fiware-facts component, user and password are empty by default. You can copy the default configuration file to the folder defined for your OS, and complete data about Cloto MySQL configuration (user and password).
In addition, user could have a copy of this file in other location and pass its location to the server in running execution defining an environment variable called FACTS_SETTINGS_FILE.
Options that user could define:
[common]
brokerPort: 5000 # Port listening fiware-facts
clotoPort: 8000 # Port listening fiware-cloto
redisPort: 6379 # Port listening redis-server
redisHost: localhost # Address of redis-server
redisQueue: policymanager
rabbitMQ: localhost # Address of RabbitMQ server
cloto: 127.0.0.1 # Address of fiware-cloto
clotoVersion: v1.0
name: policymanager.facts
maxTimeWindowsize: 10
[mysql]
host: localhost # address of mysql that fiware-cloto is using
charset: utf8
db: cloto
user: # mysql user
password: # mysql password
[loggers]
keys: root
[handlers]
keys: console, file
[formatters]
keys: standard
[formatter_standard]
class: logging.Formatter
format: %(asctime)s %(levelname)s policymanager.facts %(message)s
[logger_root]
level: INFO # Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
handlers: console, file
[handler_console]
level: DEBUG
class: StreamHandler
formatter: standard
args: (sys.stdout,)
[handler_file]
level: DEBUG
class: handlers.RotatingFileHandler
formatter: standard
logFilePath: /var/log/fiware-facts
logFileName: fiware-facts.log
logMaxFiles: 3
logMaxSize: 5*1024*1024 ; 5 MB
args: ('%(logFilePath)s/%(logFileName)s', 'a', %(logMaxSize)s, %(logMaxFiles)s)
Top.
Running
Execute command:
$ gunicorn facts.server:app -b $IP:5000Where $IP should be the IP assigned to the network interface that should be listening (ej. 192.168.1.33)
You can also execute the server with a different settings file providing an environment variable with the location of the file:
$ gunicorn facts.server:app -b $IP:5000 --env FACTS_SETTINGS_FILE=/home/user/fiware-facts.cfgNOTE: if you want to see gunicorn log if something is going wrong, you could execute the command before adding --log-file=- at the end of the command. This option will show the logs in your prompt.
Finally, ensure that you create a folder for logs /var/log/fiware-facts/ (by default), with the right permissions to write in that folder.
$ sudo mkdir -p /var/log/fiware-factsRunning with supervisor
Optionally you can add a new layer to manage gunicorn process with a supervisor. Just install supervisor on your system:
$ sudo apt-get install supervisorCopy the file utils/facts_start to /etc/fiware.d. Make this script executable:
$ sudo chmod a+x /etc/fiware.d/facts_startCopy the file utils/fiware-facts.conf to /etc/supervisor/conf.d.
Start fiware-facts using supervisor:
$ sudo supervisorctl reread
$ sudo supervisorctl update
$ sudo supervisorctl start fiware-factsTo stop fiware-facts just execute:
$ sudo supervisorctl stop fiware-factsNOTE: Supervisor provides an “event listener” to subscribe to “event notifications”. The purpose of the event notification/subscription system is to provide a mechanism for arbitrary code to be run (e.g. send an email, make an HTTP request, etc) when some condition is satisfied. That condition usually has to do with subprocess state. For instance, you may want to notify someone via email when a process crashes and is restarted by Supervisor. For more information check also the Supervisor Documentation.
Top.
API Overview
Servers will update their context. The context information contains the description of the CPU, Memory, Disk and Network usages.
An example of this operation could be:
$ curl --include \
--request POST \
--header "Content-Type: application/json" \
--data-binary "{
"contextResponses": [
{
"contextElement": {
"attributes": [
{
"value": "0.12",
"name": "usedMemPct",
"type": "string"
},
{
"value": "0.14",
"name": "cpuLoadPct",
"type": "string"
},
{
"value": "0.856240",
"name": "freeSpacePct",
"type": "string"
},
{
"value": "0.8122",
"name": "netLoadPct",
"type": "string"
}
],
"id": "Trento:193.205.211.69",
"isPattern": "false",
"type": "host"
},
"statusCode": {
"code": "200",
"reasonPhrase": "OK"
}
}
]
}" \
'http://policymanager-host.org:5000/v1.0/d3fdddc6324c439780a6fd963a9fa148/servers/52415800-8b69-11e0-9b19-734f6af67565'This message follows the NGSI-10 information model but using JSON format.
The response has no body and should return 200 OK.
Top.
API Reference Documentation
Top.
Testing
Unit tests
To execute the unit tests you must have a redis-server and a rabbitmq-server up and running. Please take a look to the installation manual in order to configure those components.
After that, you can execute this folloing commands:
$ pip install -r requirements_dev.txt
$ export PYTHONPATH=$PWD
$ nosetests -s -v --cover-package=facts --with-coverTop.
End-to-end tests
Once you have fiware-facts running you can check the server executing:
$ curl http://$HOST:5000/v1.0Where:
$HOST: is the url/IP of the machine where fiware facts is installed, for example: (policymanager-host.org, 127.0.0.1, etc)
The request before should return a response with this body if everything is ok:
{"fiware-facts":"Up and running..."}
Please refer to the Installation and administration guide for details.
Top.
Acceptance tests
All detailed documentation about acceptance tests can be consulted in FACTS Acceptance Test Project
Requirements
Environment preparation
Create a virtual environment somewhere:
$ virtualenv $WORKON_HOME/venv
Activate the virtual environment:
$ source $WORKON_HOME/venv/bin/activate)
Go to $FACTS_HOME/tests/acceptance folder in the project.
Install the requirements for the acceptance tests in the virtual environment:
$ pip install -r requirements.txt --allow-all-external)
Execution
Execute the following command in the acceptance test project directory:
$ cd $FACTS_HOME/tests/acceptance $ behave features/component --tags ~@skip
Before executing, you shoud configure properly the project settings file in $FACTS_HOME/tests/acceptance/settings/settings.json. Take a look at the FACTS Acceptance Test Project documentation.
Top.
Advanced topics
Top.
Support
Ask your thorough programming questions using stackoverflow and your general questions on FIWARE Q&A. In both cases please use the tag fiware-bosun.
Top.
License
(c) 2014-2016 Telefónica Investigación y Desarrollo S.A.U., Apache License 2.0
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
