This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Backend worker daemon for Fedora Notifications

Project Description

fmn is a family of systems to manage end-user notifications triggered by fedmsg, the Fedora FEDerated MESsage bus.

This module contains the backend worker daemon for Fedora Notifications.

There is a parental placeholder repo with some useful information you might want to read through, like an overview, a little architecture diagram.

HACKING

Find development instructions here: https://github.com/fedora-infra/fmn/#hacking

RUNNING

There are three components to run in section of the project:

  • the fedmsg consumer
  • one or more workers
  • the backend

The consumer is the part of the application that receives the messages from fedmsg, it then sends them to the worker(s) who will see whose preferences the message match (ie: to whom to send the notification and on which channel). The worker(s) then send its results to the backend that receives them and just do the IO: sending the email, posting to IRC and so on.

To run these parts, simply, start rabbitmq-server and redis and call, in three different terminals (all running the same virtual-environment):

$ fedmsg-hub
$ python fmn/consumer/worker.py
$ python fmn/consumer/backend.py

Handy Script

There’s a handy script in the scripts/ dir for debugging why some user did or did not receive a message. It takes a username and a fedmsg msg_id and tries to see if the two match up or not based on the production preferences for that user.

Architecture

                                                     +-------------+
                                              Read   |             |   Write
                                              +------+  prefs DB   +<------+
                                              |      |             |       |
   +                                          |      +-------------+       |
   |                                          |                            |   +------------------+   +--------+
   |                                          |                            |   |    |fmn.lib|     |   |        |
   |                                          v                            |   |    +-------+     |<--+  User  |
   |                                    +----------+                       +---+                  |   |        |
   |                                    |   fmn.lib|                           |  Central WebApp  |   +--------+
   |                                    |          |                           +------------------+
   |                             +----->|  Worker  +--------+
   |                             |      |          |        |
fedmsg                           |      +----------+        |
   |                             |                          |
   |                             |      +----------+        |
   |   +------------------+      |      |   fmn.lib|        |       +--------------------+
   |   | fedmsg consumer  |      |      |          |        |       | Backend            |
   +-->|                  +------------>|  Worker  +--------------->|                    |
   |   |                  |      |      |          |        |       +-----+   +---+  +---+
   |   +------------------+      |      +----------+        |       |email|   |IRC|  |SSE|
   |                             |                          |       +--+--+---+-+-+--+-+-+
   |                             |      +----------+        |          |        |      |
   |                             |      |   fmn.lib|        |          |        |      |
   |                             |      |          |        |          |        |      |
   |                             +----->|  Worker  +--------+          |        |      |
   |                         RabbitMQ   |          |    RabbitMQ       |        |      |
   |                                    +----------+                   |        |      |
   |                                                                   v        v      v
   |
   |
   |
   v

FAQ

Q: If the worker and backend complain about

self.db[key] = value
TypeError: String or Integer object expected for key, unicode found

A: Modify the config for fedmsg.d/fmn.py to use redis

# Some configuration for the rule processors
"fmn.rules.utils.use_pkgdb2": False,
"fmn.rules.utils.pkgdb2_api_url": "http://209.132.184.188/api/",
"fmn.rules.cache": {
    'backend': 'dogpile.cache.redis',
    'arguments': {
        'host': 'localhost',
        'port': 6379,
        'db': 0,
        'redis_expiration_time': 60*60*2,   # 2 hours
        'distributed_lock': True
    },
    # "backend": "dogpile.cache.dbm",
    # "expiration_time": 300,
    # "arguments": {
    #     "filename": "/var/tmp/fmn-cache.dbm",
    # },
},

Q: When I run the worker and backend it constantly quits right away

A: Do you have rabbitmq-server and redis running?

systemctl status rabbitmq-server redis

if not run

systemctl start rabbitmq-server redis

Q: When running fedmsg-hub it shows that stuff are not initialized in the output

A: Have you updated the repo and ran the setup again?

git pull
workon fmn
python setup.py develop

Q: I have fedmsg-hub, worker.py and backend running now what?

A: Load up the web interface and enable IRC and add your nick. Checkout fmn.web and fmn for more info.

Q: It’s saying my nick is invalid

A: Is your nick registered? If not run this with the correct information

/msg NickServ REGISTER password youremail@example.com

src: freenode

Q: How do I know if it’s working?

A: In IRC you should get contacted by threebot

<threebot> <user>.id.fedoraproject.org has requested that notifications be sent to this nick
<threebot> * To accept, visit this address:

Q: How do I stress test?

A: To be continued. Scripts are in process of being created

Release History

Release History

This version
History Node

1.0.3

History Node

1.0.2

History Node

1.0.1

History Node

1.0.0

History Node

0.8.1

History Node

0.6.3

History Node

0.6.2

History Node

0.6.1

History Node

0.6.0

History Node

0.5.2

History Node

0.5.1

History Node

0.5.0

History Node

0.4.5

History Node

0.4.4

History Node

0.4.3

History Node

0.4.1

History Node

0.3.1

History Node

0.3.0

History Node

0.2.7

History Node

0.2.6

History Node

0.2.5

History Node

0.2.4

History Node

0.2.3

History Node

0.2.2

History Node

0.2.1

History Node

0.2.0

History Node

0.1.3

History Node

0.1.2

History Node

0.1.1

History Node

0.1.0

History Node

0.0.1

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
fmn.consumer-1.0.3.tar.gz (52.5 kB) Copy SHA256 Checksum SHA256 Source Jul 27, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting