livebridge 0.27.0

Keep content in-sync across services. Or simply syndicate content to multiple services. Based on asyncio.

Livebridge by dpa-newslab

Millions of users read dpa-Live updates when a new US president is elected or an ongoing attack of unknown origin keeps the nation awake. We strive to deliver these updates live to whatever publishing system our customers chose. So we developed this open-source software as part of a project funded by the Google DNI Fund. Read Introducing Live Coverage Ecosystem for more details.

We’re already using it in production, delivering content produced using Sourcefabric’s Liveblog in the dpa newsroom to media customers who use the service of ScribbleLive.

Features

• Keep content in-sync over different services.
• syndicate one source to various targets in realtime.
• CRUD - create, update, delete of resources over different services.
• extensible for all kinds of services with plugins
• Web-UI for controlling running bridges through a convenient web-frontend.
• Web-API for controlling bridges.
• service credentials as environment vars.
• supported storage backends: MongoDB, DynamoDB, MySQL, PostgreSQL, MSSQL, Oracle and others
• await/async based, Python 3.5
• (non-persistent) queues for retrying the distribution of a post.
• focus on robustness and stability

Installation

pip3 install livebridge

Python >=3.5 is required.
See http://livebridge.readthedocs.io/en/stable/quickstart.html#installation for details.

Settings

Global settings are defined in livebridge/config.py. These values can be set through environment variables.

See http://livebridge.readthedocs.io/en/stable/quickstart.html#settings for available configuration variables.

Running

The livebridge command expects a -control= parameter (alternatively the environment vars LB_CONTROLFILE* or LB_DB_CONTROL_TABLE), to specify the place of the control data with the configured bridges. --control can be either a local control file or a remote control file on s3.

• with local control file

livebridge --control=/path/to/control.yaml

• with remote control file on S3

livebridge --control=s3://bucketname/control.yaml

• with control data stored in a database

... LB_DB_CONTROL_TABLE=lb_control livebridge

See http://livebridge.readthedocs.io/en/stable/control.html for more details.

Documentation

http://livebridge.readthedocs.io/en/stable/

Plugins

Several source and targets are available as plugins. Following plugins are currently available:

• Liveblog - Provides a Liveblog liveticker as source.
• Scribblelive - Provides a Scribblelive event stream as target. Provides also a converter from Liveblog to Scribblelive.
• Slack - Provides Slack channels as source and as target. Also provides converters from Liveblog and to Scribblelive.
• Tickaroo - Provides a Tickaroo ticker as target.

It's possbile to write own service plugins and to make them available to livebridge as a Python module via PyPI. See https://github.com/dpa-newslab/livebridge-slack as an example plugin.

Tutorial: How to build a Livebridge plugin

Developing

See http://livebridge.readthedocs.io/en/stable/developing.html

Testing

Livebridge uses py.test and asynctest for testing.

Run tests:

    py.test -v tests/

Run tests with test coverage:

    py.test -v --cov=livebridge --cov-report=html tests/

pytest-cov has to be installed. In the example above, a html summary of the test coverage is saved in ./htmlcov/.

License

Copyright 2016-2018 dpa-infocom GmbH

Apache License, Version 2.0 - see LICENSE for details