Skip to main content

mysql binlog to blinker signal

Project description

Python MySQL Replication Blinker
================================

Features
--------

This package uses
`mysql-replication <https://github.com/noplay/python-mysql-replication>`__
to read events from MySQL's binlog and send to blinker's signal.

- binlog action level
- schema level
- table level
- row level

It will send RowsEvent only.

Whenever a binlog event come, it will be dispatched into some signals:

#. binlog\_signal: 1 signal for the binlog event.
#. schema\_signal: 1 signal for the event's schema
#. table\_signal: 1 signal for the event's table.
#. row\_signal: 1+ signals for event's rows. 1 signal per row.

Signals
-------

.. code-block::

binlog event -> binlog signal -> schema signal -> table signal --> row signals

So, suppose that an event come with schema=foo, table=bar and it updated
2 rows. Those signal will be sent:

+-----------------+--------------------------+------------------------------+
| signal | signal name | sender |
+=================+==========================+==============================+
| binlog signal | ``update`` | event (RowsEvent) |
+-----------------+--------------------------+------------------------------+
| schema signal | ``update@foo`` | event (RowsEvent) |
+-----------------+--------------------------+------------------------------+
| table signal | ``update@foo.bar`` | event.rows (list of array) |
+-----------------+--------------------------+------------------------------+
| row signal | ``update@foo.bar#row`` | row1 (array) |
+-----------------+--------------------------+------------------------------+
| row signal | ``update@foo.bar#row`` | row2 object (array) |
+-----------------+--------------------------+------------------------------+

Connect to signals
------------------

To connect to a signal, you can use the signal instance or a decorator.

Suppose that you need to connect to write signal on table
*db0.table1*:

.. code-block:: python

from pymysqlblinker import signals

tbl1_signal = signals.table_write('db0', 'table1')

def subscriber1(rows, schema, table):
pass

# use connect function
tbl1_signal.connect(subscriber1)

# or use decorator
@signals.on_table_write('db0', 'table1')
def subscriber1(rows, schema, table):
pass

Signal publishing
-----------------

To start publishing signals

.. code-block:: python

from pymysqlblinker import start_publishing

start_publishing(
'mysql://root@localhost',
)

Replication
-----------

This package support a method to replicate from mysql database. It
operates by keep memory at last binlog position. By default, it save to a file.

To make it, call:

.. code-block:: python

from pymysqlblinker import start_replication

start_replication(
'mysql://root@localhost',
'/path/to/file/that/remember/binlog/position',
)

Change logs
-----------

1.2
~~~

- Add BinlogPosMemory to allow replication: replication is publishing
with
ability to remember last binlog position. For the first run, it will
start at
the end of current binlog.

1.1.1
~~~~~

- Add connect\_timeout argument to pub.start\_publishing

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
pymysqlblinker-1.2.tar.gz (5.8 kB) Copy SHA256 hash SHA256 Source None Aug 18, 2015

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page