gnsq 0.4.0

A gevent based python client for NSQ.

A gevent based python client for NSQ distributed messaging platform.

Features include:

• Free software: BSD license

• Documentation: https://gnsq.readthedocs.org

• Battle tested on billions and billions of messages </sagan>

• Based on gevent for fast concurrent networking

• Fast and flexible signals with Blinker

• Automatic nsqlookupd discovery and back-off

• Support for TLS, DEFLATE, and Snappy

• Full HTTP clients for both nsqd and nsqlookupd

Installation

At the command line:

$ easy_install gnsq

Or even better, if you have virtualenvwrapper installed:

$ mkvirtualenv gnsq
$ pip install gnsq

Currently there is support for Python 2.6+, Python 3.3+ and PyPy.

Usage

First make sure nsq is installed and running. Next create a nsqd connection and publish some messages to your topic:

import gnsq
conn = gnsq.Nsqd(address='localhost', http_port=4151)

conn.publish('topic', 'hello gevent!')
conn.publish('topic', 'hello nsq!')

Then create a Reader to consume messages from your topic:

reader = gnsq.Reader('topic', 'channel', 'localhost:4150')

@reader.on_message.connect
def handler(reader, message):
    print 'got message:', message.body

reader.start()

Dependencies

Optional snappy support depends on the python-snappy package which in turn depends on libsnappy:

# Debian
$ sudo apt-get install libsnappy-dev

# Or OS X
$ brew install snappy

# And then install python-snappy
$ pip install python-snappy

Contributing

Feedback, issues, and contributions are always gratefully welcomed. See the contributing guide for details on how to help and setup a development environment.

History

0.4.0 (2017-06-13)

• #13 - Allow use with nsq v1.0.0 (thanks @daroot)

• Add contrib package with utlities.

0.3.3 (2016-09-25)

• #11 - Make sure all socket data is sent.

• #5 - Add support for DPUB (defered publish).

0.3.2 (2016-04-10)

• Add support for Python 3 and PyPy.

• #7 - Fix undeclared variable in compression socket.

0.3.1 (2015-11-06)

• Fix negative in flight causing not throtteling after backoff.

0.3.0 (2015-06-14)

• Fix extra backoff success/failures during backoff period.

• Fix case where handle_backoff is never called.

• Add backoff parameter to message.requeue().

• Allow overriding backoff on NSQRequeueMessage error.

• Handle connection failures while starting/completing backoff.

0.2.3 (2015-02-16)

• Remove disconnected nsqd messages from the worker queue.

• #4 - Fix crash in Reader.random_ready_conn (thanks @ianpreston).

0.2.2 (2015-01-12)

• Allow finishing and requeuing in sync handlers.

0.2.1 (2015-01-12)

• Topics and channels are now valid to 64 characters.

• Ephemeral topics are now valid.

• Adjustable backoff behaviour.

0.2.0 (2014-08-03)

• Warn on connection failure.

• Add extra requires for snappy.

• Add support for nsq auth protocal.

0.1.4 (2014-07-24)

• Preemptively update ready count.

• Dependency and contributing documentation.

• Support for nsq back to 0.2.24.

0.1.3 (2014-07-08)

• Block as expected on start, even if already started.

• Raise runtime error if starting the reader without a message handler.

• Add on_close signal to the reader.

• Allow upgrading to tls+snappy or tls+deflate.

0.1.2 (2014-07-08)

• Flush delfate buffer for each message.

0.1.1 (2014-07-07)

• Fix packaging stream submodule.

• Send queued messages before closing socket.

• Continue to read from socket on EAGAIN

0.1.0 (2014-07-07)

• First release on PyPI.