A gevent based python client for NSQ distributed messaging platform.
- Free software: BSD license
- Documentation: https://gnsq.readthedocs.org
- Battle tested on billions and billions of messages </sagan>
- Based on gevent for fast concurrent networking
- Fast and flexible signals with Blinker
- Automatic nsqlookupd discovery and back-off
- Support for TLS, DEFLATE, and Snappy
- Full HTTP clients for both nsqd and nsqlookupd
At the command line:
$ easy_install gnsq
Or even better, if you have virtualenvwrapper installed:
$ mkvirtualenv gnsq
$ pip install gnsq
Currently there is support for Python 2.6+, Python 3.3+ and PyPy.
First make sure nsq is installed and running. Next create a nsqd connection
and publish some messages to your topic:
conn = gnsq.Nsqd(address='localhost', http_port=4151)
conn.publish('topic', 'hello gevent!')
conn.publish('topic', 'hello nsq!')
Then create a Reader to consume messages from your topic:
reader = gnsq.Reader('topic', 'channel', 'localhost:4150')
def handler(reader, message):
print 'got message:', message.body
Optional snappy support depends on the python-snappy package which in turn
depends on libsnappy:
$ sudo apt-get install libsnappy-dev
# Or OS X
$ brew install snappy
# And then install python-snappy
$ pip install python-snappy
Feedback, issues, and contributions are always gratefully welcomed. See the
contributing guide for details on how to help and setup a development
- #13 - Allow use with nsq v1.0.0 (thanks @daroot)
- Add contrib package with utlities.
- #11 - Make sure all socket data is sent.
- #5 - Add support for DPUB (defered publish).
- Add support for Python 3 and PyPy.
- #7 - Fix undeclared variable in compression socket.
- Fix negative in flight causing not throtteling after backoff.
- Fix extra backoff success/failures during backoff period.
- Fix case where handle_backoff is never called.
- Add backoff parameter to message.requeue().
- Allow overriding backoff on NSQRequeueMessage error.
- Handle connection failures while starting/completing backoff.
- Remove disconnected nsqd messages from the worker queue.
- #4 - Fix crash in Reader.random_ready_conn (thanks @ianpreston).
- Allow finishing and requeuing in sync handlers.
- Topics and channels are now valid to 64 characters.
- Ephemeral topics are now valid.
- Adjustable backoff behaviour.
- Warn on connection failure.
- Add extra requires for snappy.
- Add support for nsq auth protocal.
- Preemptively update ready count.
- Dependency and contributing documentation.
- Support for nsq back to 0.2.24.
- Block as expected on start, even if already started.
- Raise runtime error if starting the reader without a message handler.
- Add on_close signal to the reader.
- Allow upgrading to tls+snappy or tls+deflate.
- Flush delfate buffer for each message.
- Fix packaging stream submodule.
- Send queued messages before closing socket.
- Continue to read from socket on EAGAIN
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.