Skip to main content

Adapter to write documents from a queue to mongodb concurrently

Project description

MongoWriter provides a simple interface to write documents to a mongodb replica set. The interface is an object which behaves like a queue, MongoWriter being the consumer. It tries to handle mongo exceptions (like primary change) gracefully, only dropping documents if the input queue gets full. This is to make a best effort to insert the latest documents in the face of overload or errors but can be disabled by using an unlimited queue.

Gevent

By default MongoWriter uses Threads for its insert workers. Using monkey patches, it works with greenlets as well.

Input Queue

The input object passed to MongoWriter can be anything that acts like a Python queue. The items in the queue are expected to be (collection_name, document) tuples. There is a queue adaptation in mongowriter.util which can be useful for high troughput writing to capped collections.

Project details


Release history Release notifications

History Node

0.1.7

History Node

0.1.6

History Node

0.1.5

This version
History Node

0.1.3

History Node

0.1.2

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
mongowriter-0.1.3.tar.gz (2.5 kB) Copy SHA256 hash SHA256 Source None May 28, 2014

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page