Skip to main content

The tool that keeps all your mongos in one place

Project description

All your mongos in one place


MongoPool is the tool that manages your connections to different clusters, maps databases to clients and allows you to work only with database names without worrying about creating and managing connections.

You will never have to create a MongoClient everywhere you want to access a database again which enables you to write beautiful and maintainable code. Using MongoPool, you will keep connection regarding information in a single place and allows you to easily modify it when needed.

At UberVU, we are confidently using it to manage over 25 mongo instances to provide quality services to our customers. ## Install



$ sudo pip install mongopool


$ git clone
$ cd mongopool
$ sudo python install


Basic example

All you have to do in order to get started is to build a list of dictionaries which contains the necessary information to connect to the clusters, instantiate MongoPool and access databases through dot notation.

>>> config = [{'cluster1': {'host': '', 'port': 27017, 'dbpath':'blogs'}},
...           {'cluster2': {'host': '', 'port': 27018, 'dbpath': 'posts'}}]
>>> mongopool = MongoPool(config)
>>> mongopool.blogs
Database(MongoClient('', 27017), u'blogs')
>>> mongopool.posts
Database(MongoClient('', 27018), u'posts')

Multiple databases on the same cluster

But what if you want to work with multiple databases on the same cluster? You can specify the dbpath as an array containing the database names as in the following example:

>>> config = [{'cluster1': {'host': '', 'port': 27017, 'dbpath': ['blogs', 'posts']}}]
>>> mongopool = MongoPool(config)
>>> mongopool.blogs
Database(MongoClient('', 27017), u'blogs')
>>> mongopool.posts
Database(MongoClient('', 27017), u'posts')``

Dynamic paths

You might have databases created automatically, following a certain naming pattern. In this case, it would be impossible to add all databases on a cluster in dbpath. For this reason, you can pass it as a regexp pattern. Let’s say that you save the comments in a separate database for each month, named comments_monthyear:

>>> config = [{'cluster1': {'host': '', 'port': 27017, 'dbpath': 'comments_\d*'}}]
>>> mongopool = MongoPool(config)
>>> mongopool.comments_012014
Database(MongoClient('', 27017), u'comments_012014')
>>> mongopool.comments_032014
Database(MongoClient('', 27017), u'comments_032014')``

Caution: This is a strong feature, but it should be used carefully. Dbpaths will be matched in the order you put them in the configurations list, so make sure you order them from the most particular to the most general in order to avoid creating incorrect mappings and connect to the wrong cluster.


config = [{'cluster1': {'host': '', 'port': 27017, 'dbpath': '.*'}},
          {'cluster2': {'host': '', 'port': 27017, 'dbpath': ['blogs', 'comments'}}]


config = [{'cluster1': {'host': '', 'port': 27017, 'dbpath': ['blogs', 'comments'}},
          {'cluster2': {'host': '', 'port': 27017, 'dbpath': '.*'}}]

Connecting to a replicaSet

MongoPool also manages connections to ReplicaSets. All you have to do is to add the name of the replica set in the configuration. Also, if you want a read_preference different from PRIMARY, you can specify it in the config.

>>> config = [{'cluster1': {'host': '', 'port': 27018, 'replicaSet': 'rset0',
...'read_preference': 'secondary','dbpath': 'blogs'}}]
>>> mongopool = MongoPool(config)
>>> mongopool.blogs    Database(MongoReplicaSetClient([u'', u'', u'']), u'blogs')

Setting a timeout

By default, MongoClient and MongoReplicaSetClient do not have a timeout set, though sometimes it is handy. To set a timeout for you connection you can either pass it as a second argument while instantiating MongoPool or use the set_timeout method which will recreate all connections with the new timeout and create all new connections with the new value.

mongopool = MongoPool(config, network_timeout=2)

Custom connection classes support

If you want to use your custom connection classes instead of MongoClient and MongoReplicaSetClient, you can do this by passing 2 optional arguments: connection_class and rset_connection_class.

mongopool = MongoPool(config, connection_class=MyClass, rset_connection_class=MyOther(Class)

Setting it up

Along with the project we provide a sample config file to easily get started. In order to work with it, you have to launch multiple mongod instances on different ports. For this purpose, you can run the script. If you don’t wish to open many mongod instances, you can change all port values in the config file to 27017 and delete label3 entry which uses a replicaSet.

# make sure that you are in the mongopool main directory
$ cd mongopool
# run the provided script or modify sample_config.yml file
$ ./ $ python

And then run the following commands:

python import os import yaml
from mongopool import MongoPool

filename = os.path.join(os.getcwd(), 'sample\_config.yml')
options = yaml.load(open(filename))
config = options['mongopool']
pool = MongoPool(config)

Now you should have a working mongopool instance which you can play with. When you are done, run:

$ ./clean\

This will ensure that all created databases are deleted and all mongod instances are shutdown.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mongopool-0.2.tar.gz (11.7 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page