Skip to main content

qdb async client.

Project description

# os-dbnetget

[![Build Status](](
[![PyPI - Python Version](](

Library and command line tool not just for qdb.

This project implement a python qdb toolkit as dbnetget at first. With the progress of development, it is separated into multiple sub-projects as [os-qdb-protocal](, [os-m3-engine](, this project become a framework for similar work not just for qdb.

The main components are client libraries and command line tool.

Client is used for network processing(connect, close, send, receive, etc.) . It is protocol-independent and transparent to user. By now, there are sync/tornado-async clients, thread-safe client pool and tornado-async client pool are also convenient.

Command line tool's features can be extended by installing extra packages. See [Install](#install).

Greatly appreciate [**Bear Tian**]( and his dbnetget!

# Install

* install package
`pip install os-dbnetget`

* install extra packages

| subpackage | install command | enables |
| :--------- | :-------------- | :------ |
|m3 | ``pip install os-dbnetget[m3]`` | Install [m3]( for command line tool support m3(multi-thread) engine |
| tornado | ``pip install os-dbnetget[tornado]`` | Install [Tornado]( for async client and command line tool support tornado engine |
| rotate | ``pip install os-dbnetget[rotate]`` | Enable write data to rotate file |

# Client API

There are sync/async clients, generally speaking, you should not use them directly. The pool may be your first choice.

## SyncClientPool

* native multi-thread, do not need extra packages
* thread safe
* retry, when network error
* support multi connection with one endpoint


from os_qdb_protocal import create_protocal
from os_dbnetget.clients.sync_client import SyncClientPool

endpoints = ['host%02d:8012' % i for i in range(1, 10)]
pool = SyncClientPool(endpoints)

proto = create_protocal('test', b'test-key')
result = pool.execute(proto)


## TornadoClientPool

* support tornado async
* retry, when network error
* support multi connection with one endpoint


from tornado import gen
from tornado.ioLoop import IOLoop

from os_qdb_protocal import create_protocal
from os_dbnetget.clients.tonado_client import TornadoClientPool

def main():

endpoints = ['host%02d:8012' % i for i in range(1, 10)]
pool = TornadoClientPool(endpoints)

proto = create_protocal('test', b'test-key')
result = yield pool.execute(proto)

yield pool.close()


# Command line

* command line tool is on progress, the support sub-commands:

os-dbnetget -h

* each sub-command has is own features

os-dbnetget [sub-command] -h

* some extra packages can be installed for enhancement, see [Install](#install)

for example, you can install m3 engine to improve processing capacity

pip install os-dbnetget[m3]

cat data.txt | os-dbnetget test --engine m3 --thread-num 100 -L endpoints.lst

# Unit Tests

`$ tox`

# License

MIT licensed.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for os-dbnetget, version 0.1
Filename, size File type Python version Upload date Hashes
Filename, size os-dbnetget-0.1.tar.gz (14.4 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page