Python client library for the Kyoto Tycoon key-value store
This is a native Python client library for the Kyoto Tycoon server, supporting both Python 2 and 3. It includes significant performance improvements and bug fixes over the (now seemingly unmaintained) original library by Toru Maesaka and Stephen Hamer.
For more information on Kyoto Tycoon server, please refer to:
Along with this library we’re also maintaining an updated and ready-to-go fork of Kyoto Tycoon here:
The more efficient binary protocol is also supported along with the HTTP protocol. It provides a performance improvement of up to 6x, but only the following operations are available:
- get() and get_bulk()
- set() and set_bulk()
- remove() and remove_bulk()
Atomic operations aren’t supported with the binary protocol, the use of “atomic=False” is mandatory when using it. Operations besides these will raise a NotImplementedError exception.
It’s possible to have two KyotoTycoon objects open to the same server in the same application, one using HTTP and the other using the binary protocol, if necessary.
The library does automatic packing and unpacking (marshalling) of values coming from/to the database. The following data storage formats are available by default:
- KT_PACKER_PICKLE - Python “pickle” format.
- KT_PACKER_JSON - JSON format (compact representation).
- KT_PACKER_STRING - Strings (UTF-8).
- KT_PACKER_BYTES - Binary data.
There is also a KT_PACKER_CUSTOM format available where you can specify your own object to do the marshalling. This object needs to provide the following two methods:
- .pack(self, data) - convert “data” to bytes()
- .unpack(self, data) - convert “data” from bytes()
Marshalling is done for all methods except play_script(), because the server can return data in more than one format at once. The caller will most likely know the type of data that the called script returns and must do the marshalling itself.
Since version 0.7.0 this library also contains a replication slave class. This class provides a consume() generator function that receives and parses transaction log entries from a Kyoto Tycoon master server into dictionaries.
This can be used to build your own custom replication schemes, like key filtering, or react to server “set”, “remove” or “clear” database operations.
Do note that only explicit operations create transaction log events. Implicit operations like key removal upon expiration don’t.
Unlike the client library, the replication slave always handles the “key” and “value” attributes as opaque binary data.
Kyoto Tycoon supports a subset of the memcached protocol. When a server has this enabled including item flags, these are stored as the last 4 bytes of the value. Since version 0.7.3 of this library, there’s a custom packer included that transparently handles this and also includes gzip compression/decompression for scenarios where, for example, a Python application is writing HTML pages to the Kyoto server and an HTTP server is reading from it using a memcached client library.
from kyototycoon import KyotoTycoon, KT_PACKER_CUSTOM from kyototycoon.packers import MemcachePacker mp = MemcachePacker(gzip_enabled=True, gzip_flag=1) kt = KyotoTycoon(pack_type=KT_PACKER_CUSTOM, custom_packer=mp) kt.open("127.0.0.1", 1978) kt.set("key", "value") value = kt.get("key") kt.close()
To handle (value, flags) pairs without any additional processing, the SimpleMemcachePacker can be used:
from kyototycoon import KyotoTycoon, KT_PACKER_CUSTOM from kyototycoon.packers import SimpleMemcachePacker smp = SimpleMemcachePacker() kt = KyotoTycoon(pack_type=KT_PACKER_CUSTOM, custom_packer=smp) kt.open("127.0.0.1", 1978) kt.set("key", ("value", 123)) value, flags = kt.get("key") kt.close()
This library is still not at version 1.0, which means the API and behavior are not guaranteed to remain consistent between versions.
Support for using an error object has been removed. If you need the old behavior for compatibility reasons, use a version up to (and including) v0.5.9. Versions later than this raise exceptions on all Kyoto Tycoon errors.
You can install the latest version of this library from source:
python setup.py build python setup.py install
Or, you can install the latest stable release from PyPI:
pip install python-kyototycoon-ng