Skip to main content

A collection of toolkit lib for distributed system development in python

Project description

Table of Content


pykit: A collection of python libs those are used in project s2: Storage-Service at


This library is in beta phase.

It has been used heavily in our object storage service, as a foundamental library of our devops platform.


Module List

There is a for each module.

name description
argchecker Validates arguments against the schema
aws AWS error codes and so on
cacheable Cache data which access frequently
cachepool Reusable object cache in process
cluster Some helper function for the server in a cluster
daemonize Start, stop or restart a daemon process
dictutil Dictionary helper utility
etcd etcd client
fsutil File-system Utilities
heap Min heap
http HTTP/1.1 client
humannum Convert number to human readable number string
jobq Process serial of input elements with several functions concurrently and sequentially
logutil Utility functions to create logger or make log message
mime Utility functions to handle mime type
modutil Submodule Utilities
mysqlconnpool Mysql connection pool with MySQLdb in python
mysqlutil Mysql related datatype, operations
net Network utility
portlock cross process lock
priorityqueue Priority queue
proc Utility to create sub process
rangeset Segmented range.
ratelimiter rate limiter
redisutil For using redis more easily.
shell Set different command arguments to execute different functions
strutil A collection of helper functions used to manipulate string
threadutil Utility functions for better management of threads
timeutil Support specify time format output and get current ts, ms, us api etc
utfjson Force json.dump and json.load in utf-8 encoding
utfyaml Force yaml.dump and yaml.load in utf-8 encoding
wsjobd Job daemon based on websocket protocol
zkutil Utility functions for zookeeper

Module dependency


Just clone it and copy it into your project source folder.

cd your_project_folder
git clone


from pykit import jobq

def add1(args):
    return args + 1

def printarg(args):
    print args[0, 1, 2], [add1, printarg])
# > 1
# > 2
# > 3




Run one of following to test all, a module, a TestCase or a function.

./script/t zkutil
./script/t zkutil.test
./script/t zkutil.test.test_zkutil
./script/t zkutil.test.test_zkutil.TestZKUtil
./script/t zkutil.test.test_zkutil.TestZKUtil.test_lock_id

See Details

For developer

There are several scripts for developers. See script.


Zhang Yanpo (张炎泼)

Copyright and License

The MIT License (MIT)

Copyright (c) 2015 Zhang Yanpo (张炎泼)

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pykit-bsc-0.0.5.tar.gz (219.9 kB view hashes)

Uploaded Source

Built Distribution

pykit_bsc-0.0.5-py2-none-any.whl (300.1 kB view hashes)

Uploaded Python 2

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page