Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

A collection of toolkit lib for distributed system development in python

Project description

Table of Content


pykit: A collection of python libs those are used in project s2: Storage-Service at


This library is in beta phase.

It has been used heavily in our object storage service, as a foundamental library of our devops platform.


Module List

There is a for each module.

name description
argchecker Validates arguments against the schema
aws AWS error codes and so on
awssign Sign a request by using aws signature version 4
cacheable Cache data which access frequently
cachepool Reusable object cache in process
cgrouparch Build cgroup directory tree, add set cgroup pid
cluster Some helper function for the server in a cluster
csvutil Utility functions for CSV file loading and conversion
daemonize Start, stop or restart a daemon process
dictutil Dictionary helper utility
ectypes Utility functions for the server
etcd etcd client
fsutil File-system Utilities
heap Min heap
http HTTP/1.1 client
humannum Convert number to human readable number string
jobq Process serial of input elements with several functions concurrently and sequentially
jobscheduler Run jobs at certain time
logcollector Collect logs of interest locally
logutil Utility functions to create logger or make log message
mime Utility functions to handle mime type
modutil Submodule Utilities
mysqlconnpool Mysql connection pool with MySQLdb in python
mysqlutil Mysql related datatype, operations
net Network utility
portlock cross process lock
priorityqueue Priority queue
proc Utility to create sub process
rangeset Segmented range.
ratelimiter rate limiter
redisutil For using redis more easily.
shell Set different command arguments to execute different functions
strutil A collection of helper functions used to manipulate string
threadutil Utility functions for better management of threads
timeutil Support specify time format output and get current ts, ms, us api etc
utfjson Force json.dump and json.load in utf-8 encoding
utfyaml Force yaml.dump and yaml.load in utf-8 encoding
wsjobd Job daemon based on websocket protocol
zktx Transaction implementation on Zookeeper
zkutil Utility functions for zookeeper

Module dependency


Just clone it and copy it into your project source folder.

cd your_project_folder
git clone


from pykit import jobq

def add1(args):
    return args + 1

def printarg(args):
    print args[0, 1, 2], [add1, printarg])
# > 1
# > 2
# > 3




Run one of following to test all, a module, a TestCase or a function.

./script/t zkutil
./script/t zkutil.test
./script/t zkutil.test.test_zkutil
./script/t zkutil.test.test_zkutil.TestZKUtil
./script/t zkutil.test.test_zkutil.TestZKUtil.test_lock_id

See Details

For developer

There are several scripts for developers. See script.


Zhang Yanpo (张炎泼)

Copyright and License

The MIT License (MIT)

Copyright (c) 2015 Zhang Yanpo (张炎泼)

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pykit-bsc-s2, version 1.0.1
Filename, size File type Python version Upload date Hashes
Filename, size pykit_bsc_s2-1.0.1-py2-none-any.whl (309.6 kB) File type Wheel Python version py2 Upload date Hashes View hashes
Filename, size pykit-bsc-s2-1.0.1.tar.gz (226.6 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page