A collection of toolkit lib for distributed system development in python
Project description
Table of Content
Name
pykit: A collection of python libs those are used in project s2: Storage-Service at baishancloud.com
Status
This library is in beta phase.
It has been used heavily in our object storage service, as a foundamental library of our devops platform.
Description
Module List
There is a README.md
for each module.
name | description |
---|---|
argchecker | Validates arguments against the schema |
aws | AWS error codes and so on |
cacheable | Cache data which access frequently |
cachepool | Reusable object cache in process |
cluster | Some helper function for the server in a cluster |
daemonize | Start, stop or restart a daemon process |
dictutil | Dictionary helper utility |
etcd | etcd client |
fsutil | File-system Utilities |
heap | Min heap |
http | HTTP/1.1 client |
humannum | Convert number to human readable number string |
jobq | Process serial of input elements with several functions concurrently and sequentially |
logutil | Utility functions to create logger or make log message |
mime | Utility functions to handle mime type |
modutil | Submodule Utilities |
mysqlconnpool | Mysql connection pool with MySQLdb in python |
mysqlutil | Mysql related datatype, operations |
net | Network utility |
portlock | cross process lock |
priorityqueue | Priority queue |
proc | Utility to create sub process |
rangeset | Segmented range. |
ratelimiter | rate limiter |
redisutil | For using redis more easily. |
shell | Set different command arguments to execute different functions |
strutil | A collection of helper functions used to manipulate string |
threadutil | Utility functions for better management of threads |
timeutil | Support specify time format output and get current ts, ms, us api etc |
utfjson | Force json.dump and json.load in utf-8 encoding |
utfyaml | Force yaml.dump and yaml.load in utf-8 encoding |
wsjobd | Job daemon based on websocket protocol |
zkutil | Utility functions for zookeeper |
Module dependency
Install
Just clone it and copy it into your project source folder.
cd your_project_folder
git clone https://github.com/baishancloud/pykit.git
Usage
from pykit import jobq
def add1(args):
return args + 1
def printarg(args):
print args
jobq.run([0, 1, 2], [add1, printarg])
# > 1
# > 2
# > 3
Configuration
See config.md
Test
Run one of following to test all, a module, a TestCase or a function.
./script/t
./script/t zkutil
./script/t zkutil.test
./script/t zkutil.test.test_zkutil
./script/t zkutil.test.test_zkutil.TestZKUtil
./script/t zkutil.test.test_zkutil.TestZKUtil.test_lock_id
See Details
For developer
There are several scripts for developers. See script.
Author
Zhang Yanpo (张炎泼) drdr.xp@gmail.com
Copyright and License
The MIT License (MIT)
Copyright (c) 2015 Zhang Yanpo (张炎泼) drdr.xp@gmail.com
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pykit_bsc-0.0.5-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cf4bd25beb4fa130be3c76eca89f3b4cb6c78965808b7b4ae9e7064195deaf3f |
|
MD5 | 4deb464a104b1b32457b960d7e4e2d95 |
|
BLAKE2b-256 | a4a6345a432fc5a971157b03a4a03dce3a98c43501ebfb6e167f17d404cece4d |