A collection of toolkit lib for distributed system development in python
Project description
Table of Content
Name
pykit: A collection of python libs those are used in project s2: Storage-Service at baishancloud.com
Status
This library is in beta phase.
It has been used heavily in our object storage service, as a foundamental library of our devops platform.
Description
Module List
There is a README.md
for each module.
name | description |
---|---|
argchecker | Validates arguments against the schema |
aws | AWS error codes and so on |
cacheable | Cache data which access frequently |
cachepool | Reusable object cache in process |
cluster | Some helper function for the server in a cluster |
daemonize | Start, stop or restart a daemon process |
dictutil | Dictionary helper utility |
etcd | etcd client |
fsutil | File-system Utilities |
heap | Min heap |
http | HTTP/1.1 client |
humannum | Convert number to human readable number string |
jobq | Process serial of input elements with several functions concurrently and sequentially |
logutil | Utility functions to create logger or make log message |
mime | Utility functions to handle mime type |
modutil | Submodule Utilities |
mysqlconnpool | Mysql connection pool with MySQLdb in python |
mysqlutil | Mysql related datatype, operations |
net | Network utility |
portlock | cross process lock |
priorityqueue | Priority queue |
proc | Utility to create sub process |
rangeset | Segmented range. |
ratelimiter | rate limiter |
redisutil | For using redis more easily. |
shell | Set different command arguments to execute different functions |
strutil | A collection of helper functions used to manipulate string |
threadutil | Utility functions for better management of threads |
timeutil | Support specify time format output and get current ts, ms, us api etc |
utfjson | Force json.dump and json.load in utf-8 encoding |
utfyaml | Force yaml.dump and yaml.load in utf-8 encoding |
wsjobd | Job daemon based on websocket protocol |
zkutil | Utility functions for zookeeper |
Module dependency
Install
Just clone it and copy it into your project source folder.
cd your_project_folder
git clone https://github.com/baishancloud/pykit.git
Usage
from pykit import jobq
def add1(args):
return args + 1
def printarg(args):
print args
jobq.run([0, 1, 2], [add1, printarg])
# > 1
# > 2
# > 3
Configuration
See config.md
Test
Run one of following to test all, a module, a TestCase or a function.
./script/t
./script/t zkutil
./script/t zkutil.test
./script/t zkutil.test.test_zkutil
./script/t zkutil.test.test_zkutil.TestZKUtil
./script/t zkutil.test.test_zkutil.TestZKUtil.test_lock_id
See Details
For developer
There are several scripts for developers. See script.
Author
Zhang Yanpo (张炎泼) drdr.xp@gmail.com
Copyright and License
The MIT License (MIT)
Copyright (c) 2015 Zhang Yanpo (张炎泼) drdr.xp@gmail.com
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file pykit-bsc-0.0.5.tar.gz
.
File metadata
- Download URL: pykit-bsc-0.0.5.tar.gz
- Upload date:
- Size: 219.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
99a12b88f4c8c573e8eddd4702fc49277e4349bebd44cfe67ece98235a3d3776
|
|
MD5 |
7df9abf53c5fa4b3c3f7491153ecc7eb
|
|
BLAKE2b-256 |
26be4f9e402bfa3bbf44440f0a7ccac2c1b915ce3bd6364929e78e286479c61f
|
File details
Details for the file pykit_bsc-0.0.5-py2-none-any.whl
.
File metadata
- Download URL: pykit_bsc-0.0.5-py2-none-any.whl
- Upload date:
- Size: 300.1 kB
- Tags: Python 2
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
cf4bd25beb4fa130be3c76eca89f3b4cb6c78965808b7b4ae9e7064195deaf3f
|
|
MD5 |
4deb464a104b1b32457b960d7e4e2d95
|
|
BLAKE2b-256 |
a4a6345a432fc5a971157b03a4a03dce3a98c43501ebfb6e167f17d404cece4d
|