pykit-bsc-s2 1.0.1

A collection of toolkit lib for distributed system development in python

Table of Content

• Name
• Status
• Description
  • Module List
• Install
• Usage
• Configuration
  • Supported config
• Test
• For developer
  • Config
    • How it works
    • How to use
• Author
• Copyright and License

Name

pykit: A collection of python libs those are used in project s2: Storage-Service at baishancloud.com

Status

This library is in beta phase.

It has been used heavily in our object storage service, as a foundamental library of our devops platform.

Description

Module List

There is a README.md for each module.

name	description
argchecker	Validates arguments against the schema
aws	AWS error codes and so on
awssign	Sign a request by using aws signature version 4
cacheable	Cache data which access frequently
cachepool	Reusable object cache in process
cgrouparch	Build cgroup directory tree, add set cgroup pid
cluster	Some helper function for the server in a cluster
csvutil	Utility functions for CSV file loading and conversion
daemonize	Start, stop or restart a daemon process
dictutil	Dictionary helper utility
ectypes	Utility functions for the server
etcd	etcd client
fsutil	File-system Utilities
heap	Min heap
http	HTTP/1.1 client
humannum	Convert number to human readable number string
jobq	Process serial of input elements with several functions concurrently and sequentially
jobscheduler	Run jobs at certain time
logcollector	Collect logs of interest locally
logutil	Utility functions to create logger or make log message
mime	Utility functions to handle mime type
modutil	Submodule Utilities
mysqlconnpool	Mysql connection pool with MySQLdb in python
mysqlutil	Mysql related datatype, operations
net	Network utility
portlock	cross process lock
priorityqueue	Priority queue
proc	Utility to create sub process
rangeset	Segmented range.
ratelimiter	rate limiter
redisutil	For using redis more easily.
shell	Set different command arguments to execute different functions
strutil	A collection of helper functions used to manipulate string
threadutil	Utility functions for better management of threads
timeutil	Support specify time format output and get current ts, ms, us api etc
utfjson	Force json.dump and json.load in utf-8 encoding
utfyaml	Force yaml.dump and yaml.load in utf-8 encoding
wsjobd	Job daemon based on websocket protocol
zktx	Transaction implementation on Zookeeper
zkutil	Utility functions for zookeeper

Module dependency

Install

Just clone it and copy it into your project source folder.

cd your_project_folder
git clone https://github.com/baishancloud/pykit.git

Usage

from pykit import jobq

def add1(args):
    return args + 1

def printarg(args):
    print args

jobq.run([0, 1, 2], [add1, printarg])
# > 1
# > 2
# > 3

Configuration

See config.md

Test

Run one of following to test all, a module, a TestCase or a function.

./script/t
./script/t zkutil
./script/t zkutil.test
./script/t zkutil.test.test_zkutil
./script/t zkutil.test.test_zkutil.TestZKUtil
./script/t zkutil.test.test_zkutil.TestZKUtil.test_lock_id

See Details

For developer

There are several scripts for developers. See script.

Author

Zhang Yanpo (张炎泼) [e-mail]

Copyright and License

The MIT License (MIT)

Copyright (c) 2015 Zhang Yanpo (张炎泼) [e-mail]