Skip to main content

A collection of toolkit lib for distributed system development in python

Project description

Table of Content

Name

pykit: A collection of python libs those are used in project s2: Storage-Service at baishancloud.com

Status

This library is in beta phase.

It has been used heavily in our object storage service, as a foundamental library of our devops platform.

Description

Module List

There is a README.md for each module.

name description
argchecker Validates arguments against the schema
aws AWS error codes and so on
cacheable Cache data which access frequently
cachepool Reusable object cache in process
cluster Some helper function for the server in a cluster
daemonize Start, stop or restart a daemon process
dictutil Dictionary helper utility
etcd etcd client
fsutil File-system Utilities
heap Min heap
http HTTP/1.1 client
humannum Convert number to human readable number string
jobq Process serial of input elements with several functions concurrently and sequentially
logutil Utility functions to create logger or make log message
mime Utility functions to handle mime type
modutil Submodule Utilities
mysqlconnpool Mysql connection pool with MySQLdb in python
mysqlutil Mysql related datatype, operations
net Network utility
portlock cross process lock
priorityqueue Priority queue
proc Utility to create sub process
rangeset Segmented range.
ratelimiter rate limiter
redisutil For using redis more easily.
shell Set different command arguments to execute different functions
strutil A collection of helper functions used to manipulate string
threadutil Utility functions for better management of threads
timeutil Support specify time format output and get current ts, ms, us api etc
utfjson Force json.dump and json.load in utf-8 encoding
utfyaml Force yaml.dump and yaml.load in utf-8 encoding
wsjobd Job daemon based on websocket protocol
zkutil Utility functions for zookeeper

Module dependency

Install

Just clone it and copy it into your project source folder.

cd your_project_folder
git clone https://github.com/baishancloud/pykit.git

Usage

from pykit import jobq

def add1(args):
    return args + 1

def printarg(args):
    print args

jobq.run([0, 1, 2], [add1, printarg])
# > 1
# > 2
# > 3

Configuration

See config.md

Test

Run one of following to test all, a module, a TestCase or a function.

./script/t
./script/t zkutil
./script/t zkutil.test
./script/t zkutil.test.test_zkutil
./script/t zkutil.test.test_zkutil.TestZKUtil
./script/t zkutil.test.test_zkutil.TestZKUtil.test_lock_id

See Details

For developer

There are several scripts for developers. See script.

Author

Zhang Yanpo (张炎泼) drdr.xp@gmail.com

Copyright and License

The MIT License (MIT)

Copyright (c) 2015 Zhang Yanpo (张炎泼) drdr.xp@gmail.com

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pykit-bsc-0.0.5.tar.gz (219.9 kB view details)

Uploaded Source

Built Distribution

pykit_bsc-0.0.5-py2-none-any.whl (300.1 kB view details)

Uploaded Python 2

File details

Details for the file pykit-bsc-0.0.5.tar.gz.

File metadata

  • Download URL: pykit-bsc-0.0.5.tar.gz
  • Upload date:
  • Size: 219.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for pykit-bsc-0.0.5.tar.gz
Algorithm Hash digest
SHA256 99a12b88f4c8c573e8eddd4702fc49277e4349bebd44cfe67ece98235a3d3776
MD5 7df9abf53c5fa4b3c3f7491153ecc7eb
BLAKE2b-256 26be4f9e402bfa3bbf44440f0a7ccac2c1b915ce3bd6364929e78e286479c61f

See more details on using hashes here.

File details

Details for the file pykit_bsc-0.0.5-py2-none-any.whl.

File metadata

File hashes

Hashes for pykit_bsc-0.0.5-py2-none-any.whl
Algorithm Hash digest
SHA256 cf4bd25beb4fa130be3c76eca89f3b4cb6c78965808b7b4ae9e7064195deaf3f
MD5 4deb464a104b1b32457b960d7e4e2d95
BLAKE2b-256 a4a6345a432fc5a971157b03a4a03dce3a98c43501ebfb6e167f17d404cece4d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page