fastutils 0.43.0

Collection of simple utils.

fastutils

Collection of simple utils.

Install

pip install fastutils

Installed utils

1. fastutils.aesutils
   1. AES_BLOCK_SIZE
   2. decrypt
   3. decrypt_and_base64decode
   4. decrypt_and_safeb64decode
   5. decrypt_and_unhexlify
   6. encrypt
   7. encrypt_and_base64encode
   8. encrypt_and_hexlify
   9. encrypt_and_safeb64encode
   10. get_aes_mode
   11. get_md5_key
   12. get_mysql_aes_key
   13. get_padding_remove_method
   14. get_raw_aes_key
   15. get_sha1prng_key
   16. padding_ansix923
   17. padding_iso10126
   18. padding_pkcs5
   19. remove_padding_ansix923
   20. remove_padding_iso10126
   21. remove_padding_pkcs5
2. fastutils.cacheutils
   1. get_cached_value
3. fastutils.cipherutils
   1. AesCipher
   2. Base64Encoder
   3. CipherBase
   4. DecryptFailed
   5. EncoderBase
   6. HexlifyEncoder
   7. IvCipher
   8. IvfCipher
   9. MappingCipher
   10. MysqlAesCipher
   11. RawDataEncoder
   12. RawKeyAesCipher
   13. RsaCipher
   14. S12Cipher
   15. S1Cipher
   16. S2Cipher
   17. SafeBase64Encoder
   18. Utf8Encoder
   19. aes_padding_ansix923
   20. aes_padding_iso10126
   21. aes_padding_pkcs5
   22. get_aes_mode
   23. md5_key
   24. mysql_aes_key
   25. sha1prng_key
4. fastutils.dictutils
   1. Object
   2. attrgetorset
   3. attrset
   4. change
   5. changes
   6. deep_merge
   7. diff
   8. fix_object
   9. ignore_none_item
   10. prefix_key
   11. select
   12. to_object
   13. touch
   14. update
5. fastutils.fsutils
   1. TemporaryFile
   2. copy
   3. expand
   4. file_content_replace
   5. filecopy
   6. first_exists_file
   7. get_application_config_filepath
   8. get_application_config_paths
   9. get_size_deviation
   10. get_size_display
   11. get_temp_workspace
   12. get_unit_size
   13. info
   14. load_application_config
   15. mkdir
   16. move
   17. pathjoin
   18. readfile
   19. rename
   20. rm
   21. size_unit_names
   22. size_unit_upper_limit
   23. touch
   24. treecopy
   25. write
6. fastutils.funcutils
   1. BunchCallable
   2. call_with_inject
   3. chain
   4. classproperty
   5. get_all_builtin_exceptions
   6. get_builtins_dict
   7. get_class_name
   8. get_default_values
   9. get_inject_params
   10. is_a_class
   11. mcall_with_inject
   12. signature
   13. try_again_on_error
7. fastutils.hashutils
   1. get_file_hash
   2. get_file_md5
   3. get_file_sha
   4. get_file_sha1
   5. get_file_sha224
   6. get_file_sha256
   7. get_file_sha384
   8. get_file_sha512
   9. get_hash_base64
   10. get_hash_hexdigest
   11. get_md5
   12. get_md5_base64
   13. get_md5_hexdigest
   14. get_pbkdf2_hmac
   15. get_pbkdf2_md5
   16. get_pbkdf2_sha
   17. get_pbkdf2_sha1
   18. get_pbkdf2_sha224
   19. get_pbkdf2_sha256
   20. get_pbkdf2_sha384
   21. get_pbkdf2_sha512
   22. get_sha
   23. get_sha1
   24. get_sha1_base64
   25. get_sha1_hexdigest
   26. get_sha224
   27. get_sha224_base64
   28. get_sha224_hexdigest
   29. get_sha256
   30. get_sha256_base64
   31. get_sha256_hexdigest
   32. get_sha384
   33. get_sha384_base64
   34. get_sha384_hexdigest
   35. get_sha512
   36. get_sha512_base64
   37. get_sha512_hexdigest
   38. get_sha_base64
   39. get_sha_hexdigest
   40. method_load
   41. pbkdf2_hmac
   42. validate_pbkdf2_hmac
   43. validate_pbkdf2_md5
   44. validate_pbkdf2_sha
   45. validate_pbkdf2_sha1
   46. validate_pbkdf2_sha224
   47. validate_pbkdf2_sha256
   48. validate_pbkdf2_sha384
   49. validate_pbkdf2_sha512
8. fastutils.httputils
   1. download
   2. get_sitename
   3. get_url_filename
   4. get_url_save_path
   5. get_urlinfo
   6. urlparse
9. fastutils.imageutils
   1. get_base64image
   2. get_image_bytes
   3. parse_base64image
   4. resize
   5. resize_to_fixed_height
   6. resize_to_fixed_width
10. fastutils.jsonutils
    1. SimpleJsonEncoder
    2. make_simple_json_encoder
    3. register_global_encoder
    4. simple_json_dumps
11. fastutils.listutils
    1. append_new
    2. chunk
    3. clean_none
    4. compare
    5. compare_execute
    6. first
    7. group
    8. ignore_none_element
    9. int_list_to_bytes
    10. is_ordered
    11. list2dict
    12. pad
    13. replace
    14. topological_sort
    15. topological_test
    16. unique
12. fastutils.logutils
    1. get_console_handler
    2. get_file_handler
    3. get_simple_config
    4. setup
13. fastutils.msgpackutils
    1. EmptyMsgData
    2. Msg
    3. MsgDecryptFailed
    4. WrongMsgType
    5. dumps
    6. loads
14. fastutils.nameutils
    1. get_last_names
    2. get_random_name
    3. get_suggest_first_names
    4. guess_lastname
    5. guess_surname
15. fastutils.numericutils
    1. _infinity
    2. binary_decompose
    3. decimal_change_base
    4. float_split
    5. from_bytes
    6. get_float_part
    7. infinity
    8. is_infinity
    9. ninfinity
    10. pinfinity
16. fastutils.pinyinutils
    1. to_pinyin
17. fastutils.randomutils
    1. Random
    2. UuidGenerator
    3. choices
    4. uuid1
    5. uuid3
    6. uuid4
    7. uuid5
18. fastutils.rsautils
    1. decrypt
    2. encrypt
    3. export_key
    4. get_hash_method
    5. load_private_key
    6. load_public_key
    7. load_public_key_from_private_key
    8. newkeys
    9. sign
    10. verify
19. fastutils.strutils
    1. BAI
    2. BASE64_CHARS
    3. HEXLIFY_CHARS
    4. QIAN
    5. SHI
    6. URLSAFEB64_CHARS
    7. WAN
    8. YI
    9. binarify
    10. camel
    11. captital_number
    12. char_force_to_int
    13. chunk
    14. clean
    15. combinations
    16. combinations2
    17. decodable
    18. default_cn_digits
    19. default_cn_float_places
    20. default_cn_negative
    21. default_cn_places
    22. default_cn_yuan
    23. default_encoding
    24. default_encodings
    25. default_quotes
    26. default_random_string_choices
    27. do_clean
    28. encodable
    29. force_float
    30. force_int
    31. force_numberic
    32. force_type_to
    33. format_with_mapping
    34. get_all_substrings
    35. html_element_css_append
    36. int2bytes
    37. ints2bytes
    38. is_base64_decodable
    39. is_chinese_character
    40. is_hex_digits
    41. is_str_composed_by_the_choices
    42. is_unhexlifiable
    43. is_urlsafeb64_decodable
    44. is_uuid
    45. join_lines
    46. no_mapping
    47. none_to_empty_string
    48. random_string
    49. remove_prefix
    50. remove_suffix
    51. reverse
    52. simplesplit
    53. smart_get_binary_data
    54. split
    55. split2
    56. str_composed_by
    57. stringlist_append
    58. strip_string
    59. substrings
    60. text_display_length
    61. text_display_shorten
    62. unbinarify
    63. unquote
    64. wholestrip
20. fastutils.sysutils
    1. default_timeout_kill
    2. execute_script
    3. get_current_thread_id
    4. get_daemon_application_pid
    5. get_random_script_name
    6. get_worker_id
    7. get_worker_info
    8. psutil_timeout_kill
21. fastutils.threadutils
    1. Counter
    2. LoopIdle
    3. Service
    4. ServiceStop
    5. ServiceTerminate
    6. SimpleConsumer
    7. SimpleProducer
    8. SimpleProducerConsumerServer
    9. SimpleServer
    10. StartOnTerminatedService
22. fastutils.treeutils
    1. build_tree
    2. print_tree
    3. print_tree_callback
    4. tree_walk
23. fastutils.typingutils
    1. Number
    2. STRING_ENCODINGS
    3. register_global_caster
    4. smart_cast

Help

See pydoc for help.

python3 -m pydoc -b # start pydoc

Usage Examples

fastutils.strutils.chunk

from fastutils import strutils

In [33]: strutils.chunk('hello', 3)
Out[33]: ['hel', 'lo']

In [34]: strutils.chunk('hello', 6)
Out[34]: ['hello']

In [35]: strutils.chunk('hello', 5)
Out[35]: ['hello']

fastutils.strutils.get_all_substrings

from fastutils import strutils

In [4]: strutils.get_all_substrings('a')
Out[4]: {'a'}

In [5]: strutils.get_all_substrings('ab')
Out[5]: {'a', 'ab', 'b'}

In [6]: strutils.get_all_substrings('abc')
Out[6]: {'a', 'ab', 'abc', 'b', 'bc', 'c'}

In [7]: strutils.get_all_substrings('abcd')
Out[7]: {'a', 'ab', 'abc', 'abcd', 'b', 'bc', 'bcd', 'c', 'cd', 'd'}

fastutils.listutils.list2dict

In [1]: from fastutils import listutils

In [2]: listutils.list2dict([1,2,3], ['a', 'b'])
Out[2]: {'a': 1, 'b': 2}

In [3]: listutils.list2dict([1,2,3], ['a', 'b', 'c'])
Out[3]: {'a': 1, 'b': 2, 'c': 3}

In [4]: listutils.list2dict([1,2,3], ['a', 'b', 'c', 'd'])
Out[4]: {'a': 1, 'b': 2, 'c': 3, 'd': None}

fastutils.listutils.compare_execute

Example 1

from fastutils import listutils

old_list = [1,2,3]
new_list = [2,3,4]

created, deleted, changed = listutils.compare_execute(
      old_data=old_list,
      new_data=new_list,
      create_callback=lambda key, data: data,
      delete_callback=lambda key, instance: instance,
      change_callback=lambda key, instance, data: instance,
      old_key=lambda x: x
)
assert created == [4]
assert deleted == [1]
assert changed == [2, 3]

Example 2

from fastutils import listutils

old_list = [1,2,3]
new_list = [3,4,5]

def do_create(key, data):
   if data > 4:
         return data
   else:
         return None

def do_delete(key, instance):
   if instance > 1:
         return instance
   else:
         return None

def do_change(key, instance, data):
   return None

created, deleted, changed = listutils.compare_execute(
   old_data=old_list,
   new_data=new_list,
   create_callback=do_create,
   delete_callback=do_delete,
   change_callback=do_change,
   old_key=lambda x: x
)
assert created == [5]
assert deleted == [2]
assert changed == []

Notice

• We suggest you use pycryptodome>=3.14.1, lower versions are not tested.

Releases

v0.43.0

• Doc Update.
• Add python_requires in setup.
• Fix sysutils.get_worker_info problem in docker env.

v0.42.14

• Add fastutils.listutils.compare_execute.
• Add fastutils.dictutils.Object.fix to fix the Object instance if you added a dict thing to the Objects' list/dict item.

v0.42.13

• Add fastutils.strutils.get_all_substrings.
• Add fastutils.listutils.list2dict.

v0.42.11

• Fix fastutils.cipherutils.CipherBase incompatibility problem.

v0.43.9

• doc update.

v0.42.8

• Fix threadutils.Counter incr/decr result value duplicate problem.

v0.42.7

• Fix logutils formatter missing problem.

v0.42.6

• Add filelike-methods for fsutils.TemporaryFile.

v0.42.5

• Add TemporaryFile.open and TemporaryFile.close methods.

v0.42.4

• Change rsautils.encrypt default envelope method to PKCS1_OAEP, and rsautils.decrypt support both PKCS1_OAEP and PKCS1_v1_5 methods.
• Add rsautils.sign and resautils.verify methods.

v0.42.3

• Add console_handler_class and file_handler_class parameters for function logutils.get_simple_config.
• Add simple and simple_json formatter in logutils.get_simple_config.

v0.42.2

• Add sysutils.get_worker_info.
• Fix logutils' implicit imports problem.
• Fix funcutils.classproperty.

v0.42.1

• Fix logutils problem.

v0.42.0

• Add msgpackutils.
• Fix problem in nameutils.

v0.41.1

• Fix numericutils.get_float_part problem.

v0.41.0

• cipherutils changes.
• Fix license_files missing problem.

v0.40.2

• Add cipherutils.raw_aes_key.
• Add cipherutils.get_aes_mode.

v0.40.0

• Add listutils.chunk.
• New threadutils.
• Fix dictutils.Object.

v0.39.8

• fsutils.TemporaryFile add auto delete.
• dictutils.changes can return changed keys.
• dictutils.change can control do update or not.
• dictutils.change add ignore_empty_value support.

v0.39.7

• Fix strutils.force_byte, strutils.force_text.

v0.39.6

• Fix encode_datetime problem.

v0.39.5

• Add field name while failed to do typing cast.

v0.39.4

• Add more default jsonutils encoders.

v0.39.1

• Fix logutils.setup default config for windows.

v0.39.0

• Change logutils.setup parameter name.
• Fix bizerror.BizError encoding problem in jsonutils.

v0.38.2

• Add timeutils.TimeService.

v0.37.11

• Fix fsutils.write.

v0.37.10

• fsutils.readfile add default params. The default value will be returned if the target file is not exists.
• Add dictutils.diff function.
• Force folder create in fsutils.write.
• Make sure log folder exists in logutils.setup.
• Add sysutils.execute_script.

v0.37.3

• Add PyYAML deps in requirements.txt.

v0.37.2

• SimpleProducerConsumerServerBase.on_consume_error function add new parameter: task.
• Fix threadutils problems. Many changes are made.

v0.37.0

• Add fsutils.load_application_config.

v0.36.2

• Add fsutils.TemporaryFile
• Add threadutils.SimpleProducerConsumerServer and fix problems in ServiceCore.

v0.35.0

• strutils.is_uuid add allow_bad_characters parameter, default to False. For some old bad application using '815f5ecb-eg07-43af-8de2-87d3898093b5' as UUID.
• Add nameutils.guess_surname.
• Add fsutils.get_size_deviation, fsutils.get_unit_size and fsutils.get_size_display.

v0.34.0

• Add sysutils.get_daemon_application_pid.
• Add httputils.

v0.33.0

• Add fsutils.info
• Add strutils.encodable and strutils.decodable

v0.32.0

• Add funcutils.try_again_on_error.

v0.31.0

• Add fsutils.expands.
• Add fsutils.get_application_config_filepath.

v0.30.1

• Add cipherutils result_encoders' typing tips.

v0.30.0

• Add strutils.stringlist_append.
• Add strutils.html_element_css_append.
• Add strutils.split2.
• Add strutils.remove_prefix.
• Add strutils.remove_suffix.
• Change strutils.split("", [","]) returns [] instead of [""].
• Add cipherutils.MysqlAesCipher.
• Add listutils.topological_sort.
• Add nameutils.

v0.29.0

• Use pypinyin module instead of dragonmapper.
  • Replace the result lve to lue.

v0.28.2

• Fix dictutils.update, so that it can works on anything.

v0.28.1

• Add default parameter to fsutils.first_exists_file.

v0.28.0

• Add fsutils.first_exists_file.
• Add fsutils.expand.
• Add listutils.first.

v0.27.0

• Add lower_first parameter for strutils.camel.
• Add dictutils.prefix_key.
• Add sysutils.get_worker_id.

v0.26.0

• Add treeutils.build_tree from list.
• Add treeutils.walk_tree.
• Add treeutils.print_tree.
• Add fsutils.touch.

v0.25.0

• Add strutils.is_uuid.
• Add typingutils.cast_uuid.

v0.24.2

• fsutils.file_content_replace add ignore_errors parameter.

v0.24.1

• typingutils.cast_xxx treat empty string as None.
• randomutils.UuidGenerator.next add parameter n, so that it can generate n uuids. If n==1, returns a UUID instance, and if n>1, returns a list of UUIDs.

v0.24.0

• Add strutils.unquote.

v0.23.0

• Add funcutils.get_default_values.
• Add funcutils.chain.

v0.22.0

• Add none-dict support in dictutils.change and dictutils.changes.
• Add strutils.format_with_mapping.

v0.21.0

• Add listutils.compare(old_set, new_set).
• Add dictutils.change(object_instance, data_dict, object_key, dict_key=None) -> bool.
• Add dictutils.changes(object_instance, data_dict, keys) -> bool:
• Change listutils.replace, turn the default value of parameter inplace from True to False.
• Fix Object instance problems.

v0.20.0

• Add strutils.clean and change strutils.camel.
• Add dictutils.to_object.
• Add pinyinutils.

v0.19.0

• Add fsutils.filecopy and fsutils.treecopy. Tips: fsutils.copy combines the function of filecopy and treecopy, if the src is a file then use filecopy, and if the src is a folder then use treecopy.
• Fix problem in fsutils.copy.

v0.18.0

• Add strutils.camel.

v0.17.0

• Add fsutils.file_content_replace.
• Add fsutils.move.
• Change fsutils.rename's behavior.
• Change fsutils.copy's behavior.

v0.16.0

• Add fsutils.

v0.15.2

• Add hostname, seed1, seed4 in domain_template of randomutils.UuidGenerator.

v0.15.1

• Add incr-lock for counter incr in randomutils.UuidGenerator.

v0.15.0

• Add randomutils.UuidGenerator.

v0.14.0

• Add strutils.substrings.
• Add strutils.combinations.
• Add cipherutils.Utf8Encoder.
• Add cipherutils.S1Cipher.
• Add cipherutils.S2Cipher.
• Add funcutils.classproperty
• WARN: Change a cipherutils.CipherBase's parameter name from encoder to result_encoder.

v0.13.4

• Use new algorithm to improve randomutils.shuffle's performance.

v0.13.3

• Add randomutils.shuffle.
• Change randomutils.randint parameters from (max, min=0) to (a, b=None).

v0.13.2

• Add listutils.group.
• Add rsautils.export_key. And rsautils use Crypto.PublicKey.RSA for the base engine.

v0.13.1

• Add strutils.force_float and strutils.force_numeric.
• Add typingutils.cast_numeric support.

v0.13.0

• IvfCihper accept integer value when float_digits=0.
• Fix IvfCihper deviation problem which is caused by wrong module used in decrypt.
• IvfCihper use new algorithm in computing module and max_value, so that result encrypted by version v0.12.0 can not decrypted by version v0.13.0.

v0.12.0

• Add IvfCihper for float number encrypt and decrypt. The output of IvfCipher.encrypt is string.

v0.11.1

• Use class instead of raw api for s12 and iv ciphers. It's can avoid many times in generating seeds.

v0.11.0

• Add cipherutils.
• Add strutils.binarify and strutils.unbinarify.
• Add randomutils.Random.
• Change aesutils functions' return type. Note: use cipherutils instead.

v0.10.1

• Add bizerror dependency.

v0.10.0

• Add strutils.is_chinese_character to test if the character is a chinese character.
• Add cacheutils.get_cached_value to get or set cached value.

v0.9.0

• Add listutils.append_new to append new value and only new value to the list.

v0.8.0

• Add strutils.smart_get_binary_data.
• Add rsautils.

v0.7.0

• Add hashutils.get_file_hash.
• Add extra install requires for python 2.x.
• Add imageutils.parse_base64image and imageutils.get_image_bytes.
• Fix jsonutils.make_simple_json_encoder ignore bases problem.

v0.6.0

• Add imageutils, add imageutils.get_base64image to make base64 image that can be rendered by web browser.
• Add imageutils.resize to scale image size.
• Add Image-Object-Encode support in jsonutils.
• Add threadutils, add threadutils.Service to simplify long-run-service programming.
• Raise bizerror.MissingParameter error in funcutils.get_inject_params while missing required parameter.

v0.5.4

• Fix hashutils.get_hash_hexdigest and hashutils.get_hash_base64 problem.

v0.5.3

• Using typingutils.smart_cast in funcutils.get_inject_params.

v0.5.2

• Add unit test cases for typingutils.
• Fix cast_list, do strip for every element in comma-separated-list.
• Fix base64 import missing in typingutils.

v0.5.1

• Add typingutils.cast_str.

v0.5.0

• Set library property in get_encoder in jsonutils.
• Add typingutils.

v0.4.0

• Add jsonutils, provides simple json encoder register system.

v0.3.2

• Fix problems for python 2.7.
• Fix name error of funcutils.

v0.3.1

• Fix problem casued by str.isascii() which is new from python 3.7.

v0.3.0

• Add listutils.unique to remove duplicated elements from the list.
• Add listutils.replace to replace element value in thelist with new_value in collection of map.

v0.2.0

• Add functuils.get_inject_params to smartly choose parameters from candidates by determine with the function's signature.
• Add functuils.call_with_inject to smartly call the function by smartly choose parameters.

v0.1.1

• Add strutils.wholestrip function, use to remove all white spaces in text.
• Fix strutils.is_urlsafeb64_decodable, strutils.is_base64_decodable and strutils.is_unhexlifiable functions, that have problem to test text contains whitespaces.

v0.1.0

• Add simple utils about operations of aes, dict, hash, list and str.