A very useful tool-set for data/operation/testing engineers, specialized in ETL and finance.
Project description
LouisTools -- Tools for data/operation/testing engineers
About Louis
Louis is a data engineer working in financial area. A Python programmer, Linux operationer, loving learning and sharing.
About Tools
The tools are designed for the following technicians:
- data engineers (ETL, especially for financial data and time-series data)
- operation engineers (mainly Linux)
- testing engineers (unnittest, logging, HTML, email)
- data analysts (data I/O and data-washing, especially for financial data)
- application developers (regex, logging, config I/O, useful decorators)
A HTMLTestRunner in Python 3 version is contained in the package (Thanks to Wai Yip Tung for making such a good-to-use Python 2 package).
Examples
1: You can parse a path string including date formatters:
import LouisTools as LT
file_path = "/data/certain_data/%Y/%m/%Y%m%d.csv"
today_str = LT.TODAY
today_file_path = LT.parse_date_in_str(file_path, today_str)
print(today_file_path) # /data/certain_data/2020/06/20200616.csv
2: You can make a (parent) directory safely:
import LouisTools as LT
file_to_save = "/home/usr/new_project/data.hdf"
LT.make_parent_dir(file_to_save) # /home/usr/new_project will be created if not exist
dir_to_make = "/home/usr/new_project"
LT.make_dir(file_to_save) # /home/usr/new_project will be created if not exist
3: You can create a logger very simply and fastly:
import LouisTools as LT
LOGGER = LT.single_lvl_logger("LouisTools") # Only StreamHandler, LouisTools is the log name
LOGGER = LT.single_lvl_logger("LouisTools", "/project/log/a_log_file.log") # FileHandler is created saving in "/project/log/a_log_file.log"
4: You can interact with you config files safely and quickly (both json and ini supported):
import LouisTools as LT
json_path = "/home/usr/projectL/projectL.json" # A typical json file path in Linux
config_content = LT.read_json(json_path, encoding_="utf-8") # You can change encoding_ arg as what you want!
assert isinstance(config_content, dict)
ini_path = r"C:\Users\A_USER\projectX\projectX.ini" # A typical ini file path in Windows
config_content = LT.read_ini(ini_path, encoding_="utf-8-sig") # Windows' gift: a BOM added for freee!
assert isinstance(config_content, dict)
5: You can dump the result without overwrite possibly existing files (safe, safe, and safe! even the directory will be checked and created if not exist):
import LouisTools as LT
@LT.dump_without_overwrite(sep="\t", encoding="utf-8", header=True, index=False)
def generate_df(df, df_path):
# You can do something on df
return df, df_path
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for LouisTools-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 42ec9b38542b970f0698da51d941f12121bfff36473e3fe1a3d68f0600a25730 |
|
MD5 | b369c902ec3ff1033bb1dacea1f949e6 |
|
BLAKE2b-256 | 0f23e27c4f252183f8451a26547adf8b8d438fab786bf70ca9dd8ca03052eaa6 |