Skip to main content

A very useful tool-set for data/operation/testing engineers, specialized in ETL and finance.

Project description

LouisTools -- Tools for data/operation/testing engineers


About Louis

Louis is a data engineer working in financial area. A Python programmer, Linux operationer, loving learning and sharing.

About Tools

The tools are designed for the following technicians:

  • data engineers (ETL, especially for financial data and time-series data)
  • operation engineers (mainly Linux)
  • testing engineers (unnittest, logging, HTML, email)
  • data analysts (data I/O and data-washing, especially for financial data)
  • application developers (regex, logging, config I/O, useful decorators)

A HTMLTestRunner in Python 3 version is contained in the package (Thanks to Wai Yip Tung for making such a good-to-use Python 2 package).


1: You can parse a path string including date formatters:

import LouisTools as LT
file_path = "/data/certain_data/%Y/%m/%Y%m%d.csv"
today_str = LT.TODAY
today_file_path = LT.parse_date_in_str(file_path, today_str)
print(today_file_path)  # /data/certain_data/2020/06/20200616.csv

2: You can make a (parent) directory safely:

import LouisTools as LT
file_to_save = "/home/usr/new_project/data.hdf"
LT.make_parent_dir(file_to_save)  # /home/usr/new_project will be created if not exist
dir_to_make = "/home/usr/new_project"
LT.make_dir(file_to_save)  # /home/usr/new_project will be created if not exist

3: You can create a logger very simply and fastly:

import LouisTools as LT
LOGGER = LT.single_lvl_logger("LouisTools")  # Only StreamHandler, LouisTools is the log name
LOGGER = LT.single_lvl_logger("LouisTools", "/project/log/a_log_file.log")  # FileHandler is created saving in "/project/log/a_log_file.log"

4: You can interact with you config files safely and quickly (both json and ini supported):

import LouisTools as LT
json_path = "/home/usr/projectL/projectL.json"  # A typical json file path in Linux
config_content = LT.read_json(json_path, encoding_="utf-8")  # You can change encoding_ arg as what you want!
assert isinstance(config_content, dict)
ini_path = r"C:\Users\A_USER\projectX\projectX.ini"  # A typical ini file path in Windows
config_content = LT.read_ini(ini_path, encoding_="utf-8-sig")  # Windows' gift: a BOM added for freee!
assert isinstance(config_content, dict)

5: You can dump the result without overwrite possibly existing files (safe, safe, and safe! even the directory will be checked and created if not exist):

import LouisTools as LT
@LT.dump_without_overwrite(sep="\t", encoding="utf-8", header=True, index=False)
def generate_df(df, df_path):
    # You can do something on df
    return df, df_path

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Built Distribution

LouisTools-0.2.0-py3-none-any.whl (19.4 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page