Skip to main content

A very useful tool-set for data/operation/testing engineers, specialized in ETL and finance.

Project description

LouisTools -- Tools for data/operation/testing engineers

PyPI PyPI

About Louis

Louis is a data engineer working in financial area. A Python programmer, Linux operationer, loving learning and sharing.

About Tools

The tools are designed for the following technicians:

  • data engineers (ETL, especially for financial data and time-series data)
  • operation engineers (mainly Linux)
  • testing engineers (unnittest, logging, HTML, email)
  • data analysts (data I/O and data-washing, especially for financial data)
  • application developers (regex, logging, config I/O, useful decorators)

A HTMLTestRunner in Python 3 version is contained in the package (Thanks to Wai Yip Tung for making such a good-to-use Python 2 package).

Examples

1: You can parse a path string including date formatters:

import LouisTools as LT
file_path = "/data/certain_data/%Y/%m/%Y%m%d.csv"
today_str = LT.TODAY
today_file_path = LT.parse_date_in_str(file_path, today_str)
print(today_file_path)  # /data/certain_data/2020/06/20200616.csv

2: You can make a (parent) directory safely:

import LouisTools as LT
file_to_save = "/home/usr/new_project/data.hdf"
LT.make_parent_dir(file_to_save)  # /home/usr/new_project will be created if not exist
dir_to_make = "/home/usr/new_project"
LT.make_dir(file_to_save)  # /home/usr/new_project will be created if not exist

3: You can create a logger very simply and fastly:

import LouisTools as LT
LOGGER = LT.single_lvl_logger("LouisTools")  # Only StreamHandler, LouisTools is the log name
LOGGER = LT.single_lvl_logger("LouisTools", "/project/log/a_log_file.log")  # FileHandler is created saving in "/project/log/a_log_file.log"

4: You can interact with you config files safely and quickly (both json and ini supported):

import LouisTools as LT
json_path = "/home/usr/projectL/projectL.json"  # A typical json file path in Linux
config_content = LT.read_json(json_path, encoding_="utf-8")  # You can change encoding_ arg as what you want!
assert isinstance(config_content, dict)
ini_path = r"C:\Users\A_USER\projectX\projectX.ini"  # A typical ini file path in Windows
config_content = LT.read_ini(ini_path, encoding_="utf-8-sig")  # Windows' gift: a BOM added for freee!
assert isinstance(config_content, dict)

5: You can dump the result without overwrite possibly existing files (safe, safe, and safe! even the directory will be checked and created if not exist):

import LouisTools as LT
@LT.dump_without_overwrite(sep="\t", encoding="utf-8", header=True, index=False)
def generate_df(df, df_path):
    # You can do something on df
    return df, df_path

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

LouisTools-0.2.0-py3-none-any.whl (19.4 kB view details)

Uploaded Python 3

File details

Details for the file LouisTools-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: LouisTools-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 19.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.3.1 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.6.4

File hashes

Hashes for LouisTools-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 42ec9b38542b970f0698da51d941f12121bfff36473e3fe1a3d68f0600a25730
MD5 b369c902ec3ff1033bb1dacea1f949e6
BLAKE2b-256 0f23e27c4f252183f8451a26547adf8b8d438fab786bf70ca9dd8ca03052eaa6

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page