keboola.utils 1.1.2

General utility library for Python applications running in Keboola Connection environment

Python Utility library

Introduction

The library provides a useful set of utility functions frequently used when creating Python components for Keboola Connection. The utility library should be used in cooperation with the main Python Component library.

The Python Utility library is developed the Keboola Data Services team and is officially supported by Keboola. The library aims to ease the component creation process by removing the necessity to write frequently used functions all over again.

Links

• API Documentation: API Docs
• Source code: https://github.com/keboola/python-utils
• PYPI project code: https://pypi.org/project/keboola.utils
• Documentation: https://developers.keboola.com/extend/component/python-component-library

Quick start

Installation

The package can be installed via pip using:

pip install keboola.utils

Structure and functionality

The package currently contains one core module:

• keboola.utils.date - a set of methods for date manipulation.
• keboola.utils.helpers
  • a general helper functions and classes that are relevant in Keboola Connection environment.
• keboola.utils.header_normalizer
  • Different strategies to convert column names to a valid KBC format.

Helpers

The module contains general helper functions and classes that are relevant in Keboola Connection environment.

Date Utilities

The module contains all date related functions, which can be utilized to effective work with dates, when creating components for Keboola Connection.

Initialization

All util functions can be imported from keboola.utils module.

from keboola.utils import *

or

import keboola.utils.date

to import only functions from a certain module.

Getting converted date period from string

The function parse_datetime_interval() allows to parse any string containing date format into a Python datetime; or if strformat parameter is specified, into a datetime formatted string.

The positional arguments period_from and period_to can be specified in relative format (e.g. 3 days ago, 2 months ago, etc.) or in absolute format (e.g. 2020-01-01). For full list of supported formats, please refer to dateparser documentation.

from keboola.utils import *

dt_str_1 = '5 days ago'
dt_str_2 = 'today'
dt_format = '%Y-%m-%d'

start_date, end_date = parse_datetime_interval(dt_str_1, dt_str_2, dt_format)

Generating date period chunks

The function split_dates_to_chunks() allows to split time interval into chunks of specified size.

import keboola.utils.date as dutils
from datetime import date

dt_1 = date(2021, 1, 1)
dt_2 = date(2021, 1, 10)
dt_format = '%Y-%m-%d'

intervals = dutils.split_dates_to_chunks(dt_1, dt_2, intv=2, strformat=dt_format)

for intv in intervals:
    print(intv['start_date'], intv['end_date'])

Usage Example

import keboola.utils.date as dutils

dt_str_1 = '5 days ago'
dt_str_2 = 'today'
dt_format = '%Y-%m-%d'

start_date, end_date = dutils.parse_datetime_interval(dt_str_1, dt_str_2)

intervals = dutils.split_dates_to_chunks(start_date, end_date, intv=2, strformat=dt_format)

for intv in intervals:
    print(intv['start_date'], intv['end_date'])

Header normalizer

This module provides different strategies to normalize CSV column names to a format supported by the Keboola Connection Storage:

Only alphanumeric characters and underscores are allowed in column name. Underscore is not allowed on the beginning.

Example:

import keboola.utils.header_normalizer as hnorm

head_norm = hnorm.get_normalizer(strategy=hnorm.NormalizerStrategy.ENCODER, char_encoder="unicode")
header = ["dactor#fd", "a*ruas$", "48DHBb#@"]
norm_headers = head_norm.normalize_header(header)

# Results in: ['dactor_35_fd', 'a_42_ruas_36_', '48DHBb_35__64_'])

Example transliterate:

import keboola.utils.header_normalizer as hnorm

head_norm = hnorm.get_normalizer(strategy=hnorm.NormalizerStrategy.TRANSLITERATE, transliterator_mapper="ar2safebw")
header = ["اسم","قيمة"]
norm_headers = head_norm.normalize_header(header)

# Results in: ['Asm','qymp'])

Available transliterator mappers:

Mapper	Description
ar2bw	Arabic to Buckwalter
ar2safebw	Arabic to Safe Buckwalter
ar2xmlbw	Arabic to XML Buckwalter
ar2hsb	Arabic to Habash-Soudi-Buckwalter
bw2ar	Buckwalter to Arabic
bw2safebw	Buckwalter to Safe Buckwalter
bw2xmlbw	Buckwalter to XML Buckwalter
bw2hsb	Buckwalter to Habash-Soudi-Buckwalter
safebw2ar	Safe Buckwalter to Arabic
safebw2bw	Safe Buckwalter to Buckwalter
safebw2xmlbw	Safe Buckwalter to XML Buckwalter
safebw2hsb	Safe Buckwalter to Habash-Soudi-Buckwalter
xmlbw2ar	XML Buckwalter to Arabic
xmlbw2bw	XML Buckwalter to Buckwalter
xmlbw2safebw	XML Buckwalter to Safe Buckwalter
xmlbw2hsb	XML Buckwalter to Habash-Soudi-Buckwalter
hsb2ar	Habash-Soudi-Buckwalter to Arabic
hsb2bw	Habash-Soudi-Buckwalter to Buckwalter
hsb2safebw	Habash-Soudi-Buckwalter to Safe Buckwalter
hsb2xmlbw	Habash-Soudi-Buckwalter to XML Buckwalter

=======

License

MIT licensed, see LICENSE file.