db-toolkit 0.0.3

utility tools for db access and data extraction

db-toolkit

from db_toolkit.db_extract import *
from db_toolkit.db_connect import *
from pathlib import Path

This file will become your README and also the index of your documentation.

Developer Guide

If you are new to using nbdev here are some useful pointers to get you started.

Install db_toolkit in Development mode

# make sure db_toolkit package is installed in development mode
$ pip install -e .

# make changes under nbs/ directory
# ...

# compile to have changes apply to db_toolkit
$ nbdev_prepare

Usage

Installation

To use this package in another project, install as below

Install latest from the AzureDevOps repository:

$ pip install git+https://[e-mail]/lambeth/Analytics/_git/db-toolkit

Dependencies

The YAML file for this package dependencies can be found in the envs\ DIR. If using toolkit on another project, it is advisable to define db-toolkit as the below format YAML.

name: er
channels:
  - conda-forge
dependencies:
  - jupyterlab
  - pandas
  - oracledb
  - pyodbc
  - pip
  - pip:
    - nbdev
    - git+https://[e-mail]/lambeth/Analytics/_git/db-toolkit
    - sqlalchemy

Documentation

TODO potentially when github is available.

How to use

Set ini file location path

• TODO init file struct details show example

fpath2 = '../../../data/db_connections/cfg_address.ini'
dbe = DBExtract(cfg_fpath=fpath2)

SQL files

• Saved at ../data/sql
• TODO add details on path <dbname>__<tablename>

---

source

DBConfig

DBConfig (file_path)

Constructor for the DBConfig class. Reads parameters from the specified configuration file and presents them appropriately to the application.

---

source

DBExtract

DBExtract (cfg_fpath)

Constructor for the DBExtract class. Reads parameters from the specified configuration file Pairs SQL scrit to connection engine Pulls and saves data to landing folder

data_dir = Path("../data")
sql_dir = data_dir / 'sql'
sql_files = list(sql_dir.glob("*.sql"))
sql_files

[Path('../data/sql/BIODS__address_ctax.sql')]

Process files

# files wil be saved at ../data/landing
# dbe.process_save_landing()