Vim pipeline utility to ingest data
Project description
vimdatautils
Vim python package to ingest data into a database
Features
- InboundPipeline.py utility, used by a specific integration component
- Data access layer (Dal), provides simplified methods to access data in postgres db
Installation
vimdatautils requires Python 3
pip install vimdatautils
Or to install/upgrade a specific version
pip install vimdatautils==<VERSION> --force-reinstall
Quickstart
- InboundPipeline, you will need to implement two methods: pre_load_logic, post_load_logic
from vimdatautils.inbound_pipeline import InboundPipeline
class Inbound(InboundPipeline):
def pre_load_logic(self):
print("this will be executed before the load!")
def post_load_logic(self):
print("this will be executed after the load!")
def main():
inbound = Inbound("config_file.json", "postgresql://postgres:password@127.0.0.1/postgres")
inbound.execute()
-
Dal, import vimdatautils.dal, you will need to construct the dal with a postgres connection string in the below pattern
postgresql://<db_user>:<db_password>@<db_host>/db_name
from vimdatautils.dal import Dal
dal = Dal("postgresql://postgres:password@127.0.0.1/postgres")
dal.execute_cmd("select 1;")
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
Close
Hashes for vimdatautils-0.78-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b161ba20a496500d3d8b805e01fea38115fda11e3a452aaab83dde0dcd874da7 |
|
MD5 | 119c6ec8512143b66f4354ea401da720 |
|
BLAKE2b-256 | eeea857316f97beec442c4624da08d65caafd8112cedee762f362b3f182be955 |