snowpy 0.1.2

A Python library to upload and download data from database systems

SnowPy

Current release: SnowPy [v0.1.2]

'SnowPy' is a Python package providing a set of user-friendly functions to help upload and download data from database systems such as Microsoft SQL Server, Snowflake, (more to come). 'SnowPy' intially realsed as data tools for the mltoolkit project (https://mltoolkit.github.io/MLToolKit).

Introduction

SnowPy is a Python library to upload and download data from database system

Installation

pip install SnowPy

If the installation failed with dependancy issues, execute the above command with --no-dependencies

pip install SnowPy --no-dependencies

Functions

Usage

import snowpy

Warning: Python Variable, Function or Class names

The Python interpreter has a number of built-in functions. It is possible to overwrite thier definitions when coding without any rasing a warning from the Python interpriter. (https://docs.python.org/3/library/functions.html) Therfore, AVOID THESE NAMES as your variable, function or class names.

abs	all	any	ascii	bin	bool	bytearray	bytes
callable	chr	classmethod	compile	complex	delattr	dict	dir
divmod	enumerate	eval	exec	filter	float	format	frozenset
getattr	globals	hasattr	hash	help	hex	id	input
int	isinstance	issubclass	iter	len	list	locals	map
max	memoryview	min	next	object	oct	open	ord
pow	print	property	range	repr	reversed	round	set
setattr	slice	sorted	staticmethod	str	sum	super	tuple
type	vars	zip	__import__

If you accedently overwrite any of the built-in function (e.g. list), execute the following to bring built-in defition.

del(list)

Functions

|- MSSQL
| |- execute_mssql_query
| |- read_data_mssql
| |- write_data_mssql
| |- read_data_mssql_bcp
| |- write_data_mssql_bcp
|- MySQL
|  |- To be integrated...
|- Snowflake
|  |- execute_snowflake_sql_query
|  |- read_data_snowflake
|  |- write_data_snowflake
|- CSV
|  |- read_data_csv
|  |- write_data_csv

SnowPy Example

# to be posted

Data Loading

# to be posted

License

Copyright 2019 Sumudu Tennakoon

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Cite as

@misc{SnowPy2019,
  author =  "Sumudu Tennakoon",
  title = "SnowPy: A Python library to upload and download data from database systems",
  year = 2019,
  publisher = "GitHub",
  howpublished = {\url{https://mltoolkit.github.io/SnowPy/}},
  version = "0.1.0"
}

SnowPy Project Timeline

2018-07-02 [v0.0.1]: Initialte MLToolKit project. 2019-07-02 [v0.0.1]: Snowpy functions first released as etl tools for MLToolKit project (https://mltoolkit.github.io/MLToolKit/). 2019-07-02 [v0.1.0]: First Release of SnowPy as seperate package. 2019-11-11 [v0.1.1]: Bug Fixes and enhancements 2019-11-16 [v0.1.2]: Bug Fixes, Integrate MSSQL BCP support

Future Release Plan

TBD [v0.1.5]: Integrate Data Exchage Server. TBD [v0.1.6]: Integrate MySQL support. TBD [v0.1.7]: Comprehensive Documentation

References

• https://pandas.pydata.org/
• https://docs.snowflake.net/manuals/user-guide/python-connector.html
• https://www.numpy.org/
• https://docs.python.org/3.6/library/re.html
• http://json.org/
• https://www.sqlalchemy.org/
• https://docs.microsoft.com/en-us/sql/tools/bcp-utility
• https://github.com/mkleehammer/pyodbc/wiki