Skip to main content

An opinionated lightweight wrapper around various SQL backend drivers.

Project description

An opinionated lightweight wrapper around various SQL backend drivers.

Support right now is available for the following:

  • MySQL / MariaDB via mysqlclient
  • Microsoft SQL Server via pymssql
  • mongoDB via pymongo

Features

  • Opens and closes a connection for every query. I realize this is not what everyone needs. But I use this workflow in a lot of my projects, hence, opinionated.
  • Uses yield to return a generator to fetch large amount of data from a DB without loading everything into the memory.
  • Logs last executed query and time for query execution with a unique ID so queries can be traced in log messages.

Installation

pip install rapyd_db

Make sure you are also installing the appropriate drivers for your respective backend as mentioned above.

Or you can install with driver like this:

pip install rapyd_db[mysql]

Usage

# import this
from rapyd_db.backends.mysql import MySQL

# create the DB object; no connection is done at this point
# any argument accepted by the underlying DB driver can be passed
db = MySQL(host='', user='', passwd='', db='')

# run queries like so

# select large data
rows = db.execute("SELECT * FROM blah")
for row in rows:
    print(row)

# insert data
db.execute(
    'INSERT INTO blah(key) VALUE (%s)',
    ('value1', )
)

This is an excerpt of the log messages using basicConfig. This will change depending on your logging configuration:

INFO:rapyd_db.backends:f2e47d87874d4055beba66b6c8221aff - Connecting to DB
INFO:rapyd_db.backends.mysql:f2e47d87874d4055beba66b6c8221aff - Starting executing query at 2019-10-28 15:47:31.182261
INFO:rapyd_db.backends.mysql:f2e47d87874d4055beba66b6c8221aff - SELECT * FROM blah
INFO:rapyd_db.backends.mysql:f2e47d87874d4055beba66b6c8221aff - 2844047 row(s) affected in 10 second(s)
INFO:rapyd_db.backends.mysql:f2e47d87874d4055beba66b6c8221aff - Ended query execution at 2019-10-28 15:47:41.747841
INFO:rapyd_db.backends:f2e47d87874d4055beba66b6c8221aff - Closed connection to DB

TODO

  • Add robust tests for all backends.

Testing

For these tests to run successfully, the test rig should have access to a DB backend.

Configuration

Here are some environment variables that can be set for the various backends.

MySQL Backend

  • MYSQL_HOST
  • MYSQL_PORT
  • MYSQL_USERNAME
  • MYSQL_PASSWORD

Running Tests

A very easy way to set these environment variables would be to add them to a shell script file and source them like so:

source ./test_setup.sh

To run all tests, run the following command:

python -m unittest dicover -v

To run tests for a specific backend, issue the following command:

python -m unittest sqlbootstrap.tests.test_mysql

Project details


Release history Release notifications

This version

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for rapyd_db, version 0.0.1
Filename, size File type Python version Upload date Hashes
Filename, size rapyd_db-0.0.1.tar.gz (11.5 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page