An opinionated lightweight wrapper around various DB backend drivers.
Project description
An opinionated lightweight wrapper around various SQL backend drivers.
Support right now is available for the following:
MySQL / MariaDB via mysqlclient
Microsoft SQL Server via pymssql
mongoDB via pymongo
Features
Opens and closes a connection for every query. I realize this is not what everyone needs. But I use this workflow in a lot of my projects, hence, opinionated.
Uses yield to return a generator to fetch large amount of data from a DB without loading everything into the memory.
Logs last executed query and time for query execution with a unique ID so queries can be traced in log messages.
Installation
pip install rapyd_db
Make sure you are also installing the appropriate drivers for your respective backend as mentioned above.
Or you can install with driver like this:
pip install rapyd_db[mysql]
Usage
MySQL Backend
# import this
from rapyd_db.backends.mysql import MySQL
# create the DB object; no connection is done at this point
# any argument accepted by the underlying DB driver can be passed
# `cursorclass` will be overwritten
db = MySQL(host='', user='', passwd='')
# run queries like so
# select large data with stream=True; returns a generator
rows = db.execute("SELECT * FROM blah", stream=True)
for row in rows:
print(row)
# insert data with stream=False (default); returns rows_affected, lastrowid and results of the query
rows_affected, last_inserted_id, results = db.execute(
'INSERT INTO blah(key) VALUE (%s)',
('value1', )
)
This is an excerpt of the log messages using basicConfig. This will change depending on your logging configuration:
INFO:rapyd_db.backends:f2e47d87874d4055beba66b6c8221aff - Connecting to DB INFO:rapyd_db.backends.mysql:f2e47d87874d4055beba66b6c8221aff - Starting executing query at 2019-10-28 15:47:31.182261 INFO:rapyd_db.backends.mysql:f2e47d87874d4055beba66b6c8221aff - SELECT * FROM blah INFO:rapyd_db.backends.mysql:f2e47d87874d4055beba66b6c8221aff - 2844047 row(s) affected in 10 second(s) INFO:rapyd_db.backends.mysql:f2e47d87874d4055beba66b6c8221aff - Ended query execution at 2019-10-28 15:47:41.747841 INFO:rapyd_db.backends:f2e47d87874d4055beba66b6c8221aff - Closed connection to DB
MSSQL Backend
TODO
Mongo Backend
TODO
Testing
For these tests to run successfully, the test rig should have access to a DB backend.
Running Tests
A very easy way to set these environment variables would be to add them to a shell script file and source them like so:
source ./test_environment.sh
To run all tests, run the following command:
python -m unittest discover -v
or
make testall
To run tests for a specific backend, issue the following command:
python -m unittest -v rapyd_db.tests.test_mysql
or
make testmysql
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.