Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.8+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: aioredis (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.6.13-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 725061e58fcb7a04b9e31b0fce206ac08630c29f222eb1ef5aa16a3f10de2f83 |
|
MD5 | 0dc0f76c2be8e6b9d8eeb62831d0ca1d |
|
BLAKE2b-256 | edd3ed7742cac17bfc0dacc34cc6342f4d8658a66a3645d953f9c04bd2b6debc |
Hashes for asyncdb-2.6.13-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dbbdbc95f7a80d11050357ec3019e8ce720e9da84d2bfdf8fb43300d7b2f83cc |
|
MD5 | 6b3492b3f38de34090372d54b72fc32a |
|
BLAKE2b-256 | a4b0b36da0135f28c5dc0c407e0f52bbda2bc50cb3c37df1a6b76ebc9e9b4f76 |
Hashes for asyncdb-2.6.13-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0fc9f99f1f5bd444e330b24346b245f3823826fd60e1abd97e7fa6ee2d14b453 |
|
MD5 | a192d6d55fe3b4f5d231a4e4962a0945 |
|
BLAKE2b-256 | 1253fcf42003e1fbc66ad6620e400bc731b04d0127ced4e7c99d21b4c074b50d |
Hashes for asyncdb-2.6.13-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | adfb18911cf5beaa21cc4bf4d1d5b120f5ba16687fb0c8a0554f49f46804e831 |
|
MD5 | af855b5f0785c721cbe1ca34559b13c9 |
|
BLAKE2b-256 | 9e88973e8deb36c38a7d1be2bd796bff1a2e1a91797071b0610a8facc5fa892c |
Hashes for asyncdb-2.6.13-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 81a54b06a8a1dd45b7b5b24781c1a2ffeecb2a577eb1f004307315acf07f9200 |
|
MD5 | c6afb7b39f1516991412d9b9a6ad06d1 |
|
BLAKE2b-256 | c6a826fbd1010a15c91b9e9a275932d27d1827f8c12cb69d584f29b9894cc189 |
Hashes for asyncdb-2.6.13-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 08e08c8a6527c7ca99fa66e282562a6a26bfd64d1466918ee9fbf37b15d938a1 |
|
MD5 | 2d460cde6078367d8fcfeedea7d5d704 |
|
BLAKE2b-256 | 5cfac9b5af7d3f6b27399004456f7c01aab97f6c549c13b274e192b732064391 |
Hashes for asyncdb-2.6.13-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 39babbe101984f5cfd5c81ff33255b2cb643dd9287a413dd4080bbdaa1ac00ba |
|
MD5 | d73c907a1db4638fcdbc1b965e80910a |
|
BLAKE2b-256 | 2180be38d56458c7f4b8939ee09a0a497e256b7036532344306a4d09ade62f41 |
Hashes for asyncdb-2.6.13-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d6f19a92ced37b77c2dcda7adedceb406c5c8fdd5f89dada916237d65510988d |
|
MD5 | 99a8635c33522478dfff1bd0798e9375 |
|
BLAKE2b-256 | b388f3a2ba3c5d3aeaa81e093cbdf8ce1cf14eb1a9c1c9d3c8a4f9dab2467261 |
Hashes for asyncdb-2.6.13-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 06478096aab0099f5ffb33f069e2534d938db128774dec6db7dc1eddbd5d9327 |
|
MD5 | 1ba80080a1e88ffdd3a14fd9c9e67754 |
|
BLAKE2b-256 | d9c43d884d57de5ee323e3993e062054d9bb06c5079182a55c43885a557ac4e2 |
Hashes for asyncdb-2.6.13-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4d26023ff19047b33ed9034f4ba39ebfb361e754462479bffd042d77c4f2b721 |
|
MD5 | 1b1e1177f633991900326a34ef8a9a73 |
|
BLAKE2b-256 | a25a6aa60469f0820b001c656b26e737849ebb345129dc81d0df165e3dbcec68 |