Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.8+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: aioredis (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.6.16-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 44a3f92e3c6e414a00e5186a3923a8b4b0fff59296e5dc2dee1344343f897e5a |
|
MD5 | 7e5f088641aee5c8f59da4ee391665f9 |
|
BLAKE2b-256 | d7634d0feebf2a82c54515052ab7e9f8ddbd9a1c5c9501a68512f7102218e6d7 |
Hashes for asyncdb-2.6.16-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3711e06f00b56c090c26ab63e0e1da167510d7b667a3a81542ea7c6ab0eecef3 |
|
MD5 | 3475f0ca719afa2c9a5218f14d2339d9 |
|
BLAKE2b-256 | 308e0f7ff9c8d3951f7bc37edc20acbedeec8a92963ebb60fb33147ac0fc486a |
Hashes for asyncdb-2.6.16-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 145d7beb7a21b9d4361b9b762765885a37a5fe95298207273d42a659a5a611b3 |
|
MD5 | 9ba3940adb13e7bc9413e4a98bd725de |
|
BLAKE2b-256 | 7cf26d153303c03b3dd4bc1b6c934a1a6393080959c29379f586122a8fe06451 |
Hashes for asyncdb-2.6.16-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d8ee4b027bde3956195cc0949a922d24389287ccb16858833aad40f259ba1432 |
|
MD5 | b914e7c8631d1f0d0bafb1c2ea9b44e7 |
|
BLAKE2b-256 | 2f660a0ddd7b96527693466afd07bfaa1116f53eed6a97e1c5b4080bf461fc19 |
Hashes for asyncdb-2.6.16-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 84bdfe302c54477ebb2a743bc12203eaef4b97f1063aa221876ebc25eb8b82a8 |
|
MD5 | 714ab5ec441d55e0e58600bf56056980 |
|
BLAKE2b-256 | 3b693acf70504cd38ba04af13cb6156fa928871ce90d949d589af75182b471e6 |
Hashes for asyncdb-2.6.16-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 23b6546f98da6ec3a8a0fc58b0e8cd860d562177d1d4ed9982b6286ba5163f95 |
|
MD5 | 488593513841fb5476fad2da82c57425 |
|
BLAKE2b-256 | 5da269f4f8b7f21be051083435a5fa5cdf0724954722433d5823060c882fd488 |
Hashes for asyncdb-2.6.16-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 25aae8fc16dfb0e235693d93317cf95121262d9ec63a1ac77a009271dff803ee |
|
MD5 | b16a6b512e37b0dd21d003078b482c37 |
|
BLAKE2b-256 | 27fb3992085c2c836e1317ed3861869f43eda6f869b2b5c42b76acfac72678fe |
Hashes for asyncdb-2.6.16-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b22f210aa58c03cd861174ce60c06d19e2a62df411e8f240e483bacc555962a9 |
|
MD5 | 1098de800936c79b76b22592c5e02c42 |
|
BLAKE2b-256 | eb7d9c4f763c9569b10ef4ef09ff0df433afd5926507d01deaf4172fc9d1c3de |
Hashes for asyncdb-2.6.16-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aae307ea66e1b7e991598f90444c59adff661ee5be7feb5685f5cf9b00c1d509 |
|
MD5 | 677443e3672f7ed4aab46533452bd0b1 |
|
BLAKE2b-256 | 18bcc78831fc7e9a5305e6dd43042d9e34ae0ea255a8e4a0469704a71e87a383 |
Hashes for asyncdb-2.6.16-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0991221741e3735295639d5f7a6483074cc35dfdbd80a4ea832908f6dc0d6bb5 |
|
MD5 | 6a01ea85b9bd80ebc06245fdb5cd250b |
|
BLAKE2b-256 | 360a272ce17e946edd5ea305efddc984fe33cacfd6b23bab84053c5cd3704d12 |