Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.8+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: aioredis (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.6.11-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 48b8a4f2a509fa24db08be2bbe99716a66ae686dda361a7843000414c8636b89 |
|
MD5 | 8ddea57cc6d0a72978cbc630d509bafc |
|
BLAKE2b-256 | 6f001afb03782ff66a784d4521aef89e74a3a425dcd4780fd9124522c9319f55 |
Hashes for asyncdb-2.6.11-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2669cd9a9e50fc24ab4488578eb11a4dd12bdb513fd3e67ee6517089bd4b13e8 |
|
MD5 | 5856678d74db71aac3533f9eeeb6ffab |
|
BLAKE2b-256 | 3a950fb0a8babbcc5f2577ae9c6577e535fbeab12a205fe01114fe25a46db6f4 |
Hashes for asyncdb-2.6.11-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d3ddd76bf4c5b03ee39c6ab720636fdf26d7e7d92c192b9b9677cb2a6432b573 |
|
MD5 | c609b460ea4a0baca8d54a6e6ed2d5c7 |
|
BLAKE2b-256 | 6c5495727ce1a734e131f8492fa6ba9784a5bc73f5a2d9ead86d7758ab301014 |
Hashes for asyncdb-2.6.11-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6de7d4d40064ab3d72df7373afca0d090708d7b824311c44d70cb635fc0b47d4 |
|
MD5 | 6b1836817560fef4323749e9ff569338 |
|
BLAKE2b-256 | c3787254dfa092b56f772ed36cd415116141b7f8ff8d96b35cbe15e212b80c3d |
Hashes for asyncdb-2.6.11-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eed088f14a977ebdadc2707f3d69fdfc10a8d34d9c0642511d96bfdc57b38063 |
|
MD5 | 4df6a34137d776cb4900c272cb436837 |
|
BLAKE2b-256 | ec16cf52bfa4c550bd3c44064352efff052c0322a8600f081c05689662ed6a95 |
Hashes for asyncdb-2.6.11-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ef0e342683a7323c32ba4016b69d8ce4173891687b8bf248a7bd54a6aebbdd12 |
|
MD5 | bb3629dde3cd999400a69df4f2133c41 |
|
BLAKE2b-256 | ad5938281fba989b1e55c9be5919e3387e07c929722aa920d2108c9d9c99a513 |
Hashes for asyncdb-2.6.11-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3b50e89b976384586a098384f02cebe53792267e3d96a12453c9e44a105a0340 |
|
MD5 | 4e5b55749b24a8075796b1d93a8cfafd |
|
BLAKE2b-256 | f2cfd79b8b0f1b4b5d566b2faee06da6138c515b6952117343195535a3ebcda0 |
Hashes for asyncdb-2.6.11-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5bdabc2e79f3ce7535ef50673f34aed83472c0d72e65acfdd1f628e659f3caee |
|
MD5 | 1358636eb5cccea3f16b8eb64b844d8f |
|
BLAKE2b-256 | 8db5f45ccd1410197917e8948399970b05c7ea89ade8724783852f55711e5f24 |
Hashes for asyncdb-2.6.11-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4c13f4e70429f3a83dc606b8b26ca395f8db09deb84298bb2f2a2754a531337f |
|
MD5 | 68dfffa7a4db90cf81e53448e8b101f5 |
|
BLAKE2b-256 | bbd4a24af5ba57d70d68c605a550d18b3739b094e977eceeab3e52cdebac1572 |
Hashes for asyncdb-2.6.11-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 12ef0572c8e2d5d4ca82a09363e4800f144dcee55e6b78ff6175ef20377ce2b8 |
|
MD5 | 6605cd56ccb075bdafe532982900de3b |
|
BLAKE2b-256 | 72e13d402e8474dcfeb0729b07fdc8edae8bc76b90b03bf7b8e190c956bc5f3e |