Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.9+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.7.10-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ec8d3b6f9ca2c7dce17660c9e29b5070a97eb7101469f472898c050aa8bafc62 |
|
MD5 | 73eecc932367e38f1f43954eea4ef603 |
|
BLAKE2b-256 | abfe425197f05d061c3bb0c8f20c2641c787d1ea11cd11f0065d79898dcc0481 |
Hashes for asyncdb-2.7.10-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 900a007adfea0fd459de9247210211b10c4d9ce8560fa001e809526410b28f04 |
|
MD5 | b5ca054c80b333d79d2b8212e2da22d8 |
|
BLAKE2b-256 | 9b555005057ce024712666e8832c68737de072fcd8d7268ae3b28a05bef2a89e |
Hashes for asyncdb-2.7.10-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9ed691931d733787ad6054b95bcdbcfd59f5114fe0823b53a33d537450e10ab8 |
|
MD5 | 543e5efe905fc693a46ea7904a3eef8a |
|
BLAKE2b-256 | ed703f1388b83f4137e611f9452c4139b46d19bff17042172d157b396b46118c |
Hashes for asyncdb-2.7.10-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2f6cfd4d7403eefb5436433a3d622a1e60d80de90407c60899e5386e40326586 |
|
MD5 | bd6ef2985c9b0d468a96b14f3a40eb3c |
|
BLAKE2b-256 | d0b2e9d413402d0406d71187831cf36c41b9c277969c1aad506dcbc1eb88f258 |
Hashes for asyncdb-2.7.10-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f089a764aa9d30f90bb0a02cc71854d0c491b6aba42a8da9b58aae14df619258 |
|
MD5 | 109c598d82df44bd0d4a7ebd32055e43 |
|
BLAKE2b-256 | 19849a268a2ca4d89b197020223825e0edaf9809d1453b001063d2ed2f930ab7 |
Hashes for asyncdb-2.7.10-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e85b66b46e424867b5cc4f89554ee13b67d5e7f89043071441bd4f60cdb95ffd |
|
MD5 | 15fb41487cf06056ea98b6d68142457f |
|
BLAKE2b-256 | 368f51b6c1801dbf57831e7e9325b0033c3acd36a7fd9250fa5451bd949ce0ed |
Hashes for asyncdb-2.7.10-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d47c6144cc532256e92348bc2c230627b8de30c8ddbcddafd3718193b4f2b997 |
|
MD5 | e552a01759e48d3ead884943c84a7280 |
|
BLAKE2b-256 | b6d22504a66a01c4f559ba5e0f880149c28e4609708cc9fe008c59021712af03 |
Hashes for asyncdb-2.7.10-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5ebbfd495dfd30c48d10d492cba8b016a93543ec3f0e226d1cbd8463791171a5 |
|
MD5 | e9d390cf02d43ba3e91c9c4743b883f4 |
|
BLAKE2b-256 | 4c66bd5b5453e57d6fc0d5de8a9989395d8565e96decdab29126abdaf8944296 |
Hashes for asyncdb-2.7.10-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 72afc7c0ee1706ed81e8f8b0cf3e1e296b837d5d650b6baeecaaffbf1fc8429a |
|
MD5 | d93799668f5ab12447ee6cdd4f13274c |
|
BLAKE2b-256 | 4058a64a33154c2a52bf902e56d7af0b6e68e183ab25ac7bfddfddd60d8bc4ec |
Hashes for asyncdb-2.7.10-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4186d91897615f9d4d728fd8381728243619129b6fde2e286f2e582c5aad9fbf |
|
MD5 | c772e5e778a6c0643b8e959b6589e9ff |
|
BLAKE2b-256 | 785e7a5dc857e22e69cc3c286768df88793e7fdbd7fe5c2ed9e7691bd646e196 |