Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.8+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.6.21-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 84cdfe6652dabb98b6d13e82ff48138f6953f159847f5e360bb13962c714d78f |
|
MD5 | b172f854a2d10804003066dbde53118a |
|
BLAKE2b-256 | 2fadee895acfbfcee2a829058442f64a5ad10183c61e9dae5fe3b0aaa4fdae7e |
Hashes for asyncdb-2.6.21-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4659b34cac3596e1736fc5806b550efe1e60dd20d6f5944bed80a7a1e7e81303 |
|
MD5 | 775ff3305cd5d7b73b8b701d7bcec88e |
|
BLAKE2b-256 | 8615a2d7956f2e05804cc0b6c170d34ba7219eaef198dd48c7d541be71bbd995 |
Hashes for asyncdb-2.6.21-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 738cfdb1a85e944e213151d4e3badd10f1a174c5c6ddd1fb5e2054fd55c341ac |
|
MD5 | 9c4e0cdf383bff07ccb27f22d9ca0997 |
|
BLAKE2b-256 | 9f6f246a7479b8245ef79433b9a3b110c46eccb2030924fb39ad0c549f9e7fd7 |
Hashes for asyncdb-2.6.21-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55d8598e9dba41714ffcecd8a4f0c90e61cc5ad78b3853a97a11cfb4dba86db7 |
|
MD5 | ec191476923e71052c8dc77c90e13c66 |
|
BLAKE2b-256 | db330741d9217145a0f4f86a6993ece41a818229e2f4c98f16bc34415674909f |
Hashes for asyncdb-2.6.21-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | db64d2252f79c7c14d0fca2db0a72e0f2ddcab974d25f4cd66974c35749c3e64 |
|
MD5 | 50bf79a7eb4bbf0fde41f9a342c9c7b5 |
|
BLAKE2b-256 | 9a5e601d01fc0f95950911bb921ae41049a8fad2434db2995939ac023ce290e9 |
Hashes for asyncdb-2.6.21-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a4232bee6f91a6266fe41c33e58fbddd72f58fa151f18eec7bdaad6fec0629fa |
|
MD5 | 12ef2699a7fda2d485b6c77a69c0ddf2 |
|
BLAKE2b-256 | 0887992c9f00e8eab5ec740f0aa0649a513b722e0c485b1208eb8633e878c6de |
Hashes for asyncdb-2.6.21-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5ec91724669a57714fcdcde1fd3d164cef9c170f7657a5690afbdade7c3762d1 |
|
MD5 | 2b76593ef0a663729edb10e1394c29ec |
|
BLAKE2b-256 | 48d6c0f759b19ab1b5d385c6147416c31e12768adf1cee005662428b5d649737 |
Hashes for asyncdb-2.6.21-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 997d3f9a37ac40d42c6cf20c938f9e22e99accd78d46fbceddc9595f962a3fda |
|
MD5 | 9cdef84f3c528a483d4cf9d5952743ca |
|
BLAKE2b-256 | 8d97d1c8d0fa5d66ce69160e73dab2d7c845c8c93bba6ff06c8beebc3d0a91b9 |
Hashes for asyncdb-2.6.21-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 01f85be5d2c1f9f84e2b59688ee60778d655ba470243bcc98aa99d991e076c38 |
|
MD5 | 0f4736cec0d67ea100da6ffc738069c7 |
|
BLAKE2b-256 | 51543d211a77a119bd1a037cba7fab76488c2fff1d78d69a1c05eafb0d769495 |
Hashes for asyncdb-2.6.21-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8241be8cbb41881df0b5d4bd77ba8938fa5055f28d49f27f9dc855573d21aaf4 |
|
MD5 | 40c4d06eb94086051f21e5ca29c33efd |
|
BLAKE2b-256 | e5b15be98bdb2c9ee695cd441daeda256c68667d06006a90a4e12c64cd32bf70 |