Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.8+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: aioredis (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.6.17-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ad7600accc381f90343feaad6b539ab277f6374851d8d2ae066e6cb06c0a943c |
|
MD5 | fba042d5e5e23f8cfdb97106650ac0de |
|
BLAKE2b-256 | 2c4118c2a6433e8422ec3ab5beea92c23f6354adfba50a52f1f56e25901e512d |
Hashes for asyncdb-2.6.17-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ddc7167a02bd72dbb1526e1c78be4d4ca8df2952c5b9746b9411c22a37fa7b0b |
|
MD5 | 5e93be51953dbef1785228475904ca5b |
|
BLAKE2b-256 | 6cbd847c7e1ae04ba9f7533d011c3b12b21710e92c2708a699ee245abbd1814e |
Hashes for asyncdb-2.6.17-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bc0d6c87a468b272e08bcd981be18068b00404f476cec3ea68774aa8a3fc002b |
|
MD5 | 981753f7a7fd5e039bdfeca4722733a6 |
|
BLAKE2b-256 | e4ea8311b9bd9ea43dc26343c62acd66888f8b0361a14357785e58a161a74da0 |
Hashes for asyncdb-2.6.17-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e09f62a302ff685eafdc96b3ab876f64a0dd6946881b6e184c3b8b4e48fd7348 |
|
MD5 | eb341c496793bc47e3eab6a3dacd10bd |
|
BLAKE2b-256 | 789a141feb63e64f4ee4a9f7141d464a6c0f3bf3ac5e5fed139404650a2ce283 |
Hashes for asyncdb-2.6.17-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 07bfceaa4a91fa9907ac64a314269b5ed3d1b1f8f1bc08f02cb64899df1671de |
|
MD5 | e15a278f0d44dfe719632702dce4cb08 |
|
BLAKE2b-256 | e195c0a488cd332f5760fbcf931fdb316f561f2ab080a99806a8510593cdfdde |
Hashes for asyncdb-2.6.17-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1ac389fccc8d6cf73201db7cfe6f79b22e02b7e34f2e3b241d5ac85882f03485 |
|
MD5 | 400a87a1498e97f507f515112c0c3f5d |
|
BLAKE2b-256 | 3812a8d49a0ab3206ab73d6d9d9ebcd9015b4c5b653edc9bed3ee45edf60514f |
Hashes for asyncdb-2.6.17-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1058dbeafcb9646e400e317de814109e2921ead9de41105d83e93865c53c6b7f |
|
MD5 | 8b5857d93e7810950f6970b39d45a1c2 |
|
BLAKE2b-256 | 20dfae921ca6c3390e49b1106bc74bb6e2b2f3869d7a602f4e1434943ffef7b4 |
Hashes for asyncdb-2.6.17-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8678ddf87fa1361c1daff3f9d67104ec0c85b31a22ea183f58755581987969a8 |
|
MD5 | a15702a03f0c159b50fc46960497ad2c |
|
BLAKE2b-256 | 24a4295595d045c3c084956038495a71147feab54d817c3c8566593aa6d7f4ca |
Hashes for asyncdb-2.6.17-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 042d36975f4ac6e8a865ee0f8f8e2757639ab76ea490b9059f08e18b3b27348b |
|
MD5 | 9a1723755e537ff7e66d678af993fe1d |
|
BLAKE2b-256 | 29e93c3b324af11105b5562fa06af6d95687bf0e4b670ca870189402e93fff1a |
Hashes for asyncdb-2.6.17-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 17989bedc97af881abf408cd9ca3d8b257f6669c18103affdcc3e19bb6cede2a |
|
MD5 | 48ace9425af4a60fb1da6eafcd103f5e |
|
BLAKE2b-256 | 4d9187226ad8378a7d62c6ee9f65360e88be426c4a92cf243451b9bbaaa0514d |