Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.8+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.6.20-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 80f617bf91e27620c2d0f975ed067b562f4b1558848b315fe17fb4a18d3fd88b |
|
MD5 | 5373df3546dfc9821b790c7d1c19312c |
|
BLAKE2b-256 | c6e283607d24f2c86854a52438c069fe1866acd7918c5bcb80c9148571ff34e2 |
Hashes for asyncdb-2.6.20-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 638d9eb4ce211e12a045aa4d3adc9060e1d302e10a97ba09b6a52e94f82f23f0 |
|
MD5 | ac93d61f744c01018d8ca1ac58e1ea12 |
|
BLAKE2b-256 | 1ac83a76a10a6e02a67ce77beee787fd2bb8d6931660c011eb969eb499ded1b5 |
Hashes for asyncdb-2.6.20-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 88d8612e4557ab713d192b37713ff02790a347471b6b42a62230784e0b4d12c4 |
|
MD5 | 0706c0a46f9eb7ac14a420a08321d71d |
|
BLAKE2b-256 | 1e41e76243781f7be78d7b1f27284640ada41dac770f29d3db65ae695f42877f |
Hashes for asyncdb-2.6.20-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bcb19c132ff969a2bdb5188c9a4d5b68f66dd0a776ff503a2fdfdef128918072 |
|
MD5 | c344d6484d5f4577fc52a5651a8c75d3 |
|
BLAKE2b-256 | 81116e9d9f811ed45ee1e1f0739b90d56e7dcb67cba929f2dad7757fa4db091f |
Hashes for asyncdb-2.6.20-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6f3d64b0b8ded971b96a4b5e6edf07599468afde451e5d66d1cc792b2010c112 |
|
MD5 | 00f33a855c13b855eb5213248e7ad2fe |
|
BLAKE2b-256 | 957aad11f666a31088ec2ded5bf8429054aa529eb1daea45cd611d666c4a8fee |
Hashes for asyncdb-2.6.20-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fe94d44257b5026b2fd430b9dbf61845dab135ed4e8b40f47dbd4efaee0600f3 |
|
MD5 | d0a9ede27fef3a2329ce42dfeb59cc8a |
|
BLAKE2b-256 | 74f62c6e7e3a031dd6eff49a55d3550b288a2c419cb127d9262c21c466015ae6 |
Hashes for asyncdb-2.6.20-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 32fce36ea97b20daf8e7ea30e3d82603909cd9c1424c112623ac9f3edab8d992 |
|
MD5 | ab7b181263b556c527987ae9913f1183 |
|
BLAKE2b-256 | e960b71fe8af4c7d31d0698c5edc0dc72233626e45e3d94cf4de26a3aa00f67d |
Hashes for asyncdb-2.6.20-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0cfd4e78205e11dc57c08e80c3c019c73bc700bda4fdcf4da2668c38692d5f11 |
|
MD5 | e62b90f279bef2d8f91f7e386bfc209f |
|
BLAKE2b-256 | 8dc44e65ecff2604e5118adb172ced11068a21936e1f42d11ad5d1c96dfeaaaa |
Hashes for asyncdb-2.6.20-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 61ad6789a83f8eb7ad16c994bccd097ba38253ec834cfffb84472999ed66bbc4 |
|
MD5 | 45f407738f22ebe78bb94ea277800ebd |
|
BLAKE2b-256 | a5f17d83b546ea1743d5e2d7bf91f67e2a056a6330ebc70120047f722aa89a0e |
Hashes for asyncdb-2.6.20-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4e14982f8f3e94657b74518324a755b379eb3933fc80b9b1472b4bf8f7c25a68 |
|
MD5 | 000e08b660057b84c60679fbee28ff8d |
|
BLAKE2b-256 | bf25d0f1a4a048f4c227f5474b7e4b43d989aa3c6321fd380a935032403b9768 |