Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.9+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.7.9-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e6888991fbf64b48a41ade0d5f5ccf2782f6c5d30f78bc48cbf5ee5e7d0870a |
|
MD5 | a9c753a18e4b1c10d629f22d62307901 |
|
BLAKE2b-256 | 1fea56016b01d9976cb8b2ec142c40afad488e3c64f5a62781391111bfc6e273 |
Hashes for asyncdb-2.7.9-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 28070cf4323858cfaa6df9afa3be2ba40e59bb714a8c34f04b5fc847f1ead39a |
|
MD5 | dae6304e741f68ee4de9764a4ab8d112 |
|
BLAKE2b-256 | 68d4551170926f31302faa5f4baca1865cd7dcd0bc6fcf30dd759fddeb509fd1 |
Hashes for asyncdb-2.7.9-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 56f0e941487a4e7f63e93dd290d7390042bafb64030e5eb1f9e5269847391051 |
|
MD5 | 93c2680083a8577d892844d8b37063ac |
|
BLAKE2b-256 | dc38ed2dbbd164e575eb1d45b5f896ef564b8eb4603337afef3793150bf5e0bf |
Hashes for asyncdb-2.7.9-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0d6ce60a4f0c2748f3347603cff9b8f6ad4c007577102b44abf3bea052bdc2ff |
|
MD5 | 18c782d63d8cedca7c0751cb26268a38 |
|
BLAKE2b-256 | 5d34922830866127ec4911028a9c45ea3be65a3ce22c6f60487f79bf48994e99 |
Hashes for asyncdb-2.7.9-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b9c59d16be44928c558bf7993c1be763c1d4dcab79094cbdf28833ff40bb9422 |
|
MD5 | defbba646ce897cce402363a225bee5e |
|
BLAKE2b-256 | 338ed56b4a8e29d13687edcbd9ace05ff7a2075e3e0b414858a46363f7ca8c27 |
Hashes for asyncdb-2.7.9-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55f8c40a8fd25aa71e44cb692cd11cc40e34e736fffae392c92d8c2b223416cc |
|
MD5 | 2069b5c514f02430054a279e0ccada99 |
|
BLAKE2b-256 | ff05d459f044ad6a5063baaa9c3f2555f8a641dc54c996f54cff38e22a6b21e8 |
Hashes for asyncdb-2.7.9-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e699c1e54d3fc89c82dce79b34b662721a6e2aba4c0f902d589b087445cfb9d9 |
|
MD5 | c9db6795acd8fa4803d7fc60c5ddc03c |
|
BLAKE2b-256 | f629953d029be6ea87bba8482f9cfc9355d5c2f3d3f1c5ccbc29618eab472bb3 |
Hashes for asyncdb-2.7.9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | af2a3ac371318775801a8cde560e0c2b48ca646dd7a322d1aaadeaab645295db |
|
MD5 | b844035e6fc2fd8ff3aeafa35453a606 |
|
BLAKE2b-256 | da164a99847f7c8cbb2185e5b68f8ce16a322b9bbcb4ef5bafa823e483dcc29f |
Hashes for asyncdb-2.7.9-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6f8c78813b23fe3028512f1ef1ecb051c580bc0b808d25d61dc864fd59a77c86 |
|
MD5 | c9adaa4dae6d80108fd135135002b1a7 |
|
BLAKE2b-256 | 11a4ff93a844808bcc0ee6b366908e1e5b430024d3780819a7b53a368d371747 |
Hashes for asyncdb-2.7.9-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fe59b9d9c34f9b5543cdb6b29b64f9237aff72dd3b0d0d4eaf21851f6da540e5 |
|
MD5 | 0cbdf0bf8b23837de23a319d0b6e319f |
|
BLAKE2b-256 | 09030b0f081db44ca1fe06d29220b1df7893af8610debbf87b8b7182f05a9989 |