Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.9+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.7.2-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 951b65bf81ff693a4a736a785823d12b8ea6e98e68d6a245f374d2478bad50dd |
|
MD5 | 121bcda7fd5bf76899a1453b37befa4e |
|
BLAKE2b-256 | 4aaa622809a6c4b1ce0f76dd2a1d1f9515af9efbe8c641e0d25eee7ca3dc7c4a |
Hashes for asyncdb-2.7.2-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2749a18008b761195bd996439c9f004e2c7f1a039ce72d2dfbc802942b215a26 |
|
MD5 | 586f7072fcc799ecec0bfa9a39bbe069 |
|
BLAKE2b-256 | 480d43c4ffcf25e663decfbcc08b2000a2d6e8c097ae460772487223c4068f0d |
Hashes for asyncdb-2.7.2-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 18841928820f0503edca5178dccddda7ce136b5755adcf200e5f27a89f43cc01 |
|
MD5 | 03df63df7b122f41558c3da7d17f1366 |
|
BLAKE2b-256 | 2f026df95bccf6c1f48bb5ddd897f2078a46fdff2bf35c54d04320aa62d43b18 |
Hashes for asyncdb-2.7.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d8a5c493bdbaebe4408d7a07dd469a5504eb0c8770f07c33e9f92bec2e2c71cb |
|
MD5 | affd2885fc2c64500667b872a1504f04 |
|
BLAKE2b-256 | ba9ff0d98c471025d89b4cb15fc614f910bde1c4495078b9dbfcda3c44f5d47b |
Hashes for asyncdb-2.7.2-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | de5309f534670d95f44935ddc95e3ca2d966bebb0de02de1941b842fd0a7f3ba |
|
MD5 | ac5fec407918a8bce08da5dbfeb0fb02 |
|
BLAKE2b-256 | 3b0b02436dd6d2af969b6862abc35c3126538ecfe745bc2e77f5e992fc810250 |
Hashes for asyncdb-2.7.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ecde6e3a89aa6684836b5cdf0c6a322fcdd0b2b894a8a320d23273e4fbe18422 |
|
MD5 | 1a20534fbc7428d0d3cf5fbb554d0ac6 |
|
BLAKE2b-256 | 5317bc90bdc39502bfe1ab1267397df731df5c39f8900cce8ee84f38fd4e34f9 |
Hashes for asyncdb-2.7.2-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1dc9acc06b3a01e907abc0932491a2750d26efe9b1c42a83807e2c0f07c23cfd |
|
MD5 | 409f5afdb0832700005995e8dde6231c |
|
BLAKE2b-256 | 8086a3ac2cf18c44ecd70ff40e0b28ffea6bc7cc559082b10836653a31f364a9 |
Hashes for asyncdb-2.7.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f3045c36d87b8606045820d1c98b23c57e43b0cb61df5e44e18fa12a94149b4e |
|
MD5 | e5bbdf6ae2f8821648b8289cceb8f005 |
|
BLAKE2b-256 | ecb583039633ca9de44af155f289f70b967f49a73648b1f02d66e1bef5de263c |
Hashes for asyncdb-2.7.2-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4fbf9d9e3f368b2bfc75e24aeaa539aafa1d32967a8f42ed061ef4bb46ddaba3 |
|
MD5 | 8de161feb49a8916b4290a584d0bb958 |
|
BLAKE2b-256 | 362133bb00fdbf1d5e246aa44be585db6074f9552e8b643ce2001cb4fd97802d |
Hashes for asyncdb-2.7.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3767bb0d23f35a8405f3e5c587a05ed2a6771c147994e842c07a1c15c2f883c7 |
|
MD5 | e3e845b6c0e0e45267583f16d2f96526 |
|
BLAKE2b-256 | ccab2438974382852c53c721c0b0761ae95eb8d2d36e6b5563f746bd08c930c0 |