Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.9+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.7.1-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b7822399a54a8fcff1fc6c40f9f38970707ffa91b8cc1ab29394ebfa28bc8cb7 |
|
MD5 | 28f6002a16fa67bc99dbb55dae281b8b |
|
BLAKE2b-256 | 887861fb577d401aa3cd3db91681794bd1898157a040f53e03fffeb585bf67c4 |
Hashes for asyncdb-2.7.1-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c2d4412454cb0adf48bcf12611d8631a46bd2ba702ac21bc2c968ba669fa08cb |
|
MD5 | ac69db8b6a797a5be491e4ce7c9eed51 |
|
BLAKE2b-256 | 3a4f6f1ab234e7ce4012b5444bb5cc77fa2520a0306f9c73ed6f9817966e25c2 |
Hashes for asyncdb-2.7.1-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 94a7c7f5762a7d6eb376b3a78b4ca2dc5e13238f7b9865062a93dedae0fc97b4 |
|
MD5 | 19d25af0351a77e7ca2e4f3348e91f62 |
|
BLAKE2b-256 | e980211ecc31e587e922440168e17a68aa3b62bc3601d4dd86b559fa11756e75 |
Hashes for asyncdb-2.7.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0a7ff0c2ebb0cb887fdf538ac1ed80f59e83906f28e48bdd6787f99616825423 |
|
MD5 | 678011c50fb991b881361fd7db69544a |
|
BLAKE2b-256 | e590037b22c8c8f54cb8d6cd880d9dcc280f27c8bf7827bb1f41a28e6d618a25 |
Hashes for asyncdb-2.7.1-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b08818ece8f5d11c91b4f246dbf289249018bbd70ee87168583bd36f128bf492 |
|
MD5 | fcb13a77024da0cf4ea45d1546d05e12 |
|
BLAKE2b-256 | 3886184eee8b7db3e42431d9ba31932247f42823fbf179bd497570c7dbe3e9dd |
Hashes for asyncdb-2.7.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c5a458b5cc9b42654b7d481996d063eef26f59d97458d71d9cc92e267abe66b8 |
|
MD5 | 4dc0f04eb40cd5783d804bfb3fc9b0e3 |
|
BLAKE2b-256 | 6b4ca91b7b9761ca92dfa79608e7df1af88a30a537762bcb20f8a529866aabf9 |
Hashes for asyncdb-2.7.1-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d2eb3789a2a1219fce94a778e1e62df66654b99ae5ba923d6b13ee53bc91198f |
|
MD5 | c0c27f8e9f014e6bcd72656fd80d8b04 |
|
BLAKE2b-256 | ac2d286c5f21119ca775b4dbe1ff2c9bcf413d6f21a0e3afe57e7fbaeaea7a18 |
Hashes for asyncdb-2.7.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dcde2ba3d2faa920e6956e184da159a53a9e5f9f8a8baea2dd3ce8fee76e19bd |
|
MD5 | 264af31f708b6475c48a14b51f12802c |
|
BLAKE2b-256 | 431e3b99ce05425cfd242eff5988605e049bb7fd86759432a3654d61a32f9e1d |
Hashes for asyncdb-2.7.1-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8586d25602ec4be6d42660427e7b1bcc9943872ba0f3f27d2267f15ce1360f14 |
|
MD5 | ecc3adf8e4d3a4da712abbf3bc1be99a |
|
BLAKE2b-256 | cf965634461e7535d251dcc29bbc4b9ebeafe35159d71c1989b97bfffdbc21e6 |
Hashes for asyncdb-2.7.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e5e4b056e73b6ccfa7c4f6cc1a674e515a7483e0e2d54319036ed043afdc2087 |
|
MD5 | f22e6d19a423e6cc873c311128bb5f3c |
|
BLAKE2b-256 | f7278e19e0e27314036e224282d5b4ebcdfdbe5998daae54f5c6c4322388305c |