Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.8+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: aioredis (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.6.12-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cc444b8979815861f07baa7ca686439e6586c4d919e42408d6d077e9a128129c |
|
MD5 | 878c1d725b1b55e804d87457656a8ece |
|
BLAKE2b-256 | b9586d2d125abff7b262a06b557676f4afc9e229e435f37e9f30cac9a826e5b8 |
Hashes for asyncdb-2.6.12-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a1ea54c4c1c20e7c71104998d0d35e70f8a18a8a7355ed1b583c6252ddf77124 |
|
MD5 | 713833ec81aa83bf0f33c5242bbcb792 |
|
BLAKE2b-256 | 8dd2435c44dc4b50c12b98c8a1221b94ace27b635356d0ed51fb8bb7cd4b0b46 |
Hashes for asyncdb-2.6.12-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5c6b05646a2b1c3a32e4f996abc91b652e2e60308bd831e595c1d1215aad034f |
|
MD5 | 7c346acd26df3fd7a0c5a61b3adfa542 |
|
BLAKE2b-256 | d48db742cd01e02ba2685ee9b3ab6f65b81c5143d2590cae95b8e85e25b5b994 |
Hashes for asyncdb-2.6.12-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 351ab5cb76eabe13cd64aa3eb21f3864dbe848b8f00cdde15d92e60b73fb6922 |
|
MD5 | 0770af3ba115ca8e6474a5ff059aa31c |
|
BLAKE2b-256 | d63072a1fc56ed46ca8b26aba2a287c81871595671f75d1f0616b45478d17a97 |
Hashes for asyncdb-2.6.12-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5e54702e1c5ddac43af5edf8394582d62cc455c8a1a734949f643bcdd653bb62 |
|
MD5 | 350b910468d029b932a451524e381bf2 |
|
BLAKE2b-256 | 8f4f3d4dd0466cb688fac55707b894645af9e18fe1ba4954fc0ea89f18cfedfe |
Hashes for asyncdb-2.6.12-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3ce5b82607cef062c65229d3faa70e81308ad4a83bea9495b3a6c3c24010eb28 |
|
MD5 | 72e169853903d857fe0974de5d2baa15 |
|
BLAKE2b-256 | f6b54ca7de1460b6cb8c58766b3f01723bb5329dba70b85ef7865014b8caa6f6 |
Hashes for asyncdb-2.6.12-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8ebb97a5ec7267deb9f7d59f5ca003ed5e88da0cb4c3115c0417372a32ac6319 |
|
MD5 | 95319b6a36167fd36068be3440881e42 |
|
BLAKE2b-256 | a81424f81fd885899644b7a10cede8f7958146ee0a9048cecc4e7362544581bb |
Hashes for asyncdb-2.6.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1000a891fc59d9e721183a1a2028bb4504ce250495a9b674e801a7d4a3e70c16 |
|
MD5 | 9b9829a01f86714d4a8def505b8065c3 |
|
BLAKE2b-256 | a7f70e0b5f3e55e08f14d51010bc427932c64460b00614ab19f9ca1766237fc5 |
Hashes for asyncdb-2.6.12-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 92a5a2fd45656cca7bccaadb65dd876a71094fe59cf8a270c3665234736d9487 |
|
MD5 | 26112aac785c1c81106cbccce065a416 |
|
BLAKE2b-256 | f842e32bfcf6e900a4e35b7d3bf9b707fc217e85dc7f3e4f6c1cf6074d45e2a3 |
Hashes for asyncdb-2.6.12-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cdeac1a0e5a31a178bbc7733a61e76028f37d3732baad4b8e0a977595e5f3666 |
|
MD5 | 68d991241d787483a86d5a3baaa13b4a |
|
BLAKE2b-256 | a8ec72e5254e562063ad5c56ffdb7bd9334a9a90a848185d13c3bee2896fd30d |