Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.9+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.7.11-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3decae903b502b8e918109271fdd0c19977a8d981cf222f83c95d1c673e7ed0c |
|
MD5 | 9cb9fb203664be9718d2d73e868942bb |
|
BLAKE2b-256 | 6c7c6d9e3b0c417c536906c96fc8b43c5d0511a9fe6b90d42aad952b07e547a3 |
Hashes for asyncdb-2.7.11-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f48b80f16c56ddce3a9ac67bf3776111267bd4ba9dd13710cc743e603b250e72 |
|
MD5 | 1ff191b98e1fb2b7f08f6599d77968b6 |
|
BLAKE2b-256 | 423ab11e2d7ff95e03f20c15ce8fc58225e1e23f54e52330e17c3244ab5a6e72 |
Hashes for asyncdb-2.7.11-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a29810fa58ef26fb561981d681bba2ea1e73f7474f5d4374faff9d46039daac1 |
|
MD5 | f0b5e18bb833d26b6121198a87186264 |
|
BLAKE2b-256 | 30f3d99af63e762baa6caa92039e60b644a05509429e2ec8009537cd2e648d61 |
Hashes for asyncdb-2.7.11-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7cf7d5ff6272e84ba2110550bb7b32d07aeda6bdb9580269fcacaacd2f7fe719 |
|
MD5 | d994692d581077823b2cb7b3d400b6cd |
|
BLAKE2b-256 | dedda40b1ad1ab5b278cb7f7e31d3acefc009b1344cbffb78239b9fae0a75efd |
Hashes for asyncdb-2.7.11-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 462f338eed600a9edab74c03afbc0bbf7becc4950af2c599a0757f99cdd6bd2a |
|
MD5 | f3a24437fa7ad6bf115e2d451b781084 |
|
BLAKE2b-256 | d87a1ff9b6e3ce7f63b788857760c92e4d0323982084244198235f993c05f93f |
Hashes for asyncdb-2.7.11-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c951a1d0e8e94e3699f2173787d99e140b7436c8e903216314533b0fe6732505 |
|
MD5 | e116bfc1318a8ef82f38d6a4ffc0ee99 |
|
BLAKE2b-256 | c42edb969a76f344bd0150c375fd94978d9de80e96b0c3cb7d23f0922377fb1b |
Hashes for asyncdb-2.7.11-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8fd1eb22ff4fb7d86f05db6df6710710228408add6cd5ff5c8d6b31be1a3b278 |
|
MD5 | 7ef036184d338666c548e2c4533f0f37 |
|
BLAKE2b-256 | 4be908624f8634ef75773e986c2d493ee84c1286e5410b37f1e4039002635a82 |
Hashes for asyncdb-2.7.11-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8140e8f9c5bd515504e66bd1c5cdecf29d7a1bc525a7a72553802996a34284ea |
|
MD5 | 61111f323239af92b4b425cf422a31b9 |
|
BLAKE2b-256 | 7ef13eabc25ed6ee9a22a4affeca0a5e555b0d3de70f51630ab5b7f619744f0b |
Hashes for asyncdb-2.7.11-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8215ba596ad0190f26338ff5f9b688b6d08d6c148dd6806ac47525b179c1be7f |
|
MD5 | 3164d902e44fca0c6e9fff334244b33a |
|
BLAKE2b-256 | a8a8e52be42061a891139295087436174b1bca9feee459d77c00bfd20554949d |
Hashes for asyncdb-2.7.11-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2d4ee198f4a2de2fda3d31cebf0e9a81fd9734d3296108bcf3d259467169e2bc |
|
MD5 | 3b748567ff1b1e7731556acf9891a8d9 |
|
BLAKE2b-256 | 4204a21565e7c0a63ae7feb29aeb74e41c83ff0f6355331b214751c295d5ee47 |