Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.9+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.7.4-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f3ea3b52771631142b09ed01f3a3d403d37eb650a38a69e0bd7b21ecc46d4239 |
|
MD5 | f242d62822794155cfa233e660795657 |
|
BLAKE2b-256 | f0d2297bf96fbf7fe6ce958163bd9e6e7887e76ce2f409945455afb19d090578 |
Hashes for asyncdb-2.7.4-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | feeb6651d8fd687d5488e6e1442c1cec64e844cd3ed2b96cec756b5c65584f97 |
|
MD5 | 59373ef3cd08bd824fdee042abf04644 |
|
BLAKE2b-256 | 80b416d528d36a18bd9ec152f31f17347a4862d748b611da586492dd6e747bd0 |
Hashes for asyncdb-2.7.4-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd2ae2cf903811d62d8c047185749b6a753ff8685f1137567b205ba3ef57aa8d |
|
MD5 | 1d9b66bb90db6bc01d2ff6dff2edf7a9 |
|
BLAKE2b-256 | bf13f355a49d290b4c64e9b9380a1676a4940c52e587284de0cf0dc02086a180 |
Hashes for asyncdb-2.7.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 114876d050fb776484c3a09109161fef8d752c6bfc4cf3cea3f60d2db93a52d2 |
|
MD5 | 7b289864db5df4b2cdd2137c8b5bfe7a |
|
BLAKE2b-256 | 2051453a8ac0b31971450ecd3ad88311f850d3d40d208763a2b5563810a620b8 |
Hashes for asyncdb-2.7.4-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d4ca94b2c7cd46ba96b8eca9a0053dcc634ddbf6fe6089a0b78c0471faaea7fe |
|
MD5 | 7a765be476b934b2b375843d571efb97 |
|
BLAKE2b-256 | 64f44dbd5d4e8473b1ba745e237a0d08d51a8a362827ac5bb0a11729b09b5258 |
Hashes for asyncdb-2.7.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bac3faeb9dd1c79765bb89d323c8fc20dd3f800a5dfef57916cb7a9827191d5b |
|
MD5 | 76fc760840a1e480c73e122001d100d6 |
|
BLAKE2b-256 | d209c977c9410e6525789ce28db436731a6abde9bc7524365f813e5154fdbd7a |
Hashes for asyncdb-2.7.4-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8ba9f881d5db99a52b935fc5e937b905f2d76bd0041a5fd016620ce7b4aa1381 |
|
MD5 | 5654c1e531511d1679c504d607d4b3cf |
|
BLAKE2b-256 | 2a30f0c70e4e9e70c0f2465c21d2ee1a23dd9252e9cab594ab0d92bc93a5b3f2 |
Hashes for asyncdb-2.7.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e8b596ea9c65157583172391f2d52fba6987ab791dfd68bb05f22738f79732df |
|
MD5 | a143c2afaeef4921a316ddaf2384a29f |
|
BLAKE2b-256 | 64eb80518ae396a8fa2550a625d5e876e17d1064158d4d19ed59fc254a1c5783 |
Hashes for asyncdb-2.7.4-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bf1707bbb9141074e07b9e9576629becf5141ff2bc2609da9ff08028e396771d |
|
MD5 | 2b5bae28d7c69bc54c16cce0bcf81499 |
|
BLAKE2b-256 | 06861781aab76634e554ebcd942339e5be7e9c82c49fd7b14c7c7accddf6cef6 |
Hashes for asyncdb-2.7.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 36efb82f0711e9f686f572c6ce8b284f139c09fbe4fff81c9caa6c681d86ccd5 |
|
MD5 | 3882c603b7b3015b042ebf578ab4bcf1 |
|
BLAKE2b-256 | 48876965f070dc16eed5b6cd783fa2b591807b650f1e569a1a24374ff22febbb |