Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.8+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: aioredis (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.6.15-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dd6b5b74a58574df5ecfa5ff00f03c8ed705e3f0d9214e641b4041b91476e1e7 |
|
MD5 | da2e051108a6abe8b6389da8eafe5df5 |
|
BLAKE2b-256 | b28795cffb2919203f4711dd510baa390041749bced2a42183f516228f9d9f30 |
Hashes for asyncdb-2.6.15-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 57a0ded1530c6c4f3daa139c7fbb54b9976f7a9e3c3c6ab5bd73b418b034bd4a |
|
MD5 | 50cf793dedf7cd1a49b8da18590d0030 |
|
BLAKE2b-256 | 2818444e2a312243e877c61808ce94dda8ef7c6b07c9906434a22eeaf6668e55 |
Hashes for asyncdb-2.6.15-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 38380ff987b632bf8f615873d1783b438518fec5becfbe6a75dcbc49fb98ee6a |
|
MD5 | 67c360274b2fc64b291e9049ac1038c0 |
|
BLAKE2b-256 | f493f494f7c2c03c4867783c193f7c50e8db697aeebf2ecedc572e1e6fb37a82 |
Hashes for asyncdb-2.6.15-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 030b9fab6acaa4af74a8cb739e742cb4ad6774ea6e703c5bfa0cd1750d543bc3 |
|
MD5 | 8c7f8da6b17f18aff2ca8af61b92efa6 |
|
BLAKE2b-256 | 9a8455a09dec23224dc8e64bea98e858e1538ac98380ceaf850c190f95fb8c8b |
Hashes for asyncdb-2.6.15-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f009757a74fb186a1f7899ae40d062eee585411073252ba98305868b867c9093 |
|
MD5 | 0f69a93fb0e4a16b495d9d9ee6860137 |
|
BLAKE2b-256 | 2e882109fdb0877f8e68e693924f3aac317c4621e77030def3f6d8d296b2690f |
Hashes for asyncdb-2.6.15-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1e0c31374e4467a111a31baff091d44c6ae4cd44d5705e340abf3be835c62fef |
|
MD5 | 7cd2974db3b4d788bca9b7f67fe3230a |
|
BLAKE2b-256 | 658cce7417cb5c03cc6d7934b09b2bfdc8552e401cf19c4070176b1b8b7e09a8 |
Hashes for asyncdb-2.6.15-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0a3536af139590fd095172d4fbbab1d1cf4b2d9a682fedfbffc0581b5901e4e2 |
|
MD5 | aa62e04588a30b2a27397fc5592e8b4a |
|
BLAKE2b-256 | 9bfa64b2b96d52eff96e0082c9591dd65c7ae2824e5e6b9a058ee5a99d0318dc |
Hashes for asyncdb-2.6.15-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 98a2540f2c5041da63d737b301b8f6d2bb05fe6842cfe404a497d00e5c09bc26 |
|
MD5 | 73a1a30b8d6d30d8c10afb0ec5b9fb11 |
|
BLAKE2b-256 | 93f27a9417153b5c25ddabcb9d2d9a801979e89c2cff0cb9e32f87acb9ddb746 |
Hashes for asyncdb-2.6.15-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 32f6900a4f3a36b399814810bdabb1dd45405acf110562f084532097e7d456e0 |
|
MD5 | ef7835007adce3d37e64784846f1885e |
|
BLAKE2b-256 | 36b4d9f2cd4982254b1aa58fcc6cf2d01e46948fab0dc1fb760f7b2ec523fb84 |
Hashes for asyncdb-2.6.15-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8467e598163a017f1d50b5884d77b148449099bfb0117c49e85876b05f7d7fcb |
|
MD5 | 2c2cdd5c9b813f20c9286c53b2f27a6f |
|
BLAKE2b-256 | d63d4c7db66818df5a13cb87f8e252274db25e2381879bd5fbd275dd64bbcd26 |