Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.8+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.6.22-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 76ca82588315c8e43dfa83056ac745d44c2eff0e689a5e1d105b8cdb97a1a589 |
|
MD5 | f70dce86b73328c633594ec6a990c737 |
|
BLAKE2b-256 | dc5f402bdd154d1c591c5fb8431d7d988e90102de52ed00ffb75696148deb9ec |
Hashes for asyncdb-2.6.22-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2181f8b21bb543a1ecd6b8f2808837b63352ec7407104425191c6b26dcf8af8c |
|
MD5 | 23c9dd80e34f068c6355ea20a6e815f5 |
|
BLAKE2b-256 | f439217fbd6f6b4a737463d02594b04bc1f6f57951f9d4ca43e4633a3af23be5 |
Hashes for asyncdb-2.6.22-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2bfe6955d7cf65ab61f1ef326cd99855eaf560306fb8c8a826dcc3e68b4e6637 |
|
MD5 | 583a8ef34582fc57264ce3517f35d2fc |
|
BLAKE2b-256 | 410ed741bc3d7c2b51de98b8d58cb31368f492f147fd870177619a85fccd216f |
Hashes for asyncdb-2.6.22-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 78541cf94e3f2f9161b7177fe1feeb17ef7dd7b660ec72376c3a62e15a24496d |
|
MD5 | 52bff06fa5ae78206f4878b7bd2b69c8 |
|
BLAKE2b-256 | 36ab00b83695dc07d79d9e7e451a6714aaafc9e0a584db55a7751c0dbfcec2a3 |
Hashes for asyncdb-2.6.22-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 71d73fd79db52af2893fab2de069714234248065c9251e0934d65944dd74e6a2 |
|
MD5 | d8bfbd5869016fc185e93743e4b2c1c0 |
|
BLAKE2b-256 | 5a3a781e31d558911c3ced63884df156c5d407d32b65f5347543797ea8e9f00a |
Hashes for asyncdb-2.6.22-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6e46a2281350ac825ecbac9df387442bba4fdc2e5695bbbe5494de34952be0f9 |
|
MD5 | 874c47be928ac6377eb43edd1b5fab52 |
|
BLAKE2b-256 | 42180bccf211d722b5969e473f033ebd24c9c12d7318f445d812aa8e5dc9eb20 |
Hashes for asyncdb-2.6.22-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 97710d100017d5e4af8a725e4a16f788d9f0d5432b0cee8c835ec06f161ec680 |
|
MD5 | f3a30f3bc4e9dc611ce86558240a4727 |
|
BLAKE2b-256 | 474d32004fb8ddf0bb7aa56e3a134462b30abfaeef189439216335e208202b4c |
Hashes for asyncdb-2.6.22-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 363c086f4aafc50de13516f8f9e62894ca31fe38aa3ea1f8de69e61c257a834f |
|
MD5 | b477ba615707e9f157a063e45e97c344 |
|
BLAKE2b-256 | 1b37d367db25537b857e3290c28dddfe2b3f92bf0a6d57bd9ff181a42c4d0438 |
Hashes for asyncdb-2.6.22-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6b983051df1ea05c177d5ebe091d0ec1e1abc4298ae66a7438a7ab5ff8ea7e4c |
|
MD5 | 7c8bdc667fe3afb45711f0f276899a09 |
|
BLAKE2b-256 | d9108b122d03ba87f890a33ead981f18cb14f2b87d9860b9dd6a0652790ea1af |
Hashes for asyncdb-2.6.22-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 08c0a5524f92e7702eb1164e16002da1e7b0a1621133b9141d5b09121df6dfad |
|
MD5 | b74f95f09ba5c634b3cdba838e54555f |
|
BLAKE2b-256 | 8730c1927ea13e7b57e753b24e2464509a3dd61a9a29171293a2ef515800ff78 |