Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.8+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: aioredis (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.6.18-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9affbae7603a5ff7df2c5ca91107891aca2e86327a0497b39216046ee44c5068 |
|
MD5 | ea602879ad747b79a44175c65f989e55 |
|
BLAKE2b-256 | 433f3c38cadd06b715904b48d5c0c36d6286d07ae0c3a2c1d41149a4b5cebac7 |
Hashes for asyncdb-2.6.18-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 657f1befea0cbe6e2076f946008dbb49eca0b514c69bc41c9f69300dc859642f |
|
MD5 | 9fa0fb817c2758b623c43f5a3257e5c0 |
|
BLAKE2b-256 | fe88a0cb980365718ccb5c95892dc26ed995efccda6ab1199971e0f3f78121fd |
Hashes for asyncdb-2.6.18-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 74e6063213025faea182e328bc410162f53462f1ea8dba994c3257b6b94ae69a |
|
MD5 | f71a741c3a6c56bdf6018f4b545a6d40 |
|
BLAKE2b-256 | 1847c71f519399e29b2250fcc05d7a6369e776d29216dcaa4d0f9b4e31ab219f |
Hashes for asyncdb-2.6.18-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 20f20a21e4c4f2e4356e0fa5ec219a0d63d25be42dda4466c868706bde7347ea |
|
MD5 | fc9a100bca2681a3bda099f9d3e85192 |
|
BLAKE2b-256 | 289674b0179d26cbaf20e6fa039a99a73fef1869146927e5b06497d7212f952b |
Hashes for asyncdb-2.6.18-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fb672ff151f07f08cede61ba1582255a2771f049cdff1bbd180575c9c2e20602 |
|
MD5 | 1fc218daa9ab4bf59e02bd7dd7b6e3e9 |
|
BLAKE2b-256 | aecb24c82e0f665b2db87bae505db2f9cd93218730616fea8a58374f5c16609f |
Hashes for asyncdb-2.6.18-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5273511ed89288ea6066b84b742db7423752c070d8b40193cc1396f2af14078b |
|
MD5 | 5c97465575546af94ed84f4d03017320 |
|
BLAKE2b-256 | 009c711507e7076b773211f62c5492de609e348c3b2e840f88a1f23d83836b1e |
Hashes for asyncdb-2.6.18-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cf84ecfd0ea9393acd9ffa35301ca960c9e689aecd30cb62fa655998f4cbc607 |
|
MD5 | f7d68044e89058ff6828526b0537438b |
|
BLAKE2b-256 | 4ff474c4ed317ecdf10d7c4a111ca47bb4e6f2680334fe2d2d69690b55bc1f87 |
Hashes for asyncdb-2.6.18-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e3dd6bbf5e59d8572057aa8bc5285ae94234798c9624b8be99e8dfe102fce3fc |
|
MD5 | 7a8a5128e26257535ad848825a6f71d6 |
|
BLAKE2b-256 | ed08527d74018d995c078340d8e23195ba6fdec63986a6e6e9bd5b8ed5c10532 |
Hashes for asyncdb-2.6.18-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2a79cc12a25e1b17fbcc773695c7584a388d19030e0218f4a2630d433e2734bd |
|
MD5 | 86877a9d0c389477303f55b8f5862459 |
|
BLAKE2b-256 | b535d5bc55ba8cbea64cd78f1256297bdf6288e14647904c6e98ce04caaebc56 |
Hashes for asyncdb-2.6.18-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 213c87cf80626faf4516be15c4d227a3b36a9390d0ae0437624837c6507c185f |
|
MD5 | e6fe58752807e775d3994d042f94d2be |
|
BLAKE2b-256 | 18194c7aec4ea330a38489920c98559b9ec3f0e0d8231c2b26cf8ac8df4b4a65 |