Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.8+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: aioredis (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.6.14-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 83ac7c7d83aee0652e2e7ab2c359fd1ceee49c62ad3b2f844ba6db75733ef5f6 |
|
MD5 | fb9df15419f7f919250b16ec19349bae |
|
BLAKE2b-256 | 4d18dee2e70ae9406f10c6dfb4734a71ba4da05bbc4a3e438fcb6889384f1daa |
Hashes for asyncdb-2.6.14-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3aa284820f325040ab8944b0c13997f88ae4c2b8eeaa279fd1dedc9bad63281f |
|
MD5 | 1b57b982753439d5301f13150f65012a |
|
BLAKE2b-256 | c368af682c5d7cd174706f3e000731aadc5c2497222ec9d5ab8cb8af2fb17652 |
Hashes for asyncdb-2.6.14-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 560b2a3168ef84ff3f3b8fe9a0e7226c6b6092567cd883ad4285a75d2426f8ec |
|
MD5 | 72c0e21fef89778966b327cd7cc229ad |
|
BLAKE2b-256 | fda6c2d89f2d765bc39102425f691e9fa70bf157a2fa1dc8fd6e1c66418779a0 |
Hashes for asyncdb-2.6.14-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a4db2fc3b65d6a0c1d15a0aa6790ca9d1cde4747ef9c62c2c8541a95bef7b9a6 |
|
MD5 | 147225caaddfe19870f7bbbad973bacf |
|
BLAKE2b-256 | 110ef6b0613f785f17581da528aaba6f42fed6bd632397a62f397d24fc58c4a7 |
Hashes for asyncdb-2.6.14-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cdadb2fd17b2756cadb39ddb59799a81f7d48cfc4bada889f0f67e3ef912bcb0 |
|
MD5 | 5f9e9eb9ed039d8c63dc7e85158958de |
|
BLAKE2b-256 | ba4096d3f18734699a4c378f1d4ecb4ee4bd32d160dd5669d1245bad366c7b2e |
Hashes for asyncdb-2.6.14-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 89378ec17d55461b08f9842ef71791a6b4071ab8712512f67181b24083088ba5 |
|
MD5 | f5abb98b1bc246cf58617205f6e4bfca |
|
BLAKE2b-256 | 74c6c73b6d4c2b8ffb1acb76e95b9844973755c837f6a54f427d7b734bb8c99d |
Hashes for asyncdb-2.6.14-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e17731435ac97644dce1b44302af830c7c4148ee167dc69c39ba7c46bf3a0bb6 |
|
MD5 | 0c1e2b796cba43c6bda5036faae66d97 |
|
BLAKE2b-256 | 454e3b853c46b679298e7599b71a9b74f71fde17f5cb7c461253c8a7895d6a6e |
Hashes for asyncdb-2.6.14-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7f24c90be9cc019fd35c3f92351b270ca47b8c3f0a7c0878f7d31d3dcf4d47d4 |
|
MD5 | b010461bfe173dc6f1fd2025dcf2bc5f |
|
BLAKE2b-256 | 5eaa88807b9cecf3790a77b859ecd88f615ed0fcf4781a0881f09c8de6f07d54 |
Hashes for asyncdb-2.6.14-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a049678247ce894284c190397b8fbecfde85d5022ec4040ebdccb89aa391be7c |
|
MD5 | 07b91a97e78dd946a797af133559ab45 |
|
BLAKE2b-256 | b8a609f094be415e0d2463daf8eb825cb273cf48a3b1b3826f2924c2d755cdd5 |
Hashes for asyncdb-2.6.14-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 256e5522abf9e3ec773a12ef09188830b3b2a460f7ca752506bfad44afc0cec9 |
|
MD5 | 837a7f80b3691f8feb925f8ada07798b |
|
BLAKE2b-256 | deec9ffabe6f6424ea566642a6089cf0f6c678d84dccccc743a0af48437b6bcc |