Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.9+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.7.7-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8accb1b26647ad36a007c93dc5603e52309a8bb7409d96771e027c3ca272621b |
|
MD5 | 69227e9d130e9b46aef9e0f4872206cf |
|
BLAKE2b-256 | 38cb68e4848a66511a22c5b88024e345d04b03a018ceb45db173b8e4d32b11bb |
Hashes for asyncdb-2.7.7-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | af5679086e0b03d0b3d14af45c6865b99b00526f1c15303b5be6b8a1e1737cf7 |
|
MD5 | 036dd44017526dee2db8ea74a533f6b8 |
|
BLAKE2b-256 | c68afde8481797b76c223599dc4442c874037b320a013892ca5e353f977f4fe2 |
Hashes for asyncdb-2.7.7-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e19ec8a8155172ac3ee58c70f2f232d1b89b0eb390f4212309edb47f25b700a |
|
MD5 | a9af934b65ec3e7bdaa14fcb3e5ef4ad |
|
BLAKE2b-256 | f2773bed79ebbaccc2dc089f74dc36e75cb4b1ff3b5f23db5610a909e02b6a17 |
Hashes for asyncdb-2.7.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0e35367238eab13a2e3b789afa6c19f6693f949ff3844fdfc1356883e78e470e |
|
MD5 | 940af5633c36a2d5477ba4656aa1ad71 |
|
BLAKE2b-256 | 87e38045571bb7d0eb51957f00683446a17927fcad9c668ff1e9d7260bd9ccfc |
Hashes for asyncdb-2.7.7-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 89cd02af1a6ebb5c219a8f70e6df1aa309b6b2495aa29b539dae71e99dc2453c |
|
MD5 | 7aa466d965517bd5c3454db6e97c5e2e |
|
BLAKE2b-256 | f570c0b5c2b884f22e9dd9699e22129346837563371e90478fbdd186b1d67673 |
Hashes for asyncdb-2.7.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8fa50269d0812d7e321c5fe56d98091cefc1bb092f87319733a2ce6d7144ef6d |
|
MD5 | c98d719903aa0d4c76059c0cb0e5aff4 |
|
BLAKE2b-256 | 14f944766288b63b761c21df57af926a202954bc7fed31346635832143594a11 |
Hashes for asyncdb-2.7.7-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d39cd71890d43c69b4ae536937781670b9b8dca0faf1a8b25fa4f48ebe9e8484 |
|
MD5 | 281fae706e51fba926a4298fce92602a |
|
BLAKE2b-256 | 5aa699b52ccc195e21e34c1a27550b534f2d030d85a66168bd7c58b7131a280c |
Hashes for asyncdb-2.7.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 578c5052928ca24791fb49cf1e6800613755b4d0456c86fbb967ba4977446fba |
|
MD5 | b901d2dc7f1ea8b9c269e9f6f7194dda |
|
BLAKE2b-256 | b39fca5e19e840348f0944d2b55b2fb666dcc79c664fbd117200d72ec74b494f |
Hashes for asyncdb-2.7.7-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ba3e63a4052f84733e69d27a49c6ed860c7c5c7b3e2f9a4624dd51b2c8ebc915 |
|
MD5 | 90c09c7d7485ecb1343e5da9ca45ab3d |
|
BLAKE2b-256 | a794a7ff641b16ffb8ede6178744735071b78d0a31f0cc4626ceadbee7402848 |
Hashes for asyncdb-2.7.7-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e613b8c529cddf14b4143eb9c365fc465eea5c327832c75002fb6f8de1fd6624 |
|
MD5 | 79197bf45971a324319e3791efc2ece4 |
|
BLAKE2b-256 | 09e2c8212c1b61ca1e15d3c352bdd7ffc3140f1dc1757e1057472e446333991b |