Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.9+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.7.3-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cfb92731ce76d9a1cf870ce4384c9620ce77e4c5ffaf50381fc1372e1d75f572 |
|
MD5 | 72344341d8cea6a149aaa360f9a03f2e |
|
BLAKE2b-256 | a9208a0e1933b6f36f31e64b8433a3ac6e3ae43e315a30e0cfd2e4f140c62a5c |
Hashes for asyncdb-2.7.3-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 63c6ddbf20d31890acc5943efc0ada152846b5a85a87f7c79bf1c5b4db335a7a |
|
MD5 | bb8fff6ee4cef6af2dc5179db21f2cc9 |
|
BLAKE2b-256 | 4ac95f4d9010e13c79ecd39cc08e381989c070acfea445811aedde955990fbf0 |
Hashes for asyncdb-2.7.3-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9bad02a297b60d006ea28c194be0b45b6229af25480eb20b4ce9c6ec5e3b5f10 |
|
MD5 | 4fdf45a1173dc4c860f8677503aff0fc |
|
BLAKE2b-256 | a7d168c3e454d1ad566363a3523599c4104d5da181a1298c14c630082560d505 |
Hashes for asyncdb-2.7.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 11453d9a9574c09364652271e7dcddc584f17f63039d54f1eba20d332296ed3d |
|
MD5 | dab2dceea09eddbc17322043c61c71af |
|
BLAKE2b-256 | f7e5d6490378303fc2cea2e57eb96f9025ca414c1eab1de75252e3e3fbc27478 |
Hashes for asyncdb-2.7.3-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a971ca5c6aafcfadba8a9aa014105b943c8171da7adf7c87390837c6062da379 |
|
MD5 | 8a7687fa429f086eaf108ed76d03764d |
|
BLAKE2b-256 | 25c0b821254e78c358ad2ef38604f61dfe7173899c6356c9afb074ec1fd4d2da |
Hashes for asyncdb-2.7.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8aa1bdaee53c6994492be7dc065152f0e5d79c87a08db562f2ec012b7e0819f1 |
|
MD5 | 6eb85046e2138cf61e0675546deaa8e5 |
|
BLAKE2b-256 | 25a7e20b7b6a01efc0c9a7bf743b684f901cb076b962a14f9b3f9ff1fcf5d2c9 |
Hashes for asyncdb-2.7.3-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 76fe553126002dcc21da492a644f12ba3ef999c9e39be94400321e13c7948ddd |
|
MD5 | ea2286ed3ad15ff2fe12518d06fb9c7c |
|
BLAKE2b-256 | f04b9357a87b2f4336f9f7b5ebada4457f49445c42f74ad4cb3673807ddba83f |
Hashes for asyncdb-2.7.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e89aafac60136e2cd3432bbcd9421e5ba4f96aa6d2c5aef09784706ef0c1373b |
|
MD5 | 72ae2fa55acc2f59ed93854819fd68c7 |
|
BLAKE2b-256 | 015d017d3a961fcf618574421ac15f9fe14f77ad6c8ce61def0595e1f16ec779 |
Hashes for asyncdb-2.7.3-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a4ce2de7f32bbca2ce7aebc2c010a4af04cc18508cc12992e53586fc5e33050e |
|
MD5 | 089c0e31b14ad717182c361a2c363b4c |
|
BLAKE2b-256 | 48c471049128265ce2e6d3fdd87d1a3ad3e62a2b2c7e6fb1526d94f40bcbccfd |
Hashes for asyncdb-2.7.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 51aa73cd0572720ab7c1db73da6df73a57ba565e8c11fd1e90a992a98de89d40 |
|
MD5 | d1c5833771d36789d8cf08b42c939189 |
|
BLAKE2b-256 | 5d5c9af68bccab4c46b2b10cf21b2c0e7f10b6116c648deae35d1c116ea30367 |