Library for Asynchronous data source connections Collection of asyncio drivers.
Project description
AsyncDB
AsyncDB is a collection of different Database Drivers using asyncio-based connections, binary-connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.
Why AsyncDB?
The finality of AsyncDB is to provide us a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is using asyncio-based technologies.
Getting Started
Requirements
Python 3.9+
Installation
$ pip install asyncdb
---> 100%
Successfully installed asyncdb
Can also install only drivers required like:
$ pip install asyncdb[pg] # this install only asyncpg
Or install all supported drivers as:
$ pip install asyncdb[all]
Requirements
- Python >= 3.8
- asyncio (https://pypi.python.org/pypi/asyncio/)
Currently AsyncDB supports the following databases:
- PostgreSQL (supporting two different connectors: asyncpg or aiopg)
- SQLite (requires aiosqlite)
- mySQL/MariaDB (requires aiomysql and mysqlclient)
- ODBC (using aioodbc)
- JDBC(using JayDeBeApi and JPype)
- RethinkDB (requires rethinkdb)
- Redis (requires aioredis)
- Memcache (requires aiomcache)
- MS SQL Server (non-asyncio using freeTDS and pymssql)
- Apache Cassandra (requires official cassandra driver)
- InfluxDB (using influxdb)
- CouchBase (using aiocouch)
- MongoDB (using motor)
- SQLAlchemy (requires sqlalchemy async (+3.14))
Quick Tutorial
from asyncdb import AsyncDB
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
# Or you can also passing a dictionary with parameters like:
params = {
"user": "user",
"password": "password",
"host": "localhost",
"port": "5432",
"database": "database",
"DEBUG": True,
}
db = AsyncDB('pg', params=params)
async with await db.connection() as conn:
result, error = await conn.query('SELECT * FROM test')
And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.
Every Driver has a simple name to call it:
- pg: AsyncPG (PostgreSQL)
- postgres: aiopg (PostgreSQL)
- mysql: aiomysql (mySQL)
- influx: influxdb (InfluxDB)
- redis: redis-py (Redis)
- mcache: aiomcache (Memcache)
- odbc: aiodbc (ODBC)
Future work:
- Prometheus
Output Support
With Output Support results can be returned into a wide-range of variants:
from datamodel import BaseModel
class Point(BaseModel):
col1: list
col2: list
col3: list
db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
# changing output format to Pandas:
conn.output_format('pandas') # change output format to pandas
result, error = await conn.query('SELECT * FROM test')
conn.output_format('csv') # change output format to CSV
result, _ = await conn.query('SELECT TEST')
conn.output_format('dataclass', model=Point) # change output format to Dataclass Model
result, _ = await conn.query('SELECT * FROM test')
Currently AsyncDB supports the following Output Formats:
- CSV (comma-separated or parametrized)
- JSON (using orjson)
- iterable (returns a generator)
- Recordset (Internal meta-Object for list of Records)
- Pandas (a pandas Dataframe)
- Datatable (Dt Dataframe)
- Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
- PySpark Dataframe
And others to come:
- Apache Arrow (using pyarrow)
- Polars (Using Python polars)
- Dask Dataframe
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for asyncdb-2.7.0-pp310-pypy310_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 53b80b48cbc18a61009f78aa4311ef65aa147c98d47c5791e43f0dd6f359898a |
|
MD5 | 588cfa781243308139a6f58d710a18ab |
|
BLAKE2b-256 | 52a7e05f5da03784c055293ecbce3a72c2073767538855d64a9be4abceca28a0 |
Hashes for asyncdb-2.7.0-pp39-pypy39_pp73-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9bde25ae6ee34aaa78685371b6055bbcf0b574d69f3d71a1c69de26b72fd30f1 |
|
MD5 | 62a970557ba03c8403ffd0a07c6630f6 |
|
BLAKE2b-256 | eb1b420c34af4f63476178b043599ae6171ab72d1f4c275008afb3e9b3d192cc |
Hashes for asyncdb-2.7.0-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c541c54fb4e3b00ccf38b0e750d705e1d14da3238139243e8caa61279c44fc7e |
|
MD5 | 7c1a6a97b014f57a86b4978b25f7eaaa |
|
BLAKE2b-256 | 4da9284708bb7d4c10c10e356f401e13612c1f5c62f7c7acd00ec493939347e1 |
Hashes for asyncdb-2.7.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d42dc8afd3b4678ece6f6bce54a3fd94b2c13866eebf3cfbbad250216d1e3ba5 |
|
MD5 | 3a5ad8eee38e20c4ecf5d52823196f65 |
|
BLAKE2b-256 | 0ca8a69c8c30bb64c67b1ec14fa2038bf8397b5a293a666a105836f6da423230 |
Hashes for asyncdb-2.7.0-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 12d613ff958c013d83e931c7c9bcc75654010866afe23140dd018e01aaaf4534 |
|
MD5 | ab13f2750a07db05196e87137d867ece |
|
BLAKE2b-256 | f172f9c83f7856c7de15072fd0a1373b6fc68931e2faff1bb9ff29679658b094 |
Hashes for asyncdb-2.7.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 60ce73391ac7a0607f3bb2daf5a9ec51e06f8f99cd527ac7fe74f0e32f67e9c6 |
|
MD5 | 8140e74f86521762eec49a63815f1373 |
|
BLAKE2b-256 | 36fe9c57f233b767a86ad728c4290832d759082dfb2b3c2f00f8d00ba6537e23 |
Hashes for asyncdb-2.7.0-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 203af89fb90fe7147f77ee9ba7f3bccca106015bf6616c1190cd0a9c33c56141 |
|
MD5 | 6f5575948d9231c8fbfbb9c27d1c1309 |
|
BLAKE2b-256 | 350b69968ef85802978d5c887b7f7913585afd222e3a6b8d4e0eeeb211e2e699 |
Hashes for asyncdb-2.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 56b7e1dce59bf4b412d88f9fa74ddb37e835024a6a1e4a9dea2cb66c353506fa |
|
MD5 | b0e67530036869c6788dd09159e29ae5 |
|
BLAKE2b-256 | 9cfb39f2029835e6ff22879086089e913042818c34d0c9808d070513e4efd821 |
Hashes for asyncdb-2.7.0-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f0c8e2113e52af54a188eb9608abc153a537217cbc0249d099dfd578076b0ef8 |
|
MD5 | 21d44daefd8e769e46ae4df7c3db89ce |
|
BLAKE2b-256 | daccc0939919f6f84c7b22f5dc913bd851d231e1782f5d6293375fd3480063bf |
Hashes for asyncdb-2.7.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b4a113c134d6e2f33e8e727af7a0b3308d02287daffe23d37e624c55891ca776 |
|
MD5 | a774c3f2ecde1b6e7b6a3af97a51d939 |
|
BLAKE2b-256 | 200b2ee88d591d6a38f6d91fbce94ce3fcd2ed7888600927d44e916082686c2c |