An asyncio PostgreSQL driver that returns numpy arrays
Project description
asyncpg-rkt is a fork of asyncpg, a database interface library designed specifically for PostgreSQL and Python/asyncio. asyncpg is an efficient, clean implementation of PostgreSQL server binary protocol for use with Python’s asyncio framework. You can read more about asyncpg in an introductory blog post.
asyncpg-rkt extends asyncpg as follows:
Backward compatible with the origin.
It is possible to set the numpy dtype for the fetched query.
Such “typed” queries return numpy arrays instead of lists of Record objects.
We construct numpy arrays directly from the low-level PostgreSQL protocol, without materializing any Python objects.
Although, we support object fields, too.
The time from receiving the response from PostgreSQL server until Connection.fetch() returns is ~20x less. This is because we avoid the overhead of dealing with Python objects in the result.
We return ravel()-ed indexes of nulls while writing NaN-s/NaT-s at the corresponding places in the array.
There is an option to return data by column vs. by row.
asyncpg-rkt provides the best performance when there are thousands of rows returned and the field types map to numpy.
Read the blog post with the introduction.
asyncpg-🚀 requires Python 3.8 or later and is supported for PostgreSQL versions 9.5 to 14. Older PostgreSQL versions or other databases implementing the PostgreSQL protocol may work, but are not being actively tested.
Documentation
The project documentation can be found here.
See below about how to use the fork’s special features.
Performance
In our testing asyncpg is, on average, 3x faster than psycopg2 (and its asyncio variant – aiopg).
The above results are a geometric mean of benchmarks obtained with PostgreSQL client driver benchmarking toolbench in November 2020 (click on the chart to see full details).
Further improvement from writing numpy arrays is ~20x:
Features
asyncpg implements PostgreSQL server protocol natively and exposes its features directly, as opposed to hiding them behind a generic facade like DB-API.
This enables asyncpg to have easy-to-use support for:
prepared statements
scrollable cursors
partial iteration on query results
automatic encoding and decoding of composite types, arrays, and any combination of those
straightforward support for custom data types
Installation
asyncpg-🚀 is available on PyPI and requires numpy 1.21+. Use pip to install:
$ pip install asyncpg-rkt
Basic Usage
import asyncio
import asyncpg
from asyncpg.rkt import set_query_dtype
import numpy as np
async def run():
conn = await asyncpg.connect(user='user', password='password',
database='database', host='127.0.0.1')
dtype = np.dtype([
("a", int),
("b", "datetime64[s]"),
])
array, nulls = await conn.fetch(
set_query_dtype('SELECT * FROM mytable WHERE id = $1', dtype),
10,
)
await conn.close()
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
License
asyncpg-🚀 is developed and distributed under the Apache 2.0 license, just like the original project.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for asyncpg_rkt-0.27.3-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5375dffd414f2bb53064afacfa738b2e0ca8435db41b96ca1ab7643921d3dae7 |
|
MD5 | 332004359dfa7d3f7605216827075922 |
|
BLAKE2b-256 | 67e8e017cc0a65469cb9867f8918ad3d2a0ed61d098832970e2c0455695912df |
Hashes for asyncpg_rkt-0.27.3-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c96755890cf818b40b087b8c9acdc8c1dd2b3dc13aa4b36296ba9ca89f420c8c |
|
MD5 | cb87a53679b2b98157fd19efba72a5af |
|
BLAKE2b-256 | fd77812e101c3bacf66738cbed14c23b5075ff7fc0c9fd197559e9f8ef409dc1 |
Hashes for asyncpg_rkt-0.27.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 33ff2f172414622f99947c8ef1825962111e9a58acc62d7ca1180044db1cf5b8 |
|
MD5 | 1ad0b588f4a578794b9eb9560ef3c3ed |
|
BLAKE2b-256 | f289863948155de3448832526e600ee5b076073ff001598af389f8e9b4ffc655 |
Hashes for asyncpg_rkt-0.27.3-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8d1dc0720509ae57a1827c75f7067a24227aef3ea9e757c6fc1b8066296cbaf9 |
|
MD5 | e48ff8ad3cab4a5cd515c6eaf763df68 |
|
BLAKE2b-256 | 324a648bd9121a2c7709e430923749e4e36e021cc9e3c5c98035fd5356716470 |
Hashes for asyncpg_rkt-0.27.3-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 168287edb9b9b9286ae481c646892317cd7f58664865a53341d519f28ec7d15d |
|
MD5 | a88ff637609b2878cf3239eff0875187 |
|
BLAKE2b-256 | 5c090eb9c136f0687b2bb09135ec812cc1eb8f36c0abcdaf011196c429cc7850 |
Hashes for asyncpg_rkt-0.27.3-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d714470e38305fe805a83ff9ee8056ef6d23dc88c18e28cfa8d86b0636d420d6 |
|
MD5 | da509c4e1bd61854ae8a66ec1a0a09e3 |
|
BLAKE2b-256 | 4717d8d8bc38480c34f9c02760684efa3cf8d1a125836d5fa383b70145770200 |
Hashes for asyncpg_rkt-0.27.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | de3186163c0dc042a9e099c0f08ec0d4f367b86dc501c00dc1f281b02bbe92ce |
|
MD5 | bfefadb4e9ab08874a530456c44aacd7 |
|
BLAKE2b-256 | a4792433a278febc82c8887dd1d26aa4a861a3d01c38751d653eccdb25371461 |
Hashes for asyncpg_rkt-0.27.3-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c293fc2094c07b8d01f13b120b2083515f9724f0742e79016006b516501c746c |
|
MD5 | 1fc802bcdbd78a51a121abf792dab704 |
|
BLAKE2b-256 | 45fff0e3fe8720756a45cb29a628d388cb68db0be55badd475713cb5060bcebc |
Hashes for asyncpg_rkt-0.27.3-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e1cb69641043a92e54a6a155a57a4f5babde74936853f8553aa6a64668c9c05 |
|
MD5 | cdb31e553af225f0b64ccce79da3350d |
|
BLAKE2b-256 | aa2cda46fa72113d1413da15fcbdb37eca10d05b60606fbb17ce42a11ed73c7d |
Hashes for asyncpg_rkt-0.27.3-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b9ca11382d3250ac9e28669329e0fbc18891c88063331086f229d9b2b6f4442e |
|
MD5 | ab4137849982d2fb4d7369d6cb833c3f |
|
BLAKE2b-256 | e83f07ba0d735b2fbfa981fe99d33d389cdb9b36b06b6dacc4de7657eab099cf |
Hashes for asyncpg_rkt-0.27.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0be243f8ee104fd447e3c8ac46694a85f84febddc54220774c1fad4d592b1f21 |
|
MD5 | c82bb06626f37ca201436bb50b1e9047 |
|
BLAKE2b-256 | 8478b2e3a8e571f72ed0e83b466508cf962cf1a282a9228efe29ee15e5d9d9bd |
Hashes for asyncpg_rkt-0.27.3-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee09b3d855a7fb97db3e5c51455080b2ee2f5f449dd29c26ca23615c8ed03854 |
|
MD5 | b0ef978b2881c6f2d2334d5b81d7a168 |
|
BLAKE2b-256 | 6a220d9691150fd89e45104fe333c41f33424a55629f126b2b3143ecb4b8f1ec |
Hashes for asyncpg_rkt-0.27.3-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a5675018955a2efb272c6313c916e4ec1e7a569f523a8cef752480e756534e1e |
|
MD5 | 495a14cfc7e0991b7a5bc5ed7ff422b3 |
|
BLAKE2b-256 | 8098abe7e4ffc2c32fede71727a53fb110ba003015c63fc8f032dc71e001168a |
Hashes for asyncpg_rkt-0.27.3-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 90e01f3f983607e65e7edbe315c6a0e1429ed97da3c6d13bc6179b0ca772795c |
|
MD5 | ca3dd9176dfb98d5317b174815615e0d |
|
BLAKE2b-256 | 0ab43fda468e1e248bb1fc6a4027ebace23d78a085aaa32770cd66ca547fc1fa |
Hashes for asyncpg_rkt-0.27.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2c5dfa236eefb2ff1c90d5662dca4f98521677ab9ab1ab01f6c84e26e07725e5 |
|
MD5 | 6529fbce9eaa88c53ac19b5db2156596 |
|
BLAKE2b-256 | 6cfca3973223a9880057cdac6df0ceff3c786ec0405b6d80c93b4e13220d26f6 |
Hashes for asyncpg_rkt-0.27.3-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0a093e33909f3ef82f25a6fff8006afb31894fa90c72d8f4a349b2d3de4b539c |
|
MD5 | c57920d2a9ed1c30fe5919e64b8500d2 |
|
BLAKE2b-256 | 3621804b522e1da22d64d2b55f1c92f44b6e5b8e503cb4a108cbb1f9ae62f8c7 |