Skip to main content

Library with some C and Cython code for speeding up common operations. This is externalized to avoid the hassle of building binary wheels in my other projects.

Project description

This library contains a number of functions for speeding up critical parts of your Python code without having to bother with the hassle of building binary extensions. That way you can keep your main packages simple PEP517 based packages and still get the speedups.

Currently only a few functions are available, but several more are planned.

Generic endian conversion functions in speedups.hton:

  • void pack_int16(char *buf, int16_t x)

  • void pack_int32(char *buf, int32_t x)

  • void pack_int64(char *buf, int64_t x)

  • void pack_float(char *buf, float f)

  • void pack_double(char *buf, double f)

  • int16_t unpack_int16(const char *buf)

  • uint16_t unpack_uint16(const char *buf)

  • int32_t unpack_int32(const char *buf)

  • uint32_t unpack_uint32(const char *buf)

  • int64_t unpack_int64(const char *buf)

  • uint64_t unpack_uint64(const char *buf)

  • float unpack_float(const char *buf)

  • double unpack_double(const char *buf)

These functions are used to convert between native and network byte order and are meant to be used from Cython code. Examples can be found in the speedups.psycopg_array code.

For the psycopg library we have a binary COPY loader to convert a PostgreSQL array to a numpy ndarray. This can be used with the copy() method of a psycopg cursor: https://www.psycopg.org/psycopg3/docs/basic/copy.html

It supports the following PostgreSQL types:

  • float4 (numpy.float32)

  • float8 (numpy.float64)

  • smallint (numpy.int16)

  • integer (numpy.int32)

  • bigint (numpy.int64)

Additionally, it supports arrays varying from 1D to N-D so a 2D or 3D array are supported.

cursor: psycopg.Cursor
psycopg_loaders.NumpyLoader.install(cursor)

query = '''
COPY (
    SELECT array_agg(x)
    FROM generate_series(1, 100000) x
) TO STDOUT WITH BINARY
'''

copy: psycopg.Copy
with cursor.copy(query) as copy:
    copy.set_types(['integer[]'])

    for row in copy.rows():
        print(row)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

speedups-1.2.0.tar.gz (168.8 kB view hashes)

Uploaded Source

Built Distributions

speedups-1.2.0-cp311-cp311-win_amd64.whl (256.1 kB view hashes)

Uploaded CPython 3.11 Windows x86-64

speedups-1.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (789.4 kB view hashes)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

speedups-1.2.0-cp311-cp311-macosx_10_9_x86_64.whl (270.1 kB view hashes)

Uploaded CPython 3.11 macOS 10.9+ x86-64

speedups-1.2.0-cp310-cp310-win_amd64.whl (255.3 kB view hashes)

Uploaded CPython 3.10 Windows x86-64

speedups-1.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (732.5 kB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

speedups-1.2.0-cp310-cp310-macosx_12_0_x86_64.whl (268.1 kB view hashes)

Uploaded CPython 3.10 macOS 12.0+ x86-64

speedups-1.2.0-cp310-cp310-macosx_10_9_x86_64.whl (269.7 kB view hashes)

Uploaded CPython 3.10 macOS 10.9+ x86-64

speedups-1.2.0-cp39-cp39-win_amd64.whl (255.8 kB view hashes)

Uploaded CPython 3.9 Windows x86-64

speedups-1.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (738.0 kB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

speedups-1.2.0-cp39-cp39-macosx_10_9_x86_64.whl (270.2 kB view hashes)

Uploaded CPython 3.9 macOS 10.9+ x86-64

speedups-1.2.0-cp38-cp38-win_amd64.whl (255.9 kB view hashes)

Uploaded CPython 3.8 Windows x86-64

speedups-1.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (744.7 kB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

speedups-1.2.0-cp38-cp38-macosx_10_9_x86_64.whl (269.8 kB view hashes)

Uploaded CPython 3.8 macOS 10.9+ x86-64

speedups-1.2.0-cp37-cp37m-win_amd64.whl (254.7 kB view hashes)

Uploaded CPython 3.7m Windows x86-64

speedups-1.2.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (695.7 kB view hashes)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

speedups-1.2.0-cp37-cp37m-macosx_10_9_x86_64.whl (269.5 kB view hashes)

Uploaded CPython 3.7m macOS 10.9+ x86-64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page