Skip to main content

Welcome to arrowdantic

Arrowdantic is a small Python library backed by a mature Rust implementation of Apache Arrow that can interoperate with

For simple (but data-heavy) data engineering tasks, this package essentially replaces pyarrow: it supports reading from and writing to Parquet, Arrow at the same or higher performance and higher safety (e.g. no segfaults).

Furthermore, it supports reading from and writing to ODBC compliant databases at the same or higher performance than turbodbc.

This package is also suitable for environments such as AWS Lambda functions. It takes 13M of disk space, compared to 82M taken by pyarrow.

Features

  • declare and access Arrow-backed arrays (integers, floats, boolean, string, binary)
  • read from and write to Apache Arrow IPC file
  • read from and write to Apache Parquet
  • read from and write to ODBC-compliant databases (e.g. postgres, mongoDB)

Examples

Use parquet

import io
import arrowdantic as ad

original_arrays = [ad.UInt32Array([1, None])]

schema = ad.Schema(
    [ad.Field(f"c{i}", array.type, True) for i, array in enumerate(original_arrays)]
)

data = io.BytesIO()
with ad.ParquetFileWriter(data, schema) as writer:
    writer.write(ad.Chunk(original_arrays))
data.seek(0)

reader = ad.ParquetFileReader(data)
chunk = next(reader)
assert chunk.arrays() == original_arrays

Use Arrow files

import arrowdantic as ad

original_arrays = [ad.UInt32Array([1, None])]

schema = ad.Schema(
    [ad.Field(f"c{i}", array.type, True) for i, array in enumerate(original_arrays)]
)

import io

data = io.BytesIO()
with ad.ArrowFileWriter(data, schema) as writer:
    writer.write(ad.Chunk(original_arrays))
data.seek(0)

reader = ad.ArrowFileReader(data)
chunk = next(reader)
assert chunk.arrays() == original_arrays

Use ODBC

import arrowdantic as ad


arrays = [ad.Int32Array([1, None]), ad.StringArray(["aa", None])]

with ad.ODBCConnector(r"Driver={SQLite3};Database=sqlite-test.db") as con:
    # create an empty table with a schema
    con.execute("DROP TABLE IF EXISTS example;")
    con.execute("CREATE TABLE example (c1 INT, c2 TEXT);")

    # insert the arrays
    con.write("INSERT INTO example (c1, c2) VALUES (?, ?)", ad.Chunk(arrays))

    # read the arrays
    with con.execute("SELECT c1, c2 FROM example", 1024) as chunks:
        assert chunks.fields() == [
            ad.Field("c1", ad.DataType.int32(), True),
            ad.Field("c2", ad.DataType.string(), True),
        ]
        chunk = next(chunks)
assert chunk.arrays() == arrays

Metadata

Release files for arrowdantic 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for arrowdantic 0.1.0
File Size Uploaded Provenance Actions
arrowdantic-0.1.0.tar.gz 13.5 kB Without provenance Details for arrowdantic-0.1.0.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for arrowdantic 0.1.0
File Actions
arrowdantic-0.1.0-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl PyPy 3.7 PyPy 3.7 7.3 Linux glibc 2.12+ x86-64 Details for arrowdantic-0.1.0-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
arrowdantic-0.1.0-cp310-none-win_amd64.whl CPython 3.10 none Windows x86-64 Details for arrowdantic-0.1.0-cp310-none-win_amd64.whl
arrowdantic-0.1.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.12+ x86-64 Details for arrowdantic-0.1.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
arrowdantic-0.1.0-cp310-cp310-macosx_10_7_x86_64.whl CPython 3.10 CPython 3.10 macOS 10.7+ x86-64 Details for arrowdantic-0.1.0-cp310-cp310-macosx_10_7_x86_64.whl
arrowdantic-0.1.0-cp39-none-win_amd64.whl CPython 3.9 none Windows x86-64 Details for arrowdantic-0.1.0-cp39-none-win_amd64.whl
arrowdantic-0.1.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl CPython 3.9 CPython 3.9 Linux glibc 2.12+ x86-64 Details for arrowdantic-0.1.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
arrowdantic-0.1.0-cp39-cp39-macosx_10_7_x86_64.whl CPython 3.9 CPython 3.9 macOS 10.7+ x86-64 Details for arrowdantic-0.1.0-cp39-cp39-macosx_10_7_x86_64.whl
arrowdantic-0.1.0-cp38-none-win_amd64.whl CPython 3.8 none Windows x86-64 Details for arrowdantic-0.1.0-cp38-none-win_amd64.whl
arrowdantic-0.1.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl CPython 3.8 CPython 3.8 Linux glibc 2.12+ x86-64 Details for arrowdantic-0.1.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
arrowdantic-0.1.0-cp38-cp38-macosx_10_7_x86_64.whl CPython 3.8 CPython 3.8 macOS 10.7+ x86-64 Details for arrowdantic-0.1.0-cp38-cp38-macosx_10_7_x86_64.whl
arrowdantic-0.1.0-cp37-none-win_amd64.whl CPython 3.7 none Windows x86-64 Details for arrowdantic-0.1.0-cp37-none-win_amd64.whl
arrowdantic-0.1.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.12+ x86-64 Details for arrowdantic-0.1.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
arrowdantic-0.1.0-cp37-cp37m-macosx_10_7_x86_64.whl CPython 3.7 CPython 3.7 pymalloc macOS 10.7+ x86-64 Details for arrowdantic-0.1.0-cp37-cp37m-macosx_10_7_x86_64.whl

Total release size: 34.5 MB

Release files / arrowdantic-0.1.0.tar.gz

Download URL arrowdantic-0.1.0.tar.gz
Size 13.5 kB
Tags Source
SHA-256 checksum
How to use checksums
3438ec15b1a6e138b612b969ca5a8ab946357f07ffea55e558655acc23a05b0f
BLAKE2b-256 checksum
How to use checksums
64d402c66380615d020cc5f8664301e270c09b2a1d6ff120b9cf7e027f9fb4f9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl

Download URL arrowdantic-0.1.0-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Size 2.9 MB
Tags Linux glibc 2.12+ x86-64 PyPy 3.7 PyPy 3.7 7.3
SHA-256 checksum
How to use checksums
9c54de33afcb37970fc6fcb1d8bc5f679ec569f75b731d33bb78d55e8a35d921
BLAKE2b-256 checksum
How to use checksums
195ca5ef176ccd9f0ab7b55242fe85b8e1dee1d28ad32fdb4819916521f320a7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp310-none-win_amd64.whl

Download URL arrowdantic-0.1.0-cp310-none-win_amd64.whl
Size 2.4 MB
Tags CPython 3.10 Windows x86-64
SHA-256 checksum
How to use checksums
64b76ad2fc505c980a1e92c378c4d0b82f37a64c1c450ccc868e2822bb0e2e6e
BLAKE2b-256 checksum
How to use checksums
88386080a9d9de852cdb61f00c8ee579e0f54944aa91b2e30b8a06529238cfc0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl

Download URL arrowdantic-0.1.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Size 2.9 MB
Tags CPython 3.10 Linux glibc 2.12+ x86-64
SHA-256 checksum
How to use checksums
ba966ab8e75fe9faa6bf040fffce0f8542ea139be4590bff30720be24be36ddb
BLAKE2b-256 checksum
How to use checksums
9fc699565c71213560b61d11e5ac4b4ee9ad8076e58192896aad534b6129a21c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp310-cp310-macosx_10_7_x86_64.whl

Download URL arrowdantic-0.1.0-cp310-cp310-macosx_10_7_x86_64.whl
Size 2.6 MB
Tags CPython 3.10 macOS 10.7+ x86-64
SHA-256 checksum
How to use checksums
419c60d9fdad011da6edabd0948f00bf3517d5fe7c6ef4ecf782d99b82a6b4b6
BLAKE2b-256 checksum
How to use checksums
f0861c68b071728287f86aa29b6b7ad6da9bc279c2d5f4d3122e55fbe6d09190
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp39-none-win_amd64.whl

Download URL arrowdantic-0.1.0-cp39-none-win_amd64.whl
Size 2.4 MB
Tags CPython 3.9 Windows x86-64
SHA-256 checksum
How to use checksums
49b13b2868fbbe87e63f85fa5db5192c969658de5d5bf6904e0d6e40ae33ac73
BLAKE2b-256 checksum
How to use checksums
295b0ceeb7e4277173421b99373aeb0d4f90e989f996ca21c67e74b34aaaccc2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl

Download URL arrowdantic-0.1.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Size 2.9 MB
Tags CPython 3.9 Linux glibc 2.12+ x86-64
SHA-256 checksum
How to use checksums
c025012d7a377cffd512cace0e88c16e54d6877bee06f6b4a174313f31123b68
BLAKE2b-256 checksum
How to use checksums
ab1c7325d2b03616c0d0843b6cc44830a544e02d12f119871ef9c970bfcdfc2e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp39-cp39-macosx_10_7_x86_64.whl

Download URL arrowdantic-0.1.0-cp39-cp39-macosx_10_7_x86_64.whl
Size 2.6 MB
Tags CPython 3.9 macOS 10.7+ x86-64
SHA-256 checksum
How to use checksums
4baaaa2eec98b7ef232eb40dd1830991f2a7ea918f6d5fb55484e4ddb9093ee6
BLAKE2b-256 checksum
How to use checksums
544d88652ca0f2034994aad2161dc92c856dbf122ee145f15c3f54f59b078107
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp38-none-win_amd64.whl

Download URL arrowdantic-0.1.0-cp38-none-win_amd64.whl
Size 2.4 MB
Tags CPython 3.8 Windows x86-64
SHA-256 checksum
How to use checksums
679c1b649cb907d570dbb910aeb135f8c67d315a701ae455299a4d8cf5f41e66
BLAKE2b-256 checksum
How to use checksums
d01e61b7a681d6b09bab83d0ba51057208a2d76397a5a46bede0e22491d2a1f2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl

Download URL arrowdantic-0.1.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Size 2.9 MB
Tags CPython 3.8 Linux glibc 2.12+ x86-64
SHA-256 checksum
How to use checksums
7bab9e92b57f31dc84ba78defc54784fa4c01d38498d20df4f7e39b9774fd904
BLAKE2b-256 checksum
How to use checksums
85d7eb2418ad0087d528c262b6553f4dfe567b7a514dfea8daec0a50c8779d76
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp38-cp38-macosx_10_7_x86_64.whl

Download URL arrowdantic-0.1.0-cp38-cp38-macosx_10_7_x86_64.whl
Size 2.6 MB
Tags CPython 3.8 macOS 10.7+ x86-64
SHA-256 checksum
How to use checksums
5241db27cd111e53962946b0fc7e3441116ef63fd11f3762f87f62dc92972fce
BLAKE2b-256 checksum
How to use checksums
49a004a90ac5474cf33ef782fdc5147b4cbf2a3f301b7ea385d75450a1587c2c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp37-none-win_amd64.whl

Download URL arrowdantic-0.1.0-cp37-none-win_amd64.whl
Size 2.4 MB
Tags CPython 3.7 Windows x86-64
SHA-256 checksum
How to use checksums
cc9d25e2a791f40f7dd142d9911c8b0e998172a4ce997092885097c8803418c2
BLAKE2b-256 checksum
How to use checksums
3d9e4857ddfdfc95b0e0e35454df202fc99158c0c9a87746ed43789217bb7cf0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl

Download URL arrowdantic-0.1.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Size 2.9 MB
Tags CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.12+ x86-64
SHA-256 checksum
How to use checksums
353f9238a99da65e26cb7574d975c876b139efb78c5f2f3e3f360e2fd794b5c7
BLAKE2b-256 checksum
How to use checksums
a5814d9525fa52a92247e9b266a1a0a5374a5f05d652db0af54e750b3bafad3e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Release files / arrowdantic-0.1.0-cp37-cp37m-macosx_10_7_x86_64.whl

Download URL arrowdantic-0.1.0-cp37-cp37m-macosx_10_7_x86_64.whl
Size 2.6 MB
Tags CPython 3.7 CPython 3.7 pymalloc macOS 10.7+ x86-64
SHA-256 checksum
How to use checksums
830561f80a250a45258ed4c5a1fc62a53791e013c8670a564f44e0185e7aac92
BLAKE2b-256 checksum
How to use checksums
3b89c5bb6b12370c59fe8d3ef826332a310bfc79ca8087c29af5a257e3715660
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.0 CPython/3.9.12

Security for arrowdantic 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page