linux-aio-bind 1.0.0

Python binding for Linux AIO

[english] | [한국어 (korean)]

linux_aio_bind: Python binding for Linux Kernel AIO

A low-level python binding module that matches the Linux kernel AIO system call.

If you are not using the ctypes module to develop high-level python modules directly, but you want to use AIO functionality in python, see High-level python wrapper.

What is Linux Kernel AIO?

Linux IO Models table

In summary, it allows non-blocking and asynchronous use of blocking IO operations such as read(2) and write(2).

Related documents

• Linux Asynchronous I/O
• Linux Kernel AIO Design Notes
• How to use the Linux AIO feature (in C)

It is different from POSIX AIO

The POSIX AIO APIs have the aio_ prefix, but the Linux Kernel AIO has the io_ prefix.

There is already a POSIX AIO API for asynchronous I/O, but Linux implements it in glibc, a user-space library, which is supposed to use multi-threading internally. So, as you can see from the experiment, it's much worse than using the blocking IO API.

Implementation & Structure

• ctypes module is used.
• It is defined to correspond 1:1 with the C header of Linux AIO.
  • Implemented 100% of the functionality when using C.
  • All the functions shown in the man page based on Linux man pages (4.16), and all the functions added in 4.20.3 source code, as far as I can find them.
• If you know how to use the ctypes module to operate on pointers, you can also build other types of wrappers based on this package.
  • e.g. High-level python wrapper
• It uses syscall for invoking ABI and cffi for gathering different syscall number by architecture on module installation.
  • refer the code
• python stub (pyi files - for type hint) are included.
• All error handling in the documentation (based on man-pages 4.16)

Example

Examples can be found in the code in the test directory.

Notes & Limits

• Obviously available only on Linux
• Because it is a wrapper, it brings the constraints of Linux.
  • It can not be used for files used as a kernel interface. (e.g. cgroup)
  • Sometimes it works as Blocking.
    • There are some things that have been solved through development after posting.
  • Some features are being added because they are still under development.
  • There are also some features that are not supported when the Linux version is low
    • You need to check Linux man pages (4.16) and its related API documentation
    • Poll is 4.19 or higher, fsync and fdsync require 4.18 or higher kernel