linux-aio 0.4.1

Linux aio ABI wrapper

[english] | [한국어 (korean)]

linux_aio: Python wrapper for Linux Kernel AIO

Python wrapper module that uses Linux Kernel AIO directly

What is Linux Kernel AIO?

Linux IO Models table

In summary, it allows non-blocking and asynchronous use of blocking IO operations such as read(2) and write(2).

Related documents

• Linux Asynchronous I/O
• Linux Kernel AIO Design Notes
• How to use the Linux AIO feature (in C)

It is different from POSIX AIO

The POSIX AIO APIs have the aio_ prefix, but the Linux Kernel AIO has the io_ prefix.

There is already a POSIX AIO API for asynchronous I/O, but Linux implements it in glibc, a user-space library, which is supposed to use multi-threading internally. So, as you can see from the experiment below, it's much worse than using the blocking IO API.

Implementation & Structure

Package linux_aio

• Implemented based on linux_aio_bind package which is low-level binding of Linux kernel AIO.
• Unlike linux_aio_bind, it can be used without knowledge of ctypes
• Examples can be found in the code in the test directory.

Example

Examples can be found in the code in the test directory.

Notes & Limits

• Obviously available only on Linux
• Because it is a wrapper, it brings the constraints of Linux.
  • It can not be used for files used as a kernel interface. (e.g. cgroup)
  • Sometimes it works as Blocking.
    • There are some things that have been solved through development after posting.
  • Some features are being added because they are still under development.
  • There are also some features that are not supported when the Linux version is low
    • You need to check Linux man pages (4.16) and its related API documentation

Evaluation

Experiment script (requires python 3.7)

Setup

• Distribution: Ubuntu Server 16.04.5 LTS

• Linux: 4.19.0

• CPU: 2-way Intel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz

• MEM: total 64GB

• Storage: SK hynix SC300B SATA 512GB

• Python: 3.7.2 (Ubuntu ppa)

• Attempts to read a total of 1000 times in 1ms intervals.

• The file size varies from 1KB to 100KB, but it is small.

• Experiment with increasing the number of files read at the same time

• Because we have experimented with high-performance server, there may be larger performance differences when testing on a typical desktop.

Comparison target

• aiofiles - Uses Thread pool
• aiofile - Uses POSIX AIO
• libaio - Uses libaio
• python built-in open()

It is not a perfectly fair comparison.

aiofiles and aiofile are libraries that support asyncio. Since open() is blocking, there is a disadvantage that you can not do any other work while IO is going on. libaio and linux_aio are non-blocking, but must be polled.

Results

It may differ from environment to environment.

Runtime

• Unit: second

# of files	1	6	12	24
aiofiles	1.681	3.318	5.354	9.768
aiofile	1.543	1.958	2.493	3.737
libaio	1.311	1.344	1.362	1.423
open()	1.252	1.322	1.375	1.481
linux_aio	1.305	1.327	1.353	1.431

Threads

# of files	1	6	12	24
aiofiles	321	321	321	321
aiofile	3	8	15	26
libaio	1	1	1	1
open()	1	1	1	1
linux_aio	1	1	1	1

Memory

• Physical memory (Virtual memory)

# of files	1	6	12	24
aiofiles	21MB (22.6GB)	21MB (22.6GB)	21MB (22.6GB)	21MB (22.6GB)
aiofile	17MB (258MB)	17MB (654MB)	17MB (1080MB)	18MB (1949MB)
libaio	17MB (76MB)	17MB (76MB)	17MB (76MB)	17MB (76MB)
open()	17MB (76MB)	17MB (76MB)	17MB (76MB)	17MB (76MB)
linux_aio	17MB (76MB)	17MB (76MB)	17MB (76MB)	17MB (76MB)

CPU Utilization

# of files	1	6	12	24
aiofiles	42.8%	85.0%	102.2%	113.2%
aiofile	31.4%	52.4%	67.0%	84.0%
libaio	14.0%	16.0%	17.2%	20.6%
open()	13.4%	17.6%	21.0%	26.2%
linux_aio	13.0%	15.0%	16.0%	21.0%