efa

Low-level Python bindings for AWS Elastic Fabric Adapter (EFA), including Scalable Reliable Datagram (SRD), one-sided RDMA, and GPUDirect transfers to and from torch CUDA tensors.

The package wraps libibverbs and EFA's libefa direct-verbs API in Cython. Its data path calls the provider's inline verbs directly, releases the GIL around blocking and posting operations, and does not import torch or link against CUDA.

  • No Python runtime dependencies.
  • SRD and UD queue pairs with SEND, RDMA read, and RDMA write operations.
  • Host buffers, CUDA device pointers, and dma-buf memory registration.
  • Classic and extended CQs, sender GID and unsolicited-write metadata, and direct EFA CQ, SQ, RQ, MR, and AH queries.
  • One Linux abi3 wheel for CPython 3.9 and newer.

Requirements

  • Linux on an AWS instance with one or more EFA devices attached.
  • libibverbs.so.1 and libefa.so.1 at runtime. The AWS EFA installer and current rdma-core distributions provide both.
  • The EFA device nodes available inside the process or container, normally /dev/infiniband/uverbs*.
  • A C compiler, Cython, and the rdma-core development headers only when building from source.

For GPUDirect, the EFA device and instance type must support RDMA read/write, and the NVIDIA driver must support dma-buf export or nvidia_peermem.

Install

pip install efa

To build from this checkout:

pip install "Cython>=3.0" "setuptools>=77" wheel
pip install ./efa

SRD Quickstart

SRD is reliable and connectionless. Each process creates a ready-to-send QP, exchanges a 24-byte EndpointInfo out of band, and resolves the remote GID to an address handle:

import numpy as np
import efa

dev = efa.get_efa_device_list()[0]
ctx = dev.open()
pd = ctx.alloc_pd()
cq = ctx.create_cq(256)
qp = pd.create_qp(
    efa.QPInitAttr(send_cq=cq, recv_cq=cq)
).prepare(qkey=0x1234)

local_info = efa.local_endpoint_info(qp, qkey=0x1234)
# Exchange local_info.to_bytes() with the other process.
remote_info = efa.EndpointInfo.from_bytes(remote_bytes)
peer = remote_info.peer(pd)

buf = np.zeros(4096, dtype=np.uint8)
access = (
    efa.AccessFlags.LOCAL_WRITE
    | efa.AccessFlags.REMOTE_WRITE
    | efa.AccessFlags.REMOTE_READ
)
mr = efa.reg_tensor(pd, buf, access)

qp.post_send(efa.SendWR(
    wr_id=1,
    sg_list=[mr.sge()],
    opcode=efa.WROpcode.SEND,
    send_flags=efa.SendFlags.SIGNALED,
    dest=peer,
))
for wc in cq.poll(16):
    wc.raise_for_status()

Every handle is an idempotent context manager. A QP retains its PD and CQs, an MR retains its PD and backing tensor, and an SGE retains the MR it addresses.

One-sided RDMA

EFA requires the responder to have an address handle for the requester before it accepts RDMA reads or writes. Both processes should therefore resolve the other process's EndpointInfo, even if traffic is currently one-way. A missing reverse AH completes with REM_OP_ERR and EFA vendor status 0x0e (REMOTE_ERROR_UNKNOWN_PEER).

Large buffers can be split at the device's max_rdma_size:

wrs = efa.write_wrs(local_mr, peer, remote_addr, remote_rkey)
qp.post_send(wrs)

read_wrs provides the corresponding RDMA-read operation. Every generated WR is signaled, as required by EFA.

GPUDirect With Torch

The optional efa.cuda module is torch-free and duck-types objects exposing data_ptr(), numel(), and element_size(). For torch's dma-buf path, enable VMM-backed allocations before CUDA initializes:

import os
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

import torch
import efa
import efa.cuda

src = torch.arange(1 << 20, dtype=torch.float32, device="cuda:0")
dst = torch.zeros_like(src)

src_mr = efa.cuda.register_tensor(pd, src, access)
dst_mr = efa.cuda.register_tensor(pd, dst, access)

torch.cuda.synchronize(src.device)
qp.post_send(efa.SendWR(
    wr_id=2,
    sg_list=[src_mr.sge()],
    opcode=efa.WROpcode.RDMA_WRITE,
    send_flags=efa.SendFlags.SIGNALED,
    remote_addr=remote_dst_addr,
    rkey=remote_dst_rkey,
    dest=peer,
))

After the receiver observes a completion or protocol-level write notification, it must order inbound NIC writes before CUDA consumes the destination:

with torch.cuda.device(dst.device):
    efa.cuda.flush_gpudirect_writes()

register_tensor first exports a dma-buf fd and calls ibv_reg_dmabuf_mr. If that path is unavailable, it falls back to ibv_reg_mr, which requires nvidia_peermem. The returned GpuMR retains the tensor allocation and keeps the actual CUDA virtual address because ibv_mr.addr is not meaningful for a dma-buf MR.

Direct EFA API

Area API
Device capabilities Context.query_efa_device
EFA CQ creation Context.create_cq_ex
Sender and unsolicited metadata CQEx.poll, WC.sgid, WC.unsolicited
CQ layout CQ.query_efa, CQEx.query_efa
SQ/RQ layout QP.query_wqs
MR interconnect IDs MR.query_efa
Address-handle number AH.ahn
SRD QP creation PD.create_qp with QPType.SRD

When using unsolicited RDMA write-with-immediate completions, create every communicating QP with QPInitAttr(..., unsolicited_write_recv=True) and use an extended CQ created with unsolicited=True. EFA requires peers to negotiate the same QP feature set.

The direct layout queries expose process-local addresses for advanced consumers. They do not transfer ownership of provider memory.

Testing

pip install -e "./efa[test,gpu]"
cd efa
pytest -rs

Tests marked integration exercise real EFA hardware. Tests marked gpu perform torch-verified GPU-to-GPU, GPU-to-host, and host-to-GPU transfers. Unavailable hardware capabilities are skipped explicitly. Set RDMA4PY_SKIP_HARDWARE_TESTS=1 to force all hardware-dependent tests to skip.

License

BSD-3-Clause. See LICENSE.

Project links

Data verified by PyPI on Aug 16, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2026.9.6)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for d4l3k from gravatar.com d4l3k

Credits

Author: Tristan Rice

GitHub Statistics

Data verified by PyPI on Aug 16, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

BSD-3-Clause
View SPDX License List

Requires

Python >=3.9

Provides Extra

test gpu

Tags

efa srd rdma ibverbs aws gpudirect networking

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

efa-2026.8.16.tar.gz (360.5 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

efa-2026.8.16-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ ARM64manylinux: glibc 2.28+ ARM64

efa-2026.8.16-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ x86-64manylinux: glibc 2.5+ x86-64

File details

Details for the file efa-2026.8.16.tar.gz.

File metadata

  • Download URL: efa-2026.8.16.tar.gz
  • Upload date:
  • Size: 360.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for efa-2026.8.16.tar.gz
Algorithm Hash digest
SHA256 06d6caf591e65d04270ae0fb87e100107b08221e218a633bc126057ad5b5a82c
MD5 6f3af6e9c6d10206e1cb34f7766fbbb4
BLAKE2b-256 2d2bd0dbd1d7bc28e3b498daabdac2cd7c477e17e7cfa7d8b4ba67bce622621f

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.8.16.tar.gz:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file efa-2026.8.16-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for efa-2026.8.16-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 6e870e82daeb61f78c2c0d082765812dc2e5e6ca9c5121058acc25a54f2c803a
MD5 e0f03911956b555cb5efc4784b218b83
BLAKE2b-256 85fd2cc8901608e16555858013e57680028a1ae26976c3b729cd88ecd584f738

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.8.16-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file efa-2026.8.16-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.

File metadata

File hashes

Hashes for efa-2026.8.16-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
Algorithm Hash digest
SHA256 7749cac945471fd9729b2727fa80322df3d7fbec54566b1830ec256f9d30bc2e
MD5 83abb87d9dd01934b766fdf3c47ad5df
BLAKE2b-256 aa1dd659ac62e905b29aace1ae35620ff81507b6bf3b355c86cb39cdfe7b04cf

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.8.16-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

2026.9.6

3 files

2026.9.5

3 files

2026.9.4

3 files

2026.9.3

3 files

2026.9.2

3 files

2026.9.1

3 files

2026.8.31

3 files

2026.8.30

3 files

2026.8.29

3 files

2026.8.28

3 files

2026.8.27

3 files

2026.8.26

3 files

2026.8.25

3 files

2026.8.24

3 files

2026.8.23

3 files

2026.8.22

3 files

2026.8.21

3 files

2026.8.20

3 files

2026.8.19

3 files

2026.8.18

3 files

2026.8.17

3 files

This release

2026.8.16 This release

3 files

2026.8.15

3 files

2026.8.14

3 files

2026.8.13

3 files

2026.8.12

3 files

2026.8.11

3 files

2026.8.10

3 files

2026.8.9

3 files

2026.8.8

3 files

2026.8.7

3 files

2026.8.6

3 files

2026.8.5

3 files

2026.8.4

3 files

2026.8.3

3 files

2026.8.2

3 files

2026.8.1

3 files

2026.7.31

3 files

2026.7.30

3 files

2026.7.29

3 files

2026.7.28

3 files

2026.7.27

3 files

2026.7.26

3 files

2026.7.25

3 files

2026.7.24

3 files

2026.7.23

3 files

2026.7.22

3 files

2026.7.21

3 files

2026.7.20

3 files

2026.7.19

3 files

2026.7.18

3 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page