efa

Low-level Python bindings for AWS Elastic Fabric Adapter (EFA), including Scalable Reliable Datagram (SRD), one-sided RDMA, and GPUDirect transfers to and from torch CUDA tensors.

The package wraps libibverbs and EFA's libefa direct-verbs API in Cython. Its data path calls the provider's inline verbs directly, releases the GIL around blocking and posting operations, and does not import torch or link against CUDA.

  • No Python runtime dependencies.
  • SRD and UD queue pairs with SEND, RDMA read, and RDMA write operations.
  • Host buffers, CUDA device pointers, and dma-buf memory registration.
  • Classic and extended CQs, sender GID and unsolicited-write metadata, and direct EFA CQ, SQ, RQ, MR, and AH queries.
  • One Linux abi3 wheel for CPython 3.9 and newer.

Requirements

  • Linux on an AWS instance with one or more EFA devices attached.
  • libibverbs.so.1 and libefa.so.1 at runtime. The AWS EFA installer and current rdma-core distributions provide both.
  • The EFA device nodes available inside the process or container, normally /dev/infiniband/uverbs*.
  • A C compiler, Cython, and the rdma-core development headers only when building from source.

For GPUDirect, the EFA device and instance type must support RDMA read/write, and the NVIDIA driver must support dma-buf export or nvidia_peermem.

Install

pip install efa

To build from this checkout:

pip install "Cython>=3.0" "setuptools>=77" wheel
pip install ./efa

SRD Quickstart

SRD is reliable and connectionless. Each process creates a ready-to-send QP, exchanges a 24-byte EndpointInfo out of band, and resolves the remote GID to an address handle:

import numpy as np
import efa

dev = efa.get_efa_device_list()[0]
ctx = dev.open()
pd = ctx.alloc_pd()
cq = ctx.create_cq(256)
qp = pd.create_qp(
    efa.QPInitAttr(send_cq=cq, recv_cq=cq)
).prepare(qkey=0x1234)

local_info = efa.local_endpoint_info(qp, qkey=0x1234)
# Exchange local_info.to_bytes() with the other process.
remote_info = efa.EndpointInfo.from_bytes(remote_bytes)
peer = remote_info.peer(pd)

buf = np.zeros(4096, dtype=np.uint8)
access = (
    efa.AccessFlags.LOCAL_WRITE
    | efa.AccessFlags.REMOTE_WRITE
    | efa.AccessFlags.REMOTE_READ
)
mr = efa.reg_tensor(pd, buf, access)

qp.post_send(efa.SendWR(
    wr_id=1,
    sg_list=[mr.sge()],
    opcode=efa.WROpcode.SEND,
    send_flags=efa.SendFlags.SIGNALED,
    dest=peer,
))
for wc in cq.poll(16):
    wc.raise_for_status()

Every handle is an idempotent context manager. A QP retains its PD and CQs, an MR retains its PD and backing tensor, and an SGE retains the MR it addresses.

One-sided RDMA

EFA requires the responder to have an address handle for the requester before it accepts RDMA reads or writes. Both processes should therefore resolve the other process's EndpointInfo, even if traffic is currently one-way. A missing reverse AH completes with REM_OP_ERR and EFA vendor status 0x0e (REMOTE_ERROR_UNKNOWN_PEER).

Large buffers can be split at the device's max_rdma_size:

wrs = efa.write_wrs(local_mr, peer, remote_addr, remote_rkey)
qp.post_send(wrs)

read_wrs provides the corresponding RDMA-read operation. Every generated WR is signaled, as required by EFA.

GPUDirect With Torch

The optional efa.cuda module is torch-free and duck-types objects exposing data_ptr(), numel(), and element_size(). For torch's dma-buf path, enable VMM-backed allocations before CUDA initializes:

import os
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

import torch
import efa
import efa.cuda

src = torch.arange(1 << 20, dtype=torch.float32, device="cuda:0")
dst = torch.zeros_like(src)

src_mr = efa.cuda.register_tensor(pd, src, access)
dst_mr = efa.cuda.register_tensor(pd, dst, access)

torch.cuda.synchronize(src.device)
qp.post_send(efa.SendWR(
    wr_id=2,
    sg_list=[src_mr.sge()],
    opcode=efa.WROpcode.RDMA_WRITE,
    send_flags=efa.SendFlags.SIGNALED,
    remote_addr=remote_dst_addr,
    rkey=remote_dst_rkey,
    dest=peer,
))

After the receiver observes a completion or protocol-level write notification, it must order inbound NIC writes before CUDA consumes the destination:

with torch.cuda.device(dst.device):
    efa.cuda.flush_gpudirect_writes()

register_tensor first exports a dma-buf fd and calls ibv_reg_dmabuf_mr. If that path is unavailable, it falls back to ibv_reg_mr, which requires nvidia_peermem. The returned GpuMR retains the tensor allocation and keeps the actual CUDA virtual address because ibv_mr.addr is not meaningful for a dma-buf MR.

Direct EFA API

Area API
Device capabilities Context.query_efa_device
EFA CQ creation Context.create_cq_ex
Sender and unsolicited metadata CQEx.poll, WC.sgid, WC.unsolicited
CQ layout CQ.query_efa, CQEx.query_efa
SQ/RQ layout QP.query_wqs
MR interconnect IDs MR.query_efa
Address-handle number AH.ahn
SRD QP creation PD.create_qp with QPType.SRD

When using unsolicited RDMA write-with-immediate completions, create every communicating QP with QPInitAttr(..., unsolicited_write_recv=True) and use an extended CQ created with unsolicited=True. EFA requires peers to negotiate the same QP feature set.

The direct layout queries expose process-local addresses for advanced consumers. They do not transfer ownership of provider memory.

Testing

pip install -e "./efa[test,gpu]"
cd efa
pytest -rs

Tests marked integration exercise real EFA hardware. Tests marked gpu perform torch-verified GPU-to-GPU, GPU-to-host, and host-to-GPU transfers. Unavailable hardware capabilities are skipped explicitly. Set RDMA4PY_SKIP_HARDWARE_TESTS=1 to force all hardware-dependent tests to skip.

License

BSD-3-Clause. See LICENSE.

Project links

Data verified by PyPI on Aug 4, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2026.9.6)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for d4l3k from gravatar.com d4l3k

Credits

Author: Tristan Rice

GitHub Statistics

Data verified by PyPI on Aug 4, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

BSD-3-Clause
View SPDX License List

Requires

Python >=3.9

Provides Extra

test gpu

Tags

efa srd rdma ibverbs aws gpudirect networking

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

efa-2026.8.4.tar.gz (360.5 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

efa-2026.8.4-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ ARM64manylinux: glibc 2.28+ ARM64

efa-2026.8.4-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ x86-64manylinux: glibc 2.5+ x86-64

File details

Details for the file efa-2026.8.4.tar.gz.

File metadata

  • Download URL: efa-2026.8.4.tar.gz
  • Upload date:
  • Size: 360.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for efa-2026.8.4.tar.gz
Algorithm Hash digest
SHA256 ef6553e5067cb51b705312396a8f2bcd783e0274655b6fb396dff7145625bc19
MD5 61eb04e8cab20a701739888cd4cacea5
BLAKE2b-256 ef40abe4753ccafe2919613c90794a35338a15bb04d6ea5990bd313d98b85270

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.8.4.tar.gz:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file efa-2026.8.4-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for efa-2026.8.4-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 5c8c80c4e203c7cbaf60a2dc0bb78a185efbb76e85b0fb7dcfab1ae3d8b0363c
MD5 5731a2d19039c48d097b2d21e813960b
BLAKE2b-256 362951c7d60fc376b88bf2b34c7320266a55d10b84f7c69a8b4253f807f56973

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.8.4-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file efa-2026.8.4-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.

File metadata

File hashes

Hashes for efa-2026.8.4-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
Algorithm Hash digest
SHA256 96291ffc4aeb29a2950f0f1a2ba130edc8d983a46488468d5bdbbc986d195646
MD5 f3a7fa6212a69167f8a22e8fe0d8d821
BLAKE2b-256 d89abbff494131da3bba7a2deb4094a97905380b5593793805435b7d5c9c17c0

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.8.4-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

2026.9.6

3 files

2026.9.5

3 files

2026.9.4

3 files

2026.9.3

3 files

2026.9.2

3 files

2026.9.1

3 files

2026.8.31

3 files

2026.8.30

3 files

2026.8.29

3 files

2026.8.28

3 files

2026.8.27

3 files

2026.8.26

3 files

2026.8.25

3 files

2026.8.24

3 files

2026.8.23

3 files

2026.8.22

3 files

2026.8.21

3 files

2026.8.20

3 files

2026.8.19

3 files

2026.8.18

3 files

2026.8.17

3 files

2026.8.16

3 files

2026.8.15

3 files

2026.8.14

3 files

2026.8.13

3 files

2026.8.12

3 files

2026.8.11

3 files

2026.8.10

3 files

2026.8.9

3 files

2026.8.8

3 files

2026.8.7

3 files

2026.8.6

3 files

2026.8.5

3 files

This release

2026.8.4 This release

3 files

2026.8.3

3 files

2026.8.2

3 files

2026.8.1

3 files

2026.7.31

3 files

2026.7.30

3 files

2026.7.29

3 files

2026.7.28

3 files

2026.7.27

3 files

2026.7.26

3 files

2026.7.25

3 files

2026.7.24

3 files

2026.7.23

3 files

2026.7.22

3 files

2026.7.21

3 files

2026.7.20

3 files

2026.7.19

3 files

2026.7.18

3 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page