efa

Low-level Python bindings for AWS Elastic Fabric Adapter (EFA), including Scalable Reliable Datagram (SRD), one-sided RDMA, and GPUDirect transfers to and from torch CUDA tensors.

The package wraps libibverbs and EFA's libefa direct-verbs API in Cython. Its data path calls the provider's inline verbs directly, releases the GIL around blocking and posting operations, and does not import torch or link against CUDA.

  • No Python runtime dependencies.
  • SRD and UD queue pairs with SEND, RDMA read, and RDMA write operations.
  • Host buffers, CUDA device pointers, and dma-buf memory registration.
  • Classic and extended CQs, sender GID and unsolicited-write metadata, and direct EFA CQ, SQ, RQ, MR, and AH queries.
  • One Linux abi3 wheel for CPython 3.9 and newer.

Requirements

  • Linux on an AWS instance with one or more EFA devices attached.
  • libibverbs.so.1 and libefa.so.1 at runtime. The AWS EFA installer and current rdma-core distributions provide both.
  • The EFA device nodes available inside the process or container, normally /dev/infiniband/uverbs*.
  • A C compiler, Cython, and the rdma-core development headers only when building from source.

For GPUDirect, the EFA device and instance type must support RDMA read/write, and the NVIDIA driver must support dma-buf export or nvidia_peermem.

Install

pip install efa

To build from this checkout:

pip install "Cython>=3.0" "setuptools>=77" wheel
pip install ./efa

SRD Quickstart

SRD is reliable and connectionless. Each process creates a ready-to-send QP, exchanges a 24-byte EndpointInfo out of band, and resolves the remote GID to an address handle:

import numpy as np
import efa

dev = efa.get_efa_device_list()[0]
ctx = dev.open()
pd = ctx.alloc_pd()
cq = ctx.create_cq(256)
qp = pd.create_qp(
    efa.QPInitAttr(send_cq=cq, recv_cq=cq)
).prepare(qkey=0x1234)

local_info = efa.local_endpoint_info(qp, qkey=0x1234)
# Exchange local_info.to_bytes() with the other process.
remote_info = efa.EndpointInfo.from_bytes(remote_bytes)
peer = remote_info.peer(pd)

buf = np.zeros(4096, dtype=np.uint8)
access = (
    efa.AccessFlags.LOCAL_WRITE
    | efa.AccessFlags.REMOTE_WRITE
    | efa.AccessFlags.REMOTE_READ
)
mr = efa.reg_tensor(pd, buf, access)

qp.post_send(efa.SendWR(
    wr_id=1,
    sg_list=[mr.sge()],
    opcode=efa.WROpcode.SEND,
    send_flags=efa.SendFlags.SIGNALED,
    dest=peer,
))
for wc in cq.poll(16):
    wc.raise_for_status()

Every handle is an idempotent context manager. A QP retains its PD and CQs, an MR retains its PD and backing tensor, and an SGE retains the MR it addresses.

One-sided RDMA

EFA requires the responder to have an address handle for the requester before it accepts RDMA reads or writes. Both processes should therefore resolve the other process's EndpointInfo, even if traffic is currently one-way. A missing reverse AH completes with REM_OP_ERR and EFA vendor status 0x0e (REMOTE_ERROR_UNKNOWN_PEER).

Large buffers can be split at the device's max_rdma_size:

wrs = efa.write_wrs(local_mr, peer, remote_addr, remote_rkey)
qp.post_send(wrs)

read_wrs provides the corresponding RDMA-read operation. Every generated WR is signaled, as required by EFA.

GPUDirect With Torch

The optional efa.cuda module is torch-free and duck-types objects exposing data_ptr(), numel(), and element_size(). For torch's dma-buf path, enable VMM-backed allocations before CUDA initializes:

import os
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

import torch
import efa
import efa.cuda

src = torch.arange(1 << 20, dtype=torch.float32, device="cuda:0")
dst = torch.zeros_like(src)

src_mr = efa.cuda.register_tensor(pd, src, access)
dst_mr = efa.cuda.register_tensor(pd, dst, access)

torch.cuda.synchronize(src.device)
qp.post_send(efa.SendWR(
    wr_id=2,
    sg_list=[src_mr.sge()],
    opcode=efa.WROpcode.RDMA_WRITE,
    send_flags=efa.SendFlags.SIGNALED,
    remote_addr=remote_dst_addr,
    rkey=remote_dst_rkey,
    dest=peer,
))

After the receiver observes a completion or protocol-level write notification, it must order inbound NIC writes before CUDA consumes the destination:

with torch.cuda.device(dst.device):
    efa.cuda.flush_gpudirect_writes()

register_tensor first exports a dma-buf fd and calls ibv_reg_dmabuf_mr. If that path is unavailable, it falls back to ibv_reg_mr, which requires nvidia_peermem. The returned GpuMR retains the tensor allocation and keeps the actual CUDA virtual address because ibv_mr.addr is not meaningful for a dma-buf MR.

Direct EFA API

Area API
Device capabilities Context.query_efa_device
EFA CQ creation Context.create_cq_ex
Sender and unsolicited metadata CQEx.poll, WC.sgid, WC.unsolicited
CQ layout CQ.query_efa, CQEx.query_efa
SQ/RQ layout QP.query_wqs
MR interconnect IDs MR.query_efa
Address-handle number AH.ahn
SRD QP creation PD.create_qp with QPType.SRD

When using unsolicited RDMA write-with-immediate completions, create every communicating QP with QPInitAttr(..., unsolicited_write_recv=True) and use an extended CQ created with unsolicited=True. EFA requires peers to negotiate the same QP feature set.

The direct layout queries expose process-local addresses for advanced consumers. They do not transfer ownership of provider memory.

Testing

pip install -e "./efa[test,gpu]"
cd efa
pytest -rs

Tests marked integration exercise real EFA hardware. Tests marked gpu perform torch-verified GPU-to-GPU, GPU-to-host, and host-to-GPU transfers. Unavailable hardware capabilities are skipped explicitly. Set RDMA4PY_SKIP_HARDWARE_TESTS=1 to force all hardware-dependent tests to skip.

License

BSD-3-Clause. See LICENSE.

Project links

Data verified by PyPI on Aug 12, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2026.9.6)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for d4l3k from gravatar.com d4l3k

Credits

Author: Tristan Rice

GitHub Statistics

Data verified by PyPI on Aug 12, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

BSD-3-Clause
View SPDX License List

Requires

Python >=3.9

Provides Extra

test gpu

Tags

efa srd rdma ibverbs aws gpudirect networking

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

efa-2026.8.12.tar.gz (360.5 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

efa-2026.8.12-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ ARM64manylinux: glibc 2.28+ ARM64

efa-2026.8.12-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ x86-64manylinux: glibc 2.5+ x86-64

File details

Details for the file efa-2026.8.12.tar.gz.

File metadata

  • Download URL: efa-2026.8.12.tar.gz
  • Upload date:
  • Size: 360.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for efa-2026.8.12.tar.gz
Algorithm Hash digest
SHA256 eec819b5a645d074a3adb91dad014379b54edd885ff376a66cca063bc21ea912
MD5 cf7968faf668722ab56f787f57161f58
BLAKE2b-256 13eb0fce01a8443ab462147b8bc2a3b64040e92befb3beaa25769e4f889445cc

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.8.12.tar.gz:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file efa-2026.8.12-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for efa-2026.8.12-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 b98924a52a27fdd419073ca8a79561f2a101da4e76c918a6c36224b135896da9
MD5 87a9355422dc6964d186cee934d0fb7e
BLAKE2b-256 64154ef24c7dba4cd3781c796176f40df471cfa39a3aba423c594313b7ad1a17

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.8.12-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file efa-2026.8.12-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.

File metadata

File hashes

Hashes for efa-2026.8.12-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
Algorithm Hash digest
SHA256 2c67de0fe2b7b0a78564135a5f110cff4aa193afd6c505053f31ab8d8099ff6c
MD5 0d6990e1925d77bc3310ae7fab4d8804
BLAKE2b-256 e5b964b5d40fe5bcb56ad15083c8162025bf118d47df60e0ccff944982edc16d

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.8.12-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

2026.9.6

3 files

2026.9.5

3 files

2026.9.4

3 files

2026.9.3

3 files

2026.9.2

3 files

2026.9.1

3 files

2026.8.31

3 files

2026.8.30

3 files

2026.8.29

3 files

2026.8.28

3 files

2026.8.27

3 files

2026.8.26

3 files

2026.8.25

3 files

2026.8.24

3 files

2026.8.23

3 files

2026.8.22

3 files

2026.8.21

3 files

2026.8.20

3 files

2026.8.19

3 files

2026.8.18

3 files

2026.8.17

3 files

2026.8.16

3 files

2026.8.15

3 files

2026.8.14

3 files

2026.8.13

3 files

This release

2026.8.12 This release

3 files

2026.8.11

3 files

2026.8.10

3 files

2026.8.9

3 files

2026.8.8

3 files

2026.8.7

3 files

2026.8.6

3 files

2026.8.5

3 files

2026.8.4

3 files

2026.8.3

3 files

2026.8.2

3 files

2026.8.1

3 files

2026.7.31

3 files

2026.7.30

3 files

2026.7.29

3 files

2026.7.28

3 files

2026.7.27

3 files

2026.7.26

3 files

2026.7.25

3 files

2026.7.24

3 files

2026.7.23

3 files

2026.7.22

3 files

2026.7.21

3 files

2026.7.20

3 files

2026.7.19

3 files

2026.7.18

3 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page