efa

Low-level Python bindings for AWS Elastic Fabric Adapter (EFA), including Scalable Reliable Datagram (SRD), one-sided RDMA, and GPUDirect transfers to and from torch CUDA tensors.

The package wraps libibverbs and EFA's libefa direct-verbs API in Cython. Its data path calls the provider's inline verbs directly, releases the GIL around blocking and posting operations, and does not import torch or link against CUDA.

  • No Python runtime dependencies.
  • SRD and UD queue pairs with SEND, RDMA read, and RDMA write operations.
  • Host buffers, CUDA device pointers, and dma-buf memory registration.
  • Classic and extended CQs, sender GID and unsolicited-write metadata, and direct EFA CQ, SQ, RQ, MR, and AH queries.
  • One Linux abi3 wheel for CPython 3.9 and newer.

Requirements

  • Linux on an AWS instance with one or more EFA devices attached.
  • libibverbs.so.1 and libefa.so.1 at runtime. The AWS EFA installer and current rdma-core distributions provide both.
  • The EFA device nodes available inside the process or container, normally /dev/infiniband/uverbs*.
  • A C compiler, Cython, and the rdma-core development headers only when building from source.

For GPUDirect, the EFA device and instance type must support RDMA read/write, and the NVIDIA driver must support dma-buf export or nvidia_peermem.

Install

pip install efa

To build from this checkout:

pip install "Cython>=3.0" "setuptools>=77" wheel
pip install ./efa

SRD Quickstart

SRD is reliable and connectionless. Each process creates a ready-to-send QP, exchanges a 24-byte EndpointInfo out of band, and resolves the remote GID to an address handle:

import numpy as np
import efa

dev = efa.get_efa_device_list()[0]
ctx = dev.open()
pd = ctx.alloc_pd()
cq = ctx.create_cq(256)
qp = pd.create_qp(
    efa.QPInitAttr(send_cq=cq, recv_cq=cq)
).prepare(qkey=0x1234)

local_info = efa.local_endpoint_info(qp, qkey=0x1234)
# Exchange local_info.to_bytes() with the other process.
remote_info = efa.EndpointInfo.from_bytes(remote_bytes)
peer = remote_info.peer(pd)

buf = np.zeros(4096, dtype=np.uint8)
access = (
    efa.AccessFlags.LOCAL_WRITE
    | efa.AccessFlags.REMOTE_WRITE
    | efa.AccessFlags.REMOTE_READ
)
mr = efa.reg_tensor(pd, buf, access)

qp.post_send(efa.SendWR(
    wr_id=1,
    sg_list=[mr.sge()],
    opcode=efa.WROpcode.SEND,
    send_flags=efa.SendFlags.SIGNALED,
    dest=peer,
))
for wc in cq.poll(16):
    wc.raise_for_status()

Every handle is an idempotent context manager. A QP retains its PD and CQs, an MR retains its PD and backing tensor, and an SGE retains the MR it addresses.

One-sided RDMA

EFA requires the responder to have an address handle for the requester before it accepts RDMA reads or writes. Both processes should therefore resolve the other process's EndpointInfo, even if traffic is currently one-way. A missing reverse AH completes with REM_OP_ERR and EFA vendor status 0x0e (REMOTE_ERROR_UNKNOWN_PEER).

Large buffers can be split at the device's max_rdma_size:

wrs = efa.write_wrs(local_mr, peer, remote_addr, remote_rkey)
qp.post_send(wrs)

read_wrs provides the corresponding RDMA-read operation. Every generated WR is signaled, as required by EFA.

GPUDirect With Torch

The optional efa.cuda module is torch-free and duck-types objects exposing data_ptr(), numel(), and element_size(). For torch's dma-buf path, enable VMM-backed allocations before CUDA initializes:

import os
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

import torch
import efa
import efa.cuda

src = torch.arange(1 << 20, dtype=torch.float32, device="cuda:0")
dst = torch.zeros_like(src)

src_mr = efa.cuda.register_tensor(pd, src, access)
dst_mr = efa.cuda.register_tensor(pd, dst, access)

torch.cuda.synchronize(src.device)
qp.post_send(efa.SendWR(
    wr_id=2,
    sg_list=[src_mr.sge()],
    opcode=efa.WROpcode.RDMA_WRITE,
    send_flags=efa.SendFlags.SIGNALED,
    remote_addr=remote_dst_addr,
    rkey=remote_dst_rkey,
    dest=peer,
))

After the receiver observes a completion or protocol-level write notification, it must order inbound NIC writes before CUDA consumes the destination:

with torch.cuda.device(dst.device):
    efa.cuda.flush_gpudirect_writes()

register_tensor first exports a dma-buf fd and calls ibv_reg_dmabuf_mr. If that path is unavailable, it falls back to ibv_reg_mr, which requires nvidia_peermem. The returned GpuMR retains the tensor allocation and keeps the actual CUDA virtual address because ibv_mr.addr is not meaningful for a dma-buf MR.

Direct EFA API

Area API
Device capabilities Context.query_efa_device
EFA CQ creation Context.create_cq_ex
Sender and unsolicited metadata CQEx.poll, WC.sgid, WC.unsolicited
CQ layout CQ.query_efa, CQEx.query_efa
SQ/RQ layout QP.query_wqs
MR interconnect IDs MR.query_efa
Address-handle number AH.ahn
SRD QP creation PD.create_qp with QPType.SRD

When using unsolicited RDMA write-with-immediate completions, create every communicating QP with QPInitAttr(..., unsolicited_write_recv=True) and use an extended CQ created with unsolicited=True. EFA requires peers to negotiate the same QP feature set.

The direct layout queries expose process-local addresses for advanced consumers. They do not transfer ownership of provider memory.

Testing

pip install -e "./efa[test,gpu]"
cd efa
pytest -rs

Tests marked integration exercise real EFA hardware. Tests marked gpu perform torch-verified GPU-to-GPU, GPU-to-host, and host-to-GPU transfers. Unavailable hardware capabilities are skipped explicitly. Set RDMA4PY_SKIP_HARDWARE_TESTS=1 to force all hardware-dependent tests to skip.

License

BSD-3-Clause. See LICENSE.

Project links

Data verified by PyPI on Jul 30, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2026.9.6)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for d4l3k from gravatar.com d4l3k

Credits

Author: Tristan Rice

GitHub Statistics

Data verified by PyPI on Jul 30, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

BSD-3-Clause
View SPDX License List

Requires

Python >=3.9

Provides Extra

test gpu

Tags

efa srd rdma ibverbs aws gpudirect networking

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

efa-2026.7.30.tar.gz (360.5 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

efa-2026.7.30-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ ARM64manylinux: glibc 2.28+ ARM64

efa-2026.7.30-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ x86-64manylinux: glibc 2.5+ x86-64

File details

Details for the file efa-2026.7.30.tar.gz.

File metadata

  • Download URL: efa-2026.7.30.tar.gz
  • Upload date:
  • Size: 360.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for efa-2026.7.30.tar.gz
Algorithm Hash digest
SHA256 9b1329aef7b33744143d8d12fb5f26b7b4294048d0fd2f8a402e79ba8228bd75
MD5 3cd6bac089905c665e9a1bca3ea9c4a1
BLAKE2b-256 87e4d107c975db39a7462cb5b138fa538640a13f25c63425023f0bf3bd57bd6c

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.7.30.tar.gz:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file efa-2026.7.30-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for efa-2026.7.30-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 09f5f85d0cc7a323cf0450569fbcc6687f0ccd101e0c89f3cf9671afb386d060
MD5 c530f4452804ceb659f2d3d5842eee8b
BLAKE2b-256 2229d4eba07faff93e1f5912cdb8934a6314c7649802409fcbc1e864aca40eb8

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.7.30-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file efa-2026.7.30-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.

File metadata

File hashes

Hashes for efa-2026.7.30-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
Algorithm Hash digest
SHA256 f78649a16d16926470df70f87d78c14f9c3e3006d28e45600a081d25013c0bbe
MD5 ba610b558285f72b082a7703a10e4438
BLAKE2b-256 5edd387c03168b546869de1ac80cb66b343f1fdf822015777c27d858b21ffbbf

See more details on using hashes here.

Provenance

The following attestation bundles were made for efa-2026.7.30-cp39-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl:

Publisher: publish_efa.yml on d4l3k/rdma4py

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

2026.9.6

3 files

2026.9.5

3 files

2026.9.4

3 files

2026.9.3

3 files

2026.9.2

3 files

2026.9.1

3 files

2026.8.31

3 files

2026.8.30

3 files

2026.8.29

3 files

2026.8.28

3 files

2026.8.27

3 files

2026.8.26

3 files

2026.8.25

3 files

2026.8.24

3 files

2026.8.23

3 files

2026.8.22

3 files

2026.8.21

3 files

2026.8.20

3 files

2026.8.19

3 files

2026.8.18

3 files

2026.8.17

3 files

2026.8.16

3 files

2026.8.15

3 files

2026.8.14

3 files

2026.8.13

3 files

2026.8.12

3 files

2026.8.11

3 files

2026.8.10

3 files

2026.8.9

3 files

2026.8.8

3 files

2026.8.7

3 files

2026.8.6

3 files

2026.8.5

3 files

2026.8.4

3 files

2026.8.3

3 files

2026.8.2

3 files

2026.8.1

3 files

2026.7.31

3 files

This release

2026.7.30 This release

3 files

2026.7.29

3 files

2026.7.28

3 files

2026.7.27

3 files

2026.7.26

3 files

2026.7.25

3 files

2026.7.24

3 files

2026.7.23

3 files

2026.7.22

3 files

2026.7.21

3 files

2026.7.20

3 files

2026.7.19

3 files

2026.7.18

3 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page