auditwheel-emscripten 0.0.10

auditwheel-like tool for Pyodide

auditwheel-emscripten

auditwheel-like tool for wheels targeting Emscripten platform

$ pip install auditwheel-emscripten

What is this?

auditwheel-emscripten is a tiny tool to facilitate the creation of Python wheel packages for Emscripten. auditwheel-emscripten is originally created for Pyodide, but it can be used in any other projects that target Python-in-the-browser using Emscripten.

• pyodide auditwheel show: shows external shared libraries that the wheel depends on.
• pyodide auditwheel copy: copies these external shared libraries into the wheel itself.

Usage (CLI)

 Usage: pyodide auditwheel [OPTIONS] COMMAND [ARGS]...

 Auditwheel-like tool for emscripten wheels and shared libraries.

╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --help          Show this message and exit.                                                                                         │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ copy      Copy shared libraries to the wheel directory. Similar to repair but does not modify the needed section of WASM module.    │
│ exports   Show exports of a wheel or a shared library file.                                                                         │
│ imports   Show imports of a wheel or a shared library file.                                                                         │
│ repair    [Experimental] Repair a wheel file: copy shared libraries to the wheel directory and modify the path in the wheel file.   │
│ show      Show shared library dependencies of a wheel or a shared library file.                                                     │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

# wget https://cdn.jsdelivr.net/pyodide/v0.21.3/full/Shapely-1.8.2-cp310-cp310-emscripten_3_1_14_wasm32.whl
$ pyodide auditwheel show Shapely-1.8.2-cp310-cp310-emscripten_3_1_14_wasm32.whl

The following external shared libraries are required:
{
│   'shapely/speedups/_speedups.cpython-310-wasm32-emscripten.so': ['libgeos_c.so'],
│   'shapely/vectorized/_vectorized.cpython-310-wasm32-emscripten.so': ['libgeos_c.so']
}

$ pyodide auditwheel copy --libdir <directory which contains libgeos_c.so> Shapely-1.8.2-cp310-cp310-emscripten_3_1_14_wasm32.whl

Repaired wheel has following external shared libraries:
{
│   'Shapely.libs/libgeos.so.3.10.3': [],
│   'Shapely.libs/libgeos_c.so': ['libgeos.so.3.10.3'],
│   'shapely/speedups/_speedups.cpython-310-wasm32-emscripten.so': ['libgeos_c.so'],
│   'shapely/vectorized/_vectorized.cpython-310-wasm32-emscripten.so': ['libgeos_c.so']
}

Usage (API)

Listing shared library dependencies of a wheel file:

from auditwheel_emscripten import show
libs = show("Shapely-1.8.2-cp310-cp310-emscripten_3_1_14_wasm32.whl")
print(libs)
# {'shapely/vectorized/_vectorized.cpython-310-wasm32-emscripten.so': ['libgeos_c.so'], 'shapely/speedups/_speedups.cpython-310-wasm32-emscripten.so': ['libgeos_c.so']}

Copying shared libraries to the wheel:

from auditwheel_emscripten import repair, show
repaired_wheel = repair(
    "Shapely-1.8.2-cp310-cp310-emscripten_3_1_14_wasm32.whl",
    libdir="/path/where/shared/libraries/are/located",
    outdir="/path/to/output/directory",
    # If set this to true, modify the needed section of WASM module.
    # Note that is not compatible with WebAssembly dynamic linking ABI.
    # https://github.com/WebAssembly/tool-conventions/blob/main/DynamicLinking.md
    modify_needed_section=False,
)
libs = show(repaired_wheel)
print(libs)
# {'Shapely.libs/libgeos.so.3.10.3': [], 'Shapely.libs/libgeos_c.so': ['libgeos.so.3.10.3'], 'shapely/speedups/_speedups.cpython-310-wasm32-emscripten.so': ['libgeos_c.so'], 'shapely/vectorized/_vectorized.cpython-310-wasm32-emscripten.so': ['libgeos_c.so']}

Implementation details / limitations

Dynamic linking in Emscripten

Dynamic linking is not in the WebAssembly specification, but Emscripten has its own dynamic linking support, which is required for building Python wheels targeting Emscripten platform.

This tool is based on:

• Loosely documented Emscripten dynamic linking specification: https://github.com/WebAssembly/tool-conventions/blob/main/DynamicLinking.md
• Emscripten's internal utility for inspecting WASM module: https://github.com/emscripten-core/emscripten/blob/main/tools/webassembly.py
• Emscripten dylink implementation: https://github.com/emscripten-core/emscripten/blob/main/src/library_dylink.js

auditwheel vs auditwheel-emscripten

auditwheel is a tool that helps repair and modify Python wheels (pre-compiled packages) to be compatible with a wide range of Linux distributions. It does this by copying external shared libraries into the wheel and repairing the RPATH (a file path that is used to locate shared libraries at runtime) of the ELF binary so that the Linux operating system can locate the library when the program is run.

auditwheel-emscripten is a variation of auditwheel that is specifically designed to work with Emscripten-generated WebAssembly (WASM) modules. It does not perform an audit on the wheel, as Emscripten does not guarantee compatibility between versions. Instead, it simply copies the required libraries into the wheel without modifying the module itself. It is up to the user to manually implement a way to locate these libraries at runtime.