Motivation

The idea here is to allow for automated systems to:

• Easily obtain a python binary, that can be used in sandboxes / workstations / laptops / instances…

• Inspect any python installation, and point out how portable it is, which shared or non-standard libraries it is using

Installation

portable-python is a regular python CLI, it can be installed with:

pickley:

pickley install portable-python
portable-python --help
portable-python inspect /usr/bin/python3

Or pipx:

pipx install portable-python
portable-python inspect /usr/bin/python3

Using pip install (a CI builder would probably do this):

/usr/bin/python3 -mvenv /tmp/pp
/tmp/pp/bin/python -mpip install portable-python
/tmp/pp/bin/portable-python --help
/tmp/pp/bin/portable-python inspect /usr/bin/python3

Supported operating systems

Portable python binaries can be built for Linux and MacOS (Intel/M1/M2).

Currently Windows is NOT supported, contributions are welcome.

Python binaries can be produced as “portable” (statically linked, can run from any folder where the binary is unpacked in), or with a --prefix (build targeted to live in a pre-determined folder, like /apps/pythonM.m)

CLI reference

Main entry point:

portable-python [OPTIONS] COMMAND [ARGS]

Global options:

• --config PATH: Config file (default: portable-python.yml)

• --quiet: Turn off DEBUG logging

• --dryrun / -n: Show what would be done

• --target PLATFORM: Override detected platform (for testing)

build <PYTHON_SPEC> - Build a portable Python binary:

portable-python build 3.13.2 -m openssl,zlib

• --modules, -m CSV: External modules to include

• --prefix, -p PATH: Use --prefix (non-portable)

build-report [PYTHON_SPEC] - Show module status and what will be compiled:

• --modules, -m CSV: Specific modules to check

• Validates that modules can be built

inspect <PATH> - Check if a Python installation is portable:

portable-python inspect /usr/bin/python3

• --modules, -m MODULES: Which modules to inspect

• --verbose, -v: Show full .so report

• --prefix, -p: Built with --prefix (not portable)

• --skip-so, -s: Don’t check all .so files

list [FAMILY] - List available versions (default: cpython):

portable-python list cpython

• --json: Output as JSON

diagnostics - Show system diagnostics

recompress <PATH> <EXT> - Re-compress existing binary tarball (for comparing compression sizes)

lib-auto-correct <PATH> - Auto-correct exes/libs to use relative paths:

• --commit: Actually perform changes (dryrun by default)

• --prefix, -p PATH: Expected --prefix from build

Library

Portable Python can be used as a python library to invoke builds, or inspect an installation.

Invoke a build from python code:

from portable_python import BuildSetup, required_python_spec

setup = BuildSetup(required_python_spec("cpython:3.9.7"))
setup.compile()

Invoke an inspection from python code:

from portable_python.inspector import PythonInspector

inspector = PythonInspector("/usr/bin/python3")
print(inspector.represented())
problem = inspector.full_so_report.get_problem(portable=True)
if problem:
    print("oops, it is not portable!: %s" % problem)

From source, contributions welcome!:

git clone https://github.com/codrsquad/portable-python.git
cd portable-python
python3 -mvenv .venv
.venv/bin/pip install -r requirements.txt -r tests/requirements.txt
.venv/bin/pip install -e .
.venv/bin/portable-python --help
.venv/bin/portable-python inspect /usr/bin/python3

tox -e py311
tox -e style

Build folder structure

portable-python uses this file structure (build/ and dist/ folders configurable):

build/
    ppp-marker/3.9.7/                   # Full installation (after build completes)
    components/                         # Builds of statically compiled extension modules are here
    deps/                               # --prefix=.../deps passed to all component ./configure scripts
    sources/
        openssl-1.1.1k.tar.gz           # Downloaded artifacts (downloaded only once)
dist/
    cpython-3.9.7-macos-arm64.tar.gz    # Ready-to-go portable binary tarball

Guiding principles

• Focuses on just one thing: compile a portable python, and validate that it is indeed portable, produce outcome in (configurable) ./dist/ folder and that’s it

• No patches: C compilation is done as simply as possible without modifying the upstream source code. Rely solely on the make/configure scripts, typically via stuff like --enable-shared=no

• Builds are validated, an important part of the effort was to write up code that is able to inspect a python installation and detect whether it is portable or not (and why not if so).

• Only the last few non-EOL versions of python are supported (no historical stuff)

• As time goes on, the code of this tool will evolve so that the latest pythons keep building (but won’t worry that older versions still keep building)

For this repo itself:

• Code is pure python, it is a CLI with one entry-point called portable-python

  • Can be ran in a debugger

  • 100% test coverage, has a --dryrun mode to help with testing / debugging / seeing what would be done quickly

  • No shell scripts (those are hard to maintain/test/debug)

  • Can be pip install-ed and reused