python-embedded-launcher 0.3.1

Launcher exe for distributing Python apps on Windows

For Windows only.

The launcher is a small C program that loads the Python DLL and calls Py_Main with itself as parameter, loading a zipped Python application appended to the exe.

Yes, it is yet an other tool to make standalone Windows applications with Python.

See also py2exe, pyinstaller, cx_Freeze. I guess the basic idea is different from these tools, as we try to combine complete packages with a Python minimal distribution: Python + wheel files. There is no attempt made to search together all the required files, there is no scan for import statements etc.

See also pex, which can grab dependencies from pypi and make zip applications, but that does not bundle the Python interpreter.

Scenarios

Distribute an application

Bundle Python with an application so that users can use it without having to install Python.

In launcher27.rc set IDS_PYTHONHOME to "%SELF%\\python27-minimal" (this is already the default). This way, the Python distribution is expected at the location of the executable. The environment variable SELF is set automatically by the launcher itself (dirname of abspath of exe).

Common python-minimal package

Multiple tools can use a common copy of Python. e.g. with a package manager. Python can be provided as one package and separate application packages can use that Python distribution to run.

In launcher27.rc set IDS_PYTHONHOME to "%PACKAGE_ROOT%\\python27-minimal". This way, the Python distribution is expected to be at a fixed location, where the PACKAGE_ROOT variable points at. It is expected to be set by the package manager.

Usage

The example directory has demos where these steps are written in a batch file that is ready to run. The description here explains the steps. On Windows, once Python 3 is installed, the Python Launcher py is available, this is what is used here. Otherwise replace py -2/py -3 with python/python3. When packaging an app, the same Python version that is packaged, should be used to run the steps here (using py -2 or py -3 accordingly).

Assuming your own project has a setup.py, install to a dist directory:

py -m pip install --prefix=dist --ignore-installed /path/to/your/project

Install dependencies:

py -m pip install --prefix=dist --ignore-installed -r requirements.txt

Create a Python distribution:

py -2 create_python27_minimal.py -d dist

or for Python 3:

py -3 -m launcher_tool.download_python3_minimal -d dist

Use the launcher tool to write the exe, calling your app:

py -m launcher_tool -o dist/myapp.exe -e mymodule:main

Note

pip will also install scripts in a subdirectory called Scripts. this usually not needed for a packaged app, so this can be deleted.

Variations

Instead of --prefix=dist it is also possible to use --user when the envirionment variable PYTHONUSERBASE is set to dist. This will install into a slightly different subdirectory of dist but lanucher.py also searches this one.

It is also possible download all dependencies as wheels first, so that subsequent runs to create a distribution do not need to download from the Internet (recommended).

Fetch the dependencies once:

py -m pip wheel -w wheels -r requirements.txt

Then use these with --find-links and --no-index options:

py -m pip install --prefix=dist --ignore-installed --find-links=wheels --no-index -r requirements.txt

Alternatives

It is also possible to install pip within the embedded Python distribution and use that distribution itself to install packages:

py -3 -m launcher_tool.download_python3_minimal
cd python3-minimal
python get-pip.py
python -m pip install --find-links=/path/to/wheels --no-index -r requirements.txt
cd ..
py -3 -m launcher_tool -o myapp.exe -e mymodule:main

First we use py -3 to use the systems Python 3, then python to call the local version in the directory. The first step is installing pip with get-pip.py. Then using this to install more packages. Installing from source may not work, it is recommended to only use wheels for this step.

Python 3’s zipapp module can be used to package the application:

py -3 -m zipapp myapp.py -o myapp.pyz
py -3 -m launcher_tool -o myapp.exe --run-path myapp.pyz

Tools

launcher_tool

A tool to combine scripts with the launcher27.exe or launcher3.exe. A script is added with the name __main__.py to a zip file. launcher.py a helper module for the boot script is also appended to the zip. This zip file is appended to the exe. Optionally it can also include other files too.

launcher_tool.create_python27_minimal

Used to create a python27-minimal distribution. It copies the Python installation from the system.

launcher_tool.download_python3_minimal

Unpack a Python 3 embedded distribution. The data is downloaded from https://www.python.org/downloads/windows/ and cached locally (so that for repeated runs, it does not need to use the Internet again).

launcher_tool.copy_launcher

Copy the launcher.exe to a file. Used e.g. for customizations using launcher_tool.resource_editor.

launcher_tool.resource_editor

A small Windows resource editor that can modify the launcher. It uses Windows API functions to read and write the data.

• adding and editing strings

• retrieving and writing icons

• export resources as (binary) blob

• removing any resource type

• adding any resource type is supported partially (currently limited by data input possibilities)

• dump resources

• dump decoded string table

Attention! It will strip debug data and remove the attached ZIP file! So this tool must be used before the application is appended to the launcher.

Customization

The texts and the location of Python is stored as Windows resource in the launcher*.exe. It is possible to use resource editor tools to patch the exe.

Using launcher_tool.resource_editor it is possible to make small edits on the command line, but it does not support all resource types.

E.g. if there was a common Python package installed under %LOCALAPPDATA% a series of commands like this would create a modified launcher:

python -m launcher_tool.copy_launcher -o %DIST%/myapp.exe
python -m launcher_tool.resource_editor %DIST%/myapp.exe edit_strings --set 1:^%LOCALAPPDATA^%\python27-minimal
python -m launcher_tool.resource_editor %DIST%/myapp.exe write_icon newicon.ico
python -m launcher_tool --append-only %DIST%/myapp.exe -e mymodule:main

Note that ^ is the escape character of cmd.exe when used interactively and makes that the % is not treated specially but as normal text (and the variable is thus not expanded). For some reason %% must be used instead of ^% when these lines are put in a .bat file.

An 3rd party tool would be resourcehacker. It can even edit exe files with attached zip data without destroying them.

Alternatively use the sources here to recompile the binaries, it really just needs a mingw gcc (which is only a few dozens of megabytes large). In that case the launcher*.rc within the src/python* directory are edited with a text editor and compile.bat is used to recreate the exe.

Build

Requires a mingw gcc compiler (see Requirements).

Run compile_all.bat in the src directory.

The python27 and python3 directories contain the sources and a batch file. The compile_all.bat file runs both of them.

The resulting binaries will be placed in the launcher_tool directory so that they are available as data files for the Python tool.

Requirements

To build applications:

• pip and wheel

• requests (for download_python3_minimal)

Running pip install -r requirements.txt will install these.

To build the launcher exe:

• mingw(-64) GCC compiler, e.g. http://tdm-gcc.tdragon.net/ has one.

The either PATH must be set so that gcc can be found or the compile*.bat files have to be edited (they set PATH).

API

A small helper module called launcher is automatically packaged with the exe. It contains a few helper functions.

launcher.patch_sys_path()

Add directories (relative to executable, if existing) to sys.path.

• the direcrtory of the executable

• Python{py.major}{py.minor}/site-packages

• Python{py.major}{py.minor}/Lib/site-packages

• Lib/site-packages

These locations are also scanned for .pth files.

launcher.extend_sys_path_by_pattern(pattern)

Add files matching a pattern (e.g. *.zip, *.whl, *.egg) to sys.path. the pattern is prefixed with the location of the executable. In case of wheel files, it only works for pure Python wheels and only if they do no access the file system to load data on their own (should use pkgutil). This function is used if the command line option --extend-sys-path is used.

launcher.restore_sys_argv()

Get original command line via Windows API. Restores sys.argv (which is used by the launcher to pass the location of Python). This function is called by the default boot code (__main__).

launcher.close_console()

Useful for GUI applications, it closes a separate console window if there is one, e.g. when the exe was started by a double click.

launcher.is_separate_console_window()

Return true if the console window was opened with this process (e.g. the console was opened because the exe was started from the file Explorer).

launcher.wait_at_exit()

Wait at exit, but only if console window was opened separately. This function is called automatically if the command line option --wait is used.

launcher.wait_on_error()

Wait if the program terminates with an exception, but only if console window was opened separately. This function is called automatically if the command line option --wait-on-error is used.

Implementation Details

Some random notes…

Python 2 uses “ASCII API” while Python 3 uses “Unicode API”. Thats why separate code for the two launchers exists.

The launcher is compiled as console application, so it opens a console window when started from the explorer. However it is easily closed with a Windows API call and launcher.py, which is added to the application, has a function for that. The advantage is, that applications can be started in a console and one can see the output - and wait for the program to terminate etc.

There are currently no 64 bit versions of the launcher. Though compiling them should be no more than adding a switch to the compiler…

Starting with Python 3.5, an embedded Python distribution is already available (and used here) for download, see https://docs.python.org/3/using/windows.html#embedded-distribution

While Python 3 has a python3.dll, which would be nice to use, as it would make the launcher independent of the Python version – it won’t work. Py_SetPath is not exposed by that library. As a workaround, the name (e.g. python35) is in the resources of launcher3.exe so that it can be changed without recompiling.

Python is loaded dynamically via LoadLibrary. The launcher is not linked against the DLL. This has the advantage that the location of the DLL can be different to the one of the exe and that the DLL name can be provided and edited as resource (only in launcher.exe). The separation would also allow to check if the VC runtime is installed and direct the user to the download if it is not, but this is not implemented yet.

Why put Python in a subdirectory? Because someone could add the directory containing the exe to the PATH and then the system would potentially find multiple python.exe and pythonXY.dll…

pip install --user installs the packages into a subdirectory PythonXY/site-packages named after the Python version.

pip install --prefix=dist installs the packages to a subdirectory Lib/site-packages.