pypi2pkgbuild 0.1

A PyPI to PKGBUILD converter.

Convert PyPI packages to Arch Linux packages, inspired from pip2arch.

• Dependencies and installation

• Usage

  • Usage notes

• Build-time dependencies of packages

• Vendored packages

• Comparison with other tools

  • Comparison with pip2arch

  • Comparison with fpm

Dependencies and installation

pypi2pkgbuild.py depends on the Arch Linux packages namcap, pkgfile and python-pip.

The script can be installed with pip install [--user] ., or can also be run directly.

One can even run pypi2pkgbuild.py on itself to create a proper Arch package (pypi2pkgbuild.py git+https://github.com/anntzer/pypi2pkgbuild).

A minimal test suite (checking that pypi2pkgbuild.py can indeed package itself) can by run with pytest (pytest -s to keep the output to the terminal).

Usage

pypi2pkgbuild.py PYPINAME creates a PKGBUILD for the latest version of the given PyPI package (including prereleases if the --pre flag is passed. Because PyPI’s dependencies are somewhat unreliable, it installs the package in a virtualenv to figure out the dependencies. Note that thanks to pip’s wheel cache, the build is later reused; i.e. the procedure entails no extra work.

A -git package can be built with pypi2pkbguild.py git+https://....

The package is then built and verified with namcap.

The goal is to make this tool as automated as possible: if all the information to build a package is (reasonably) accessible, this tool should be able to build it.

In order to provide additional information to makepkg, edit PKGBUILD_EXTRAS, which is sourced at the end of PKGBUILD.

Usage notes

• It is suggested to create an alias with standard options set, e.g.

alias pypi2pkgbuild.py='PKGEXT=.pkg.tar pypi2pkgbuild.py -g cython -b /tmp/pypi2pkgbuild/ -f'

• By default, the pkgrel of (standard) packages is set to 00. This allows automatic upgrading into official packages (and AUR ones, if an AUR helper is used) whenever the repositories are updated. Additionally, the use of 00 rather than 0 serves as a (weak) marker that the package was automatically generated by this tool. In order to prevent such an upgrade, one can use the --pkgrel flag to set pkgrel to, e.g., 99.

• If one wishes to completely bypass AUR Python packages while maintaining the use of an AUR helper for non-Python packages, one can define a shell function that excludes pypi2pkgbuild.py-generated packages that do not appear in the official repositories, e.g., for pacaur:

  pacaur() {
     if [[ "$1" = "-Syu" ]]; then
        # Update, in case some packages moved in or out of the official repos.
        sudo pacman -Sy
        # Update everything except python packages with pkgver=00 or 99.
        PKGEXT=.pkg.tar command pacaur -Su --ignore \
              "$(pacman -Qm | grep '^python-.*-\(00\|99\)$' | cut -d' ' -f1 | paste -sd,)"
     else
        command pacaur "$@"
     fi
  }

  This function will not bypass Python packages explicitly installed from the AUR, as the user may have done so to bypass some incorrect packaging by pypi2pkgbuild.py. It is recommended to use the -i flag to calls to pypi2pkgbuild.py (e.g. in an alias) to exclude packages that are mishandled by pypi2pkgbuild.py (see mispackaged packages). The -i flag can be passed multiple times; passing an empty argument to it will clear the ignore list defined so far.

• In order to package a locally available git repository, use

  $ pypi2pkgbuild.py git+file://$absolute_path_to_repo  # (e.g. file:///home/...)

  In order to package a locally available sdist or wheel, use

  $ pypi2pkgbuild.py file://$absolute_path_to_file  # (e.g. file:///home/...)

  Note that in both cases absolute paths are necessary.

  Building packages from local repos or wheels needs to be done in topological order of the dependencies (so that pypi2pkgbuild.py can find that the dependencies are actually present), or by passing the -d flag (“do not build dependencies”); if it is used, the Arch package may not use the correct dependency names (if they are not of the form python-pep503-normalized-name).

• By default, pypi2pkgbuild.py ignores pip config files such as ~/.config/pip/pip.conf. An explicitly set PIP_CONFIG_FILE will be respected, but may cause pypi2pkgbuild.py to fail as some pip calls will be unexpectedly modified.

  Likewise, user-site packages are ignored unless PYTHONNOUSERSITE is explicitly set to an empty value.

Build-time dependencies of packages

pypi2pkgbuild.py attempts to guess whether Cython and SWIG are build-time dependencies by checking for the presence of .pyx and .i files, respectively. If this is not desired, set the --guess-makedepends option accordingly.

pypi2pkgbuild.py guesses whether numpy is a build-time dependency by attempting a build without numpy, then, in case of failure, a build with numpy.

Additional Python build-time dependencies (i.e., setup_requires) can be specified (as PyPI names) using the --setup-requires flag.

Vendored packages

Some Arch packages (e.g. ipython) include a number of smaller PyPI packages.

Because it is not possible to assign a meaningful version automatically, pypi2pkgbuild.py instead creates an independent Arch package for each of the PyPI packages (with two dashes in the name, to prevent name conflicts) and a master package that depends on all of them. The pkgrel of the master package is set to $official_pkgrel.99, so that the package appears more recent than the current official version but older than any future official version. All these packages conflict with all versions of the official package (except the newly created package), so updating should work fine when the official package is actually updated.

However, dependencies are still expressed using the master package (to avoid breakage on update into an official package), so internal dependencies will appear be circular.

All the packages are placed in a subfolder named meta:$pkgname, so one can easily install everything by cd’ing there and running

$ sudo pacman -U --asdeps **/*.xz
$ sudo pacman -D --asexplicit $pkgname/$pkgname.tar.xz

Comparison with other tools

Comparison with pip2arch

pypi2pkgbuild.py includes many improvements over pip2arch:

• Supports wheels (the default is to prefer any-platform wheels, then sdists, then manylinux1 wheels, but this can be changed using --pkgtypes).

• Resolves Python dependencies via installation in a temporary virtualenv, and also creates PKGBUILDs for those that are not available as official packages.

• Resolves binary dependencies via namcap and adds them to the depends array if they are installed (thus, it is suggested to first install them as --asdeps and then let the generated PKGBUILD pick them up as dependencies). Note that some packages are distributed with a copy of the required libraries; in this case, pypi2pkgbuild.py’s behavior will depend on whether the package defaults to using the system-wide library or its own copy.

• Automatically tries to fetch a missing license file from Github, if applicable.

• Automatically builds the package (with options given in --makepkg=...) and run namcap.

• Automatically builds all outdated dependencies via -u.

Comparison with fpm

Another tool for converting PyPI packages to Arch Linux packages is the generic converter fpm <https://github.com/jordansissel/fpm>; however, it seems to be incompatible with recent versions of pip.