stdeb3 0.9.0.post2

Python to Debian source package conversion utility, forked from stdeb

stdeb3 - Python to Debian source package conversion utility

This is a port of stdeb with modernisations such as removal of python2 support at build-time, removal of support for Debian versions less than 8 (jessie) (oldstable), bump of minimum required Debhelper version, newer dh-compat version, and lots more. See the RELEASE_NOTES.txt for more info.

stdeb3 produces Debian source packages from Python packages via a new distutils command, sdist_dsc. Automatic defaults are provided for the Debian package, but many aspects of the resulting package can be customized (see the customizing section, below). An additional command, bdist_deb, creates a Debian binary package, a .deb file. The install_deb command installs this .deb file. The debianize command builds a debian/ directory directly alongside your setup.py.

Several convenience utilities are also provided:

• pypi-download will query the Python Package Index (PyPI) for a package and download it.

• pypi-install will query the Python Package Index (PyPI) for a package, download it, create a .deb from it, and then install the .deb.

• py2dsc will convert a distutils-built source tarball into a Debian source package.

• py2dsc-deb will convert a distutils-built source tarball into a Debian source package and then use the Debian machinery to build a .deb file from this.

Contents

• stdeb3 - Python to Debian source package conversion utility

  • Python 3 support

  • News

  • The commands

    • pypi-download, command-line command

    • pypi-install, command-line command

    • py2dsc, command-line command

    • py2dsc-deb, command-line command

    • sdist_dsc, distutils command

    • bdist_deb, distutils command

    • install_deb, distutils command

    • debianize, distutils command

    • A note about telling distutils to use the stdeb distutils commands

  • Examples

    • Quickstart 1: Install something from PyPI now, I don’t care about anything else

    • Quickstart 2: Just tell me the fastest way to make a .deb

    • Quickstart 3: I read the warning, so show me how to make a source package, then compile it

    • Quickstart 4: Install from a Python package direct to a debian system package

    • Another example, with more explanation

  • Download

  • Install (or, using stdeb to create an stdeb installer)

  • Background

  • Features

  • Customizing the produced Debian source package (config options)

    • stdeb distutils command options

    • stdeb.cfg configuration file

  • Prerequisites

  • TODO

  • License

  • Authors

  • Additional Credits

Python 3 support

As explained in more detail below, the heart of stdeb is the sdist_dsc distutils command. This command runs once to generate a Debian source package. This Debian source package can specify building packages for Python 2, Python 3, or both. This can now only per performed by the python3 interpreter. By default, only packages are built for the version of Python being used. To override this, use --with-python2=True or --with-python3=True as an argument to the sdist_dsc distutils command (or use both to be sure). For example, to build only a Python 3 package using the Python 3 interpreter:

python3 setup.py --command-packages=stdeb3.command bdist_deb

To build both Python 2 and Python 3 packages using the Python 3 interpreter (and only the Python3 package installs scripts):

python3 setup.py --command-packages=stdeb3.command sdist_dsc --with-python2=True --with-python3=True --no-python2-scripts=True bdist_deb

News

• 2019-01-31: Version 0.9.0. Ported to stdeb3. See RELEASE_NOTES.txt for changes.

• 2015-02-18: Version 0.8.5. See the download page. Bugfixes: reverted change that installed into virtualenv when built in virtualenv. Improvements: Added –allow-virtualenv-install-location to allow installing into virtualenv location. Supports Debian Squeeze (6), Debian Wheezy (7), Ubuntu Precise (12.04), Ubuntu Trusty (14.04) and later releases.

• 2015-02-16: Version 0.8.4. See the download page. Bugfixes: works on Python 3.4 (e.g. Ubuntu Trusty) again. Improvements: Improved customization for Python 3 (Dirk Thomas added force-x-python3-version and X-Python3-Version and Louis for Recommends3, Suggests3, Provides3 and Replaces3 support. Supports Debian Squeeze (6), Debian Wheezy (7), Ubuntu Precise (12.04), Ubuntu Trusty (14.04) and later releases.

• 2015-02-14: Version 0.8.3. See the download page. This is a bugfix release which fixes several aspects of Unicode support. Tests pass on Debian Squeeze (6), Debian Wheezy (7), and Ubuntu Precise (12.04). Support for Python 3.4 (e.g. Ubuntu Trusty 14.04) was mistakenly broken and was fixed in the 0.8.3 release.

• 2014-8-14: Version 0.8.2. See the download page. This is a bugfix release fixing a serious issue that would cause a Python 2 package to be built if only a Python 3 package was requested in some circumstances.

• 2014-8-10: Version 0.8.1. See the download page. Due to bugs in 0.8.0, this release is the first announced from the 0.8 series. Highlights since 0.7.1:

  • Full support for Python 3. This includes being run from Python 3 and generating packages for Python 3. The default is to build Python 3 packages when run with Python 3 and to build Python 2 packages when run from Python 2. Command line options can be used to build packages for the other Python interpreter, too.

  • Build .changes file for source package. While this still must be signed for upload to a PPA, for example, it should still be useful in some cases.

  • Switch to Debian source format 3.0 (quilt). Practically speaking, the .diff.gz file that used to come with a source package is now replaced by a .debian.tar.gz file.

  • Verify SSL certificates when talking to PyPI using Requests. (Verification requires Requests >= 0.8.8.)

  • Many bugfixes.

• 2014-05-05: Version 0.7.1. See the download page. Highlights for this release (you may also wish to consult the full changelog). Due to bugs in 0.7.0, this release is the first announced from the 0.7 series. Highlights since 0.6.0:

  • New commands: pypi-download and pypi-install to directly download and install packages from PyPI, respectively. py2dsc-deb directly creates a .deb file from a source tarball.

  • New distutils command: install_deb lets you directly install a python package as a standard system package.

  • Many bugfixes, including the new URL for PyPI.

  • Automated runs of test suite, thanks to Travis CI

  • Thanks to many, especially Piotr Ożarowski for help with stdeb.

• 2010-06-18: Version 0.6.0. See the download page. Highlights for this release (you may also wish to consult the full changelog):

  • A new debianize command to build a debian/ directory alongside your setup.py file.

  • Bugfixes.

• 2010-01-09: Version 0.5.1. Bugfix release. See the download page, the changelog and release notes.

• 2009-12-30: Version 0.5.0. See the download page. Highlights for this release (you may also wish to consult the full changelog):

  • A new pypi-install script will automatically download, make a .deb, and install packages from the Python Package Index (PyPI).

  • Removal of the setuptools dependency.

  • New option (–guess-conflicts-provides-replaces) to query original Debian packages for Conflicts/Provides/Replaces information.

  • As a result of these changes and to fix a couple bugs/warts, some minor backwards incompatible changes and deprecations were made. Please check the release notes.

• 2009-12-28: Version 0.4.3 Released. See the download page. See the changelog and release notes.

• 2009-11-02: Version 0.4.2 Released. See the download page. See the changelog and release notes.

• 2009-10-04: Version 0.4.1 Released. See the download page. See the changelog and release notes.

• 2009-09-27: Version 0.4 Released. See the download page. This version switches to debhelper 7. See the Changelog for 0.4.

Releases up to and including 0.3.2 are compatible with Ubuntu Hardy.

• 2009-10-04: Version 0.3.2 Released. See the download page. See the Changelog for 0.3.2

• 2009-09-27: Version 0.3.1 Released. See the download page. See the Changelog for 0.3.1

• 2009-03-21: Version 0.3 Released. See the download page. See the Changelog for 0.3

• 2009-02-17: Version 0.2.3 Released. See the download page. See the Changelog for 0.2.3

• 2009-01-29: Version 0.2.2 Released. See the download page. See the Changelog for 0.2.2

• 2008-04-26: Version 0.2.1 Released. See the download page. See the Changelog for 0.2.1

• 2008-04-26: Version 0.2 Released. See the download page. See the Changelog for 0.2

• 2007-04-02: Version 0.2.a1 Released. See the old download page.

• 2006-06-19: Version 0.1 Released. See the old download page.

The commands

pypi-download, command-line command

pypi-download takes a package name, queries PyPI for it and downloads it:

pypi-download [options] mypackage

pypi-install, command-line command

pypi-install takes a package name, queries PyPI for it, downloads it, builds a Debian source package and then .deb from it, and this installs it:

pypi-install [options] mypackage

py2dsc, command-line command

py2dsc takes a .tar.gz source package and build a Debian source package from it:

py2dsc [options] mypackage-0.1.tar.gz # uses pre-built Python source package

py2dsc-deb, command-line command

py2dsc-deb takes a .tar.gz source package and build a Debian source package and then a .deb file from it:

py2dsc-deb [options] mypackage-0.1.tar.gz # uses pre-built Python source package

sdist_dsc, distutils command

All methods eventually result in a call to the sdist_dsc distutils command. You may prefer to do so directly:

python3 setup.py --command-packages=stdeb3.command sdist_dsc

A Debian source package is produced from unmodified Python packages. The following files are produced in a newly created subdirectory deb_dist:

• packagename_versionname.orig.tar.gz

• packagename_versionname-debianversion.dsc

• packagename_versionname-debianversion.diff.gz

These can then be compiled into binary packages using the standard Debian machinery (e.g. dpkg-buildpackage).

bdist_deb, distutils command

A bdist_deb distutils command is installed. This calls the sdist_dsc command and then runs dpkg-buildpackage on the result:

python3 setup.py --command-packages=stdeb3.command bdist_deb

install_deb, distutils command

The install_deb distutils command calls the bdist_deb command and then installs the result. You need to run this with superuser privilege:

sudo python3 setup.py --command-packages=stdeb3.command install_deb

debianize, distutils command

The debianize distutils command builds the same debian/ directory as used in the previous command, but the output is placed directly in the project’s root folder (alongside setup.py). This is useful for customizing the Debian package directly (rather than using the various stdeb options to tune the generated package).

python3 setup.py --command-packages=stdeb3.command debianize

A note about telling distutils to use the stdeb distutils commands

Distutils command packages can also be specified in distutils configuration files (rather than using the --command-packages command line argument to setup.py), as specified in the distutils documentation. Specifically, you could include this in your ~/.pydistutils.cfg file:

[global]
command-packages: stdeb3.command

Examples

These all assume you have stde3b installed in your system Python path. stdeb3 also works from a non-system Python path (e.g. a virtualenv).

Quickstart 1: Install something from PyPI now, I don’t care about anything else

Do this from the command line:

pypi-install mypackage

Warning: Despite doing its best, there is absolutely no way stdeb can guarantee all the Debian package dependencies will be properly fulfilled without manual intervention. Using pypi-install bypasses your ability to customize stdeb’s behavior. Read the rest of this document to understand how to make better packages.

Quickstart 2: Just tell me the fastest way to make a .deb

(First, install stdeb3 as you normally install Python packages.)

Do this from the directory with your setup.py file:

python3 setup.py --command-packages=stdeb3.command bdist_deb

This will make a Debian source package (.dsc, .orig.tar.gz and .diff.gz files) and then compile it to a Debian binary package (.deb) for your current system. The result will be in deb_dist.

Warning: installing the .deb file on other versions of Ubuntu or Debian than the one on which it was compiled will result in undefined behavior. If you have extension modules, they will probably break. Even in the absence of extension modules, bad stuff will likely happen.

For this reason, it is much better to build the Debian source package and then compile that (e.g. using Ubuntu’s PPA) for each target version of Debian or Ubuntu.

Quickstart 3: I read the warning, so show me how to make a source package, then compile it

This generates a source package:

wget http://pypi.python.org/packages/source/R/Reindent/Reindent-0.1.0.tar.gz
py2dsc Reindent-0.1.0.tar.gz

This turns it into a .deb using the standard Debian tools. (Do this on the same source package for each target distribution):

cd deb_dist/reindent-0.1.0/
dpkg-buildpackage -rfakeroot -uc -us

This installs it:

cd ..
sudo dpkg -i python-reindent_0.1.0-1_all.deb

Quickstart 4: Install from a Python package direct to a debian system package

(First, install stdeb as you normally install Python packages.)

Do this from the directory with your setup.py file:

python3 setup.py --command-packages=stdeb3.command install_deb

This will make a Debian source package (.dsc, .orig.tar.gz and .diff.gz files), compile it to a Debian binary package (.deb) for your current system and then install it using dpkg.

Another example, with more explanation

This example is more useful if you don’t have a Python source package (.tar.gz file generated by python setup.py sdist). For the sake of illustration, we do download such a tarball, but immediately unpack it (alternatively, use a version control system to grab the unpacked source of a package):

wget http://pypi.python.org/packages/source/R/Reindent/Reindent-0.1.0.tar.gz
tar xzf Reindent-0.1.0.tar.gz
cd Reindent-0.1.0

The following will generate a directory deb_dist containing the files reindent_0.1.0-1.dsc, reindent_0.1.0.orig.tar.gz and reindent_0.1.0-1.diff.gz, which, together, are a debian source package:

python3 setup.py --command-packages=stdeb3.command sdist_dsc

The source generated in the above way is also extracted (using dpkg-source -x) and placed in the deb_dist subdirectory. To continue the example above:

cd deb_dist/reindent-0.1.0
dpkg-buildpackage -rfakeroot -uc -us

Finally, the generated package can be installed:

cd ..
sudo dpkg -i python-reindent_0.1.0-1_all.deb

For yet another example of use, with still more explanation, see allmydata-tahoe ticket 251.

Download

Files are available at the download page (for ancient releases, see the old download page).

The git repository is available at http://github.com/astraw/stdeb

Install (or, using stdeb to create an stdeb installer)

For a bit of fun, here’s how to install stdeb using stdeb. Note that stdeb is also in Debian and Ubuntu, so this recipe is only necessary to install a more recent stdeb.

STDEB3_VERSION="0.9.0"

# Download stdeb3
wget http://pypi.python.org/packages/source/s/stdeb3/stdeb3-$STDEB3_VERSION.tar.gz

# Extract it
tar xzf stdeb3-$STDEB3_VERSION.tar.gz

# Enter extracted source package
cd stdeb3-$STDEB3_VERSION

# Build .deb (making use of stdeb package directory in sys.path).
python3 setup.py --command-packages=stdeb3.command bdist_deb

# Install it
sudo dpkg -i deb_dist/python-stdeb3_$STDEB3_VERSION-1_all.deb

Background

For the average Python package, its source distribution (python_package.tar.gz created with python3 setup.py sdist) contains nearly everything necessary to make a Debian source package. This near-equivalence encouraged me to write this distutils extension, which executes the setup.py file to extract relevant information. setuptools may optionally be used.

I wrote this initially to Debianize several Python packages of my own, but I have the feeling it could be generally useful. It appears similar, at least in theory, to easydeb, Logilab’s Devtools, bdist_dpkg, bdist_deb, pkgme and dh-virtualenv.

Features

• Create a package for all Python versions supported by python-support. (Limiting this range is possible with the XS-Python-Version: config option.)

• Automatic conversion of Python package names into valid Debian package names.

• Attempt to automatically convert version numbers such that ordering is maintained. See also the config option Forced-Upstream-Version.

• Fine grained control of version numbers. (Debian-Version, Forced-Upstream-Version, Upstream-Version-Prefix, Upstream-Version-Suffix config options.)

• Install .desktop files. (MIME-Desktop-Files config option.)

• Install .mime and .sharedmimeinfo files. (MIME-File and Shared-MIME-File config options.)

• Install copyright files. (Copyright-File config option.)

• Apply patches to upstream sources. (Stdeb-Patch-File config option.)

• Pass environment variables to setup.py script. (Setup-Env-Vars config option.)

Customizing the produced Debian source package (config options)

stdeb will attempt to provide reasonable defaults, but these are only guesses.

There are two ways to customize the Debian source package produced by stdeb. First, you may provide options to the distutils commands. Second, you may provide an stdeb.cfg file.

stdeb distutils command options

The sdist_dsc command takes command-line options to the distutils command. For example:

python3 setup.py --command-packages=stdeb3.command sdist_dsc --debian-version 0MyName1

This creates a Debian package with the Debian version set to “0MyName1”.

These options can also be set via distutils configuration files. (These are the setup.cfg file alongside setup.py and the ~/.pydistutils.cfg file.) In that case, put the arguments in the [sdist_dsc] section. For example, a project’s ~/.setup.cfg file might have this:

[sdist_dsc]
debian-version: 0MyName1

To pass these commands to sdist_dsc when calling bdist_deb, do this:

python3 setup.py sdist_dsc --debian-version 0MyName1 bdist_deb

Command line option	Effect
–with-python2	build Python 2 package (default=True)
–with-python3	build Python 3 package (default=False)
–no-python2-scripts	disable installation of Python 2 scripts (default=False)
–no-python3-scripts	disable installation of Python 3 scripts (default=False)
–force-x-python3-version	Override default minimum python3:any dependency with value from x-python3- version
–allow-virtualenv-install-location	Allow installing into /some/random/virtualenv-path
–sign-results	Use gpg to sign the resulting .dsc and .changes file
–sign-key	Use this gpg key signature to sign
–dist-dir (-d)	directory to put final built distributions in (default=’deb_dist’)
–patch-already-applied (-a)	patch was already applied (used when py2dsc calls sdist_dsc)
–default-distribution	deprecated (see –suite)
–suite (-z)	distribution name to use if not specified in .cfg (default=’unstable’)
–default-maintainer	deprecated (see –maintainer)
–maintainer (-m)	maintainer name and email to use if not specified in .cfg (default from setup.py)
–extra-cfg-file (-x)	additional .cfg file (in addition to stdeb.cfg if present)
–patch-file (-p)	patch file applied before setup.py called (incompatible with file specified in .cfg)
–patch-level (-l)	patch file applied before setup.py called (incompatible with file specified in .cfg)
–patch-posix (-q)	apply the patch with –posix mode
–remove-expanded-source-dir (-r)	remove the expanded source directory
–ignore-install-requires (-i)	ignore the requirements from requires.txt in the egg-info directory
–guess-conflicts-provides-replaces	If True, attempt to guess Conflicts/Provides/Replaces in debian/control based on apt-cache output. (Default=False).
–use-premade-distfile (-P)	use .zip or .tar.gz file already made by sdist command
–source	debian/control Source: (Default: <source-debianized-setup-name>)
–package	debian/control Package: (Default: python-<debianized-setup-name>)
–suite	suite (e.g. stable, lucid) in changelog (Default: unstable)
–maintainer	debian/control Maintainer: (Default: <setup-maintainer-or-author>)
–debian-version	debian version (Default: 1)
–section	debian/control Section: (Default: python)
–epoch	version epoch
–forced-upstream-version	forced upstream version
–upstream-version-prefix	upstream version prefix
–upstream-version-suffix	upstream version suffix
–uploaders	uploaders
–copyright-file	copyright file
–build-depends	debian/control Build-Depends:
–build-conflicts	debian/control Build-Conflicts:
–stdeb-patch-file	file containing patches for stdeb to apply
–stdeb-patch-level	patch level provided to patch command
–depends	debian/control Depends:
–suggests	debian/control Suggests:
–recommends	debian/control Recommends:
–xs-python-version	debian/control XS-Python-Version:
–x-python3-version	debian/control X-Python3-Version:
–dpkg-shlibdeps-params	parameters passed to dpkg-shlibdeps
–conflicts	debian/control Conflicts:
–provides	debian/control Provides:
–replaces	debian/control Replaces:
–mime-desktop-files	MIME desktop files
–mime-file	MIME file
–shared-mime-file	shared MIME file
–setup-env-vars	environment variables passed to setup.py
–udev-rules	file with rules to install to udev
–with-dh-addons	takes a comma separated list of debhelper add-ons to include in the rules file

You may also pass any arguments described below for the stdeb.cfg file via distutils options. Passing the arguments this way (either on the command line, or in the [sdist_dsc] section of a distutils .cfg file) will take precedence. The option name should be given in lower case.

stdeb.cfg configuration file

You may write config files of the format understood by ConfigParser. When building each package, stdeb looks for the existance of a stdeb.cfg in the directory with setup.py. You may specify an additional config file with the command-line option –extra-cfg-file. The section should should either be [DEFAULT] or [package_name], which package_name is specified as the name argument to the setup() command. An example stdeb.cfg file is:

[DEFAULT]
Depends: python3-numpy
XS-Python-Version: >= 2.7

All available options:

Config file option	Effect
Source	debian/control Source: (Default: <source-debianized-setup-name>)
Package	debian/control Package: (Default: python-<debianized-setup-name>)
Suite	suite (e.g. stable, lucid) in changelog (Default: unstable)
Maintainer	debian/control Maintainer: (Default: <setup-maintainer-or-author>)
Debian-Version	debian version (Default: 1)
Section	debian/control Section: (Default: python)
Epoch	version epoch
Forced-Upstream-Version	forced upstream version
Upstream-Version-Prefix	upstream version prefix
Upstream-Version-Suffix	upstream version suffix
Uploaders	debian/control Uploaders
Description	override the default auto-generated debian/control Description
Long-Description	override the default debian/control long-description
Homepage	debian/control Homepage:
Copyright-File	copyright file
Changelog	specify your own debian/changelog file
News-File	file to be used as debian/NEWS
Build-Depends	debian/control Build-Depends:
Build-Conflicts	debian/control Build-Conflicts:
Stdeb-Patch-File	file containing patches for stdeb to apply
Stdeb-Patch-Level	patch level provided to patch command
Depends	debian/control Depends:
Depends3	debian/control Depends: for python3
Suggests	debian/control Suggests:
Suggests3	debian/control Suggests: for python3
Recommends	debian/control Recommends:
Recommends3	debian/control Recommends: for python3
XS-Python-Version	debian/control XS-Python-Version:
X-Python3-Version	debian/control X-Python3-Version:
Dpkg-Shlibdeps-Params	parameters passed to dpkg-shlibdeps
Conflicts	debian/control Conflicts:
Conflicts3	debian/control Conflicts: for python3
Breaks	debian/control Breaks:
Breaks3	debian/control Breaks: for python3
Provides	debian/control Provides:
Provides3	debian/control Provides: for python3
Replaces	debian/control Replaces:
Replaces3	debian/control Replaces: for python3
MIME-Desktop-Files	MIME desktop files
MIME-File	MIME file
Shared-MIME-File	shared MIME file
Setup-Env-Vars	environment variables passed to setup.py
Udev-Rules	file with rules to install to udev

The option names in stdeb.cfg files are not case sensitive.

Prerequisites

• Python 3.4+

• Standard Debian utilities such as date, dpkg-source and Debhelper 9 (use stdeb 0.8.x if you need to support older distributions without dh9)

• If your setup.py uses the setuptools features setup_requires or install_requires, you must run apt-file update prior to running any stdeb command.

TODO

• Make output meet Debian Python Policy specifications or the new python policy. This will include several things, among which are:

  • the ability to include project-supplied documentation as a -doc package

  • the ability to include project-supplied examples, tests, and data as a separate package

  • much more not listed

• Create (better) documentation

• Log output using standard distutils mechanisms

• Refactor the source code to have a simpler, more sane design

License

MIT-style license. Copyright (c) 2006-2019 stdeb authors.

See the LICENSE.txt file provided with the source distribution for full details.

Authors

• Ashley Sommer <[e-mail]>

• Andrew Straw <[e-mail]>

• Pedro Algarvio, aka, s0undt3ch <[e-mail]>

• Gerry Reno (initial bdist_deb implementation)

Additional Credits

• Zooko O’Whielacronx for the autofind-depends patch.

• Brett (last name unknown) for the –ignore-install-requires patch.

• Ximin Luo for a bug fix.

• Alexander D. Sedov for bug fixes and suggestions.

• Michele Mattioni for bug fix.

• Alexander V. Nikolaev for the debhelper buildsystem specification.

• Roland Sommer for the description field bugfix.

• Barry Warsaw for suggesting the debianize command.

• Asheesh Laroia for updating the PyPI URL.

• Piotr Ożarowski for implementing dh_python2 support.

• Nikita Burtsev for unicode tests and fixes

• Mikołaj Siedlarek for a bugfix

• Dirk Thomas for –force-x-python3-version and X-Python3-Version

• Louis for Recommends3, Suggests3, Provides3 and Replaces3 support

• kzwin for interop with virtualenv

• GitHub for hosting services.

• WebFaction (aka python-hosting) for previous hosting services.

• TravisCI for continuous integration