Skip to main content
Archived

This project has been archived by its maintainers, and is no longer receiving any updates.

.. figure:: https://raw.githubusercontent.com/galaxyproject/starforge/master/docs/starforge_logo.png
:alt: Starforge Logo
:align: center
:figwidth: 100%
:target: https://github.com/galaxyproject/Starforge

Starforge: Build `Galaxy`_ things in virtualization

Things you can do with Starforge:

- Build `Galaxy Tool Shed`_ dependencies
- Build `Python Wheels`_ (e.g. for the `Galaxy Wheels Server`_)
- Rebuild Debian or Ubuntu source packages (for modifications)

These things will be built in Docker. Additionally, wheels can be built in
QEMU/KVM virtualized systems.

Documentation can be found at `starforge.readthedocs.org
<http://starforge.readthedocs.org/>`_

Quick Start
-----------

For all types of builds, begin by `installing Docker`_.

Tool Shed Dependencies
~~~~~~~~~~~~~~~~~~~~~~

There are two scripts that can be used, depending on the package recipes
available:

.. sourcecode:: console

$ ./build.sh <package>
$ python build.py <package> --version 1.0

``build.sh`` is the older format, and simply uses a single
``<package>build.sh`` file, like Atlas. `build.py` is the newer format, and
uses yaml metadata in ``<package>/<version>/build.yml``.

The base image for Galaxy packages is Debian Squeeze. This will hopefully
produce binaries usable on Galaxy's targeted platforms (at time of writing:
CentOS 6+, Debian 6.0+, Ubuntu 12.04+).

.. sourcecode:: console

$ ./build galaxy <package>
$ python build.py <package>

To build packages against a different OS, you can use the `--image` flag, e.g.:

.. sourcecode:: console

$ ./build <dist>[:tag] <package>
$ python build.py <package> --image <dist>[:tag]

e.g.

.. sourcecode:: console

$ ./build ubuntu:trusty nginx
$ python build.py nginx --image debian:squeeze

**Building all the things:**

There's a separate ``build-all.sh`` which allows you to build all of the
packages using their preferred build mechanism

Notes on the two build scripts
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

**build.py**

The ``<version>`` option is optional, and defaults to the string ``'default'``,
which is useful for recipes that don't have version specific changes (E.g.
bcftools 1.0 builds identically to 1.2)

Python Wheels
~~~~~~~~~~~~~

Starforge can build both pure Python and C-extension Python modules into wheels
for Linux under Docker and for Mac OS X under QEMU/KVM. To do this, you'll want
to install Starforge (preferably in a Python virtualenv) using ``pip install
starforge`` (to install from PyPI_) or ``python setup.py install`` to install
from the source.

Docker (and QEMU) images to use are specified in `starforge/config/default.yml
<https://github.com/galaxyproject/starforge/blob/master/wheels/build/wheels.yml>`_.
To modify this file, copy it to
``$XDG_CONFIG_HOME/galaxy-starforge/config.yml`` (``$XDG_CONFIG_HOME`` is, by
default ``~/.config``). The sample file `wheels/build/wheels.yml`_ is used to
determine what wheels can be built and their rules for building. To use this
file, use the ``--wheels-config`` argument to ``starforge wheel`` or copy
``wheels.yml`` to ``$XDG_CONFIG_HOME/galaxy-starforge/wheels.yml``.

Wheels can be built using ``starforge wheel <package>``, e.g.:

.. sourcecode:: console

$ starforge wheel pycrypto
$ starforge wheel --no-qemu pysam # only build on docker

See the output of ``starforge --help`` for help using the Starforge command-line interface.

Pull Request wheel builder
^^^^^^^^^^^^^^^^^^^^^^^^^^

Pull requests to the `Starforge`_ repository on Github that modify
`wheels/build/wheels.yml`_ can automatically be built for all specified
platforms on a dedicated Starforge build server by the `Galaxy Jenkins`_
service. To utilize, modify wheels.yml as appropriate and create a pull
request. Any member of the `Galaxy Committers`_ group can then authorize
Jenkins to initiate the build. If it fails, you can modify the pull request and
further builds can be triggered.

Notes on images
^^^^^^^^^^^^^^^

**Linux**

Images used to build wheels on a variety of platforms are uploaded to the
`Starforge Docker Hub`_ repo and will be pulled as necessary. Typically you
will only use the `base-wheel
<https://hub.docker.com/r/starforge/base-wheel/>`_ and `base32-wheel
<https://hub.docker.com/r/starforge/base32-wheel/>`_ images, which are Debian
Squeeze-based images that will usually produce wheels usable on all
Galaxy-supported platforms. The exception is the case when you need to install
non-standard system libraries whose versions or location will differ by Linux
distribution. In this case, a "full" set of images consisting of all
distributions with official images in Docker Hub can be built. This is
controlled in `wheels/build/wheels.yml`_, see ``psycopg2`` for an example.

**Mac OS X**

Mac OS X mages are not provided due to legal reasons. Consult the :doc:`osx`
documentation for details.

.. _Galaxy: http://galaxyproject.org/
.. _Galaxy Tool Shed: http://toolshed.g2.bx.psu.edu/
.. _Python Wheels: http://pythonwheels.com/
.. _Galaxy Wheels Server: http://wheels.galaxyproject.org/
.. _installing Docker: https://docs.docker.com/engine/installation/
.. _PyPI: https://pypi.python.org/
.. _Starforge Docker Hub: https://hub.docker.com/r/starforge/
.. _wheels/build/wheels.yml:
.. _Galaxy Jenkins: http://jenkins.galaxyproject.org
.. _Starforge: https://github.com/galaxyproject/starforge/
.. _Galaxy Committers: https://github.com/galaxyproject/galaxy/blob/dev/doc/source/project/organization.rst


History
-------

Starforge is maintained by the `Galaxy`_ Project and community. A `list of
contributors <https://github.com/galaxyproject/starforge/graphs/contributors>`_
to the project can be found on GitHub.

0.2 (2016-05-19)
~~~~~~~~~~~~~~~~

- Added support for building manylinux1 wheels. 0dbecb7_

0.1.1 (2016-01-20)
~~~~~~~~~~~~~~~~~~

- Only running prebuild during wheel builds (and not sdists) was too naive,
since this prevents changing the version number of sdists in the prebuild
action (a common use of the prebuild action). Instead, allow for separate
``wheel``, ``sdist``, and ``all`` prebuild actions. Reverts the behavior of
9008c57_. `Issue 64`_
- Install Galaxy pip from Github instead of wheels.galaxyproject.org so that
Starforge images can be built with new versions of Galaxy pip before they are
released. 97b4ba4_

0.1 (2016-01-12)
~~~~~~~~~~~~~~~~

- Reimplemented the wheel building scripts as a library and ``starforge``
command line
- Wrote some documentation

Older than 0.1
~~~~~~~~~~~~~~

Originally Galaxy docker-build and later renamed Starforge, but as a collection
of disjointed shell scripts, Python, and YAML used to build Galaxy Tool Shed
dependencies, as well as rebuilding Debian and Ubuntu source packages with
modifications (which itself came from a project created to do the same via
Vagrant and Ansible called vadebuildsible).

.. _Galaxy: http://galaxyproject.org/

.. _9008c57: https://github.com/galaxyproject/starforge/commit/9008c57b09521298b919fac1de00fb62a448bcab
.. _97b4ba4: https://github.com/galaxyproject/starforge/commit/97b4ba4a591e359b01dc69161925c301c9a7d1b7
.. _0dbecb7: https://github.com/galaxyproject/starforge/commit/0dbecb79e28baecb62546b629cae9dbebf46df19

.. _Issue 64: https://github.com/galaxyproject/starforge/issues/64

Metadata

Release files for starforge 0.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for starforge 0.2
File Size Uploaded Provenance Actions
starforge-0.2.tar.gz 21.5 kB Without provenance Details for starforge-0.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for starforge 0.2
File Interpreter ABI Platform Actions
starforge-0.2-py2-none-any.whl Python 2 none any Details for starforge-0.2-py2-none-any.whl

Total release size: 53.1 kB

Release files / starforge-0.2.tar.gz

Download URL starforge-0.2.tar.gz
Size 21.5 kB
Tags Source
SHA-256 checksum
How to use checksums
eb38a0cb41d402172bc4722e61122a40563ce419ed01e6fa68cf4e8570412eac
BLAKE2b-256 checksum
How to use checksums
7e41e5212b1f0a3a6bfb56573d3d90a7db1866b7d9812040cc19748d4c4e054f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release files / starforge-0.2-py2-none-any.whl

Download URL starforge-0.2-py2-none-any.whl
Size 31.6 kB
Tags Python 2
SHA-256 checksum
How to use checksums
8ce9edb93edd133ba43883b05f283a5d3470452e933d4b5275bd2273a66fe061
BLAKE2b-256 checksum
How to use checksums
04d87946fb781d631d6f1facc341eb25483be73eb7d66f6d5b274900d16f9f93
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Security for starforge 0.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page