packtivity

This package aims to collect implementations of both synchronous and asynchronous execution of preserved, but parametrized scientific computational tasks that come with batteries included, i.e. with a full specification of their software dependencies. In that sense they are packaged activities -- packtivities.

This package provides tools to validate and execute data processing tasks that are written according to the "packtivity" JSON schemas defined in yadage-schemas.

Packtivities define

the software environment
parametrized process descriptions (what programs to run within these environment) and
produces human and machine readable outputs (as JSON) of the resulting data fragments.

At run-time they are paired with a concrete set of parameters supplied as JSON documents and and external storage/state to actually execute these tasks.

Packtivity in Yadage

This package is used by yadage to execute the individual steps of yadage workflows.

Example Packtivity spec

This packtivity spec is part of a number of yadage workflow and runs the Delphes detector simulation on a HepMC file and outputs events in the LHCO and ROOT file formats. This packtivity is stored in a public location from which it can be later retrieved:

process:
  process_type: 'string-interpolated-cmd'
  cmd: 'DelphesHepMC  {delphes_card} {outputroot} {inputhepmc} && root2lhco {outputroot} {outputlhco}'
publisher:
  publisher_type: 'frompar-pub'
  outputmap:
    lhcofile: outputlhco
    rootfile: outputroot
environment:
  environment_type: 'docker-encapsulated'
  image: lukasheinrich/root-delphes

Usage

You can run the packtivity in a synchronous way by specifying the spec (can point to GitHub), all necessary parameters and attaching an external state (via the --read and --write flags).

packtivity-run -t from-github/phenochain delphes.yml \
  -p inputhepmc="$PWD/pythia/output.hepmc" \
  -p outputroot="'{workdir}/output.root'" \
  -p outputlhco="'{workdir}/output.lhco'" \
  -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
  --read pythia --write outdir

Asynchronous Backends

In order to facilitate usage of distributed resources, a number of Asynchronous backends can be specified. Here is an example for IPython Parallel clusters

packtivity-run -b ipcluster --asyncwait \
  -t from-github/phenochain delphes.yml \
  -p inputhepmc="$PWD/pythia/output.hepmc" \
  -p outputroot="'{workdir}/output.root'" \
  -p outputlhco="'{workdir}/output.lhco'" \
  -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
  --read pythia --write outdir

You can replacing the --asyncwait with --async flag in order to get a JSONable proxy representation with which to later on check on the job status. By default the proxy information is written to proxy.json (customizable via the -x flag):

packtivity-run -b celery --async \
  -t from-github/phenochain delphes.yml \
  -p inputhepmc="$PWD/pythia/output.hepmc" \
  -p outputroot="'{workdir}/output.root'" \
  -p outputlhco="'{workdir}/output.lhco'" \
  -p delphes_card=delphes/cards/delphes_card_ATLAS.tcl \
  --read pythia --write outdir

And at a later point in time you can check via:

packtivity-checkproxy proxy.json

External Backends

Users can implement their own backends to handle the JSON documents describing the packtivities. It can be enabled by using the fromenv backend and setting an environment variable specifying the module holding the backend and proxy classes. The format of the environment variable is module:backendclass:proxyclass. E.g.:

export PACKTIVITY_ASYNCBACKEND="externalbackend:ExternalBackend:ExternalProxy"

Metadata

Release files for packtivity 0.17.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for packtivity 0.17.1
File	Size	Uploaded
packtivity-0.17.1.tar.gz	35.0 kB	Oct 15, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for packtivity 0.17.1
File	Interpreter	ABI	Platform	Reset
packtivity-0.17.1-py3-none-any.whl 36.5 kB Oct 15, 2026	Python 3	none	any	Details

Total release size: 71.5 kB

Release files / packtivity-0.17.1.tar.gz

Download URL	packtivity-0.17.1.tar.gz
Size	35.0 kB
Tags	Source
SHA-256 checksum How to use checksums	`42b888ba32a7a1931f0e4e27f3264d14e556f7be0443215d2dbc7f464cdf1846`
BLAKE2b-256 checksum How to use checksums	`2a7240a19ad2da2472050bd4418bec113fa777e637f09e19008025b8ddb92926`
Upload date	Oct 15, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Release files / packtivity-0.17.1-py3-none-any.whl

Download URL	packtivity-0.17.1-py3-none-any.whl
Size	36.5 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`d3e9905d0d77c4a9a4a0e0fff0075597246defa8ffc5ef56949810a441bb5309`
BLAKE2b-256 checksum How to use checksums	`50b0a96cf8d68170cbe2bfd1efe7c23eca07fe7efd7c6226a568b4b3134c49a1`
Upload date	Oct 15, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/7.0.0 CPython/3.13.14`