Skip to main content

A framework for parallel processing of sequences.

Project description

Parallel Sequence Processing

This package provides a framework for parallel processing of sequences.

Copyright ® 2019, Luís Gomes luismsgomes@gmail.com. All rights reserved.

Links: https://github.com/luismsgomes/pseq | https://pypi.org/project/pseq/

There are four main concepts in this package, each one modeled as a class:

  • A WorkUnit represents a sequence element.
  • A Producer is responsible for generating a sequence of WorkUnits to be processed.
  • A Processor is responsible for processing WorkUnits individually.
  • A Consumer is responsible for handling processed WorkUnits.

These concepts/objects are the building blocks of a ParallelSequenceProcessor, which is depicted below.

The ParallelSequenceProcessor in this figure has been configured with two lanes for priority 0 (lanes 1 and 2) and a single lane for priority 1 (lane 3).

Unless specified otherwise, by setting require_in_order=False, the Consumer receives processed WorkUnits in the same order as produced by the Producer directly above it (in the same lane). This ordering is ensured by holding processed WorkUnits until all units produced before them have been processed and passed onto the Consumer.

Usage

Currently, the only documentation available is the example code provided in example.py.

License

This library is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 3 as published by the Free Software Foundation.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License version 3 for more details.

You should have received a copy of the GNU General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pseq, version 2.1.5
Filename, size File type Python version Upload date Hashes
Filename, size pseq-2.1.5.tar.gz (5.4 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page