pseq 0.1.4

A framework for parallel processing of sequences.

This package provides a framework for parallel processing of sequences.

Copyright ® 2019, Luís Gomes [e-mail]. All rights reserved.

Links: https://github.com/luismsgomes/pseq | https://pypi.org/project/pseq/

There are four main concepts in this package, each one modeled as a class:

• A WorkUnit represents a sequence element.

• A Producer is responsible for generating a sequence of WorkUnits to be processed.

• A Processor is responsible for processing WorkUnits individually.

• A Consumer is responsible for handling processed WorkUnits.

These concepts/objects are the building blocks of a ParallelSequenceProcessor, which is depicted below.

                             .-------------.
                          .->| Processor 1 |--.
                          |  `-------------'  |
.----------.  .-------.  /   .-------------.   \   .------.  .----------.
| Producer |->| Queue |-+--->| Processor 2 |----+->| Hold |->| Consumer |
`----------'  `-------'  \   `-------------'   /   `------'  `----------'
                          |       ...         |
                          |  .-------------.  |
                          `->| Processor k |--'
                             `-------------'

A ParallelSequenceProcessor coordinates one Producer, one Consumer and k Processors.

Unless specified otherwise, the Consumer receives processed WorkUnits in the same order as produced by the Producer. This ordering is ensured by holding processed WorkUnits until all units produced before them have been processed and passed onto the Consumer.

Usage

Currently, the only documentation available are two usage examples.

See examples/mpseq-core-example.py for the simplest scenario: processing a single sequence of a single type.

See examples/mpseq-poly-example.py for an example of processing multiple sequences of different types.

License

This library is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 3 as published by the Free Software Foundation.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License version 3 for more details.

You should have received a copy of the GNU General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA