deco3 0.8.0

A decorator for concurrency.

deco3

Decorated Concurrency.

A simplified parallel computing model for Python. DECO automatically parallelizes Python programs, and requires minimal modifications to existing serial programs.

Overview

deco3 is a strict fork of Alex Sherman’s deco package with a fix allowing to work with Python3 or higher and with a little code reformatting and minor improvements.

PyPI record.

Documentation.

Overview below is a copy from the original deco website (with only the necessary changes regarding deco3).

Documentation

You can reference the Wiki on Github for slightly more in-depth documentation.

General Usage

Using DECO is as simple as finding, or creating, two functions in your Python program. The first function is the one we want to run in parallel, and is decorated with @concurrent. The second function is the function which calls the @concurrent function and is decorated with @synchronized. Decorating the second function is optional, but provides some very cool benefits. Let’s take a look at an example.

@concurrent  # We add this for the concurrent function
def process_lat_lon(lat, lon, data):
  #Does some work which takes a while
  return result

@synchronized  # And we add this for the function which calls the concurrent function
def process_data_set(data):
  results = defaultdict(dict)
  for lat in range(...):
    for lon in range(...):
      results[lat][lon] = process_lat_lon(lat, lon, data)
  return results

That’s it, two lines of changes is all we need in order to parallelize this program. Now this program will make use of all the cores on the machine it’s running on, allowing it to run significantly faster.

What it does

• The @concurrent decorator uses multiprocessing.pool to parallelize calls to the target function

• Indexed based mutation of function arguments is handled automatically, which pool cannot do

• The @synchronized decorator automatically inserts synchronization events

• It also automatically refactors assignments of the results of @concurrent function calls to happen during synchronization events

Limitations

• The @concurrent decorator will only speed up functions that take longer than ~1ms

  • If they take less time your code will run slower!

• By default, @concurrent function arguments/return values must be pickleable for use with multiprocessing

• The @synchronized decorator only works on ‘simple’ functions, make sure the function meets the following criteria

  • Only calls, or assigns the result of @concurrent functions to indexable objects such as:

    • concurrent(…)

    • result[key] = concurrent(…)

  • Never indirectly reads objects that get assigned to by calls of the @concurrent function

How it works

For an in depth discussion of the mechanisms at work, we wrote a paper for a class which can be found here (or original can be found here).

As an overview, DECO is mainly just a smart wrapper for Python’s multiprocessing.pool. When @concurrent is applied to a function it replaces it with calls to pool.apply_async. Additionally when arguments are passed to pool.apply_async, DECO replaces any index mutable objects with proxies, allowing it to detect and synchronize mutations of these objects. The results of these calls can then be obtained by calling wait() on the concurrent function, invoking a synchronization event. These events can be placed automatically in your code by using the @synchronized decorator on functions that call @concurrent functions. Additionally while using @synchronized, you can directly assign the result of concurrent function calls to index mutable objects. These assignments get refactored by DECO to automatically occur during the next synchronization event. All of this means that in many cases, parallel programming using DECO appears exactly the same as simpler serial programming.

Installation

Prerequisites:

• Python 3.10 or higher

  • https://www.python.org/

• pip and setuptools

  • https://pypi.org/project/pip/

  • https://pypi.org/project/setuptools/

To install run:

python -m pip install --upgrade deco3

Development

Prerequisites:

• Development is strictly based on tox. To install it run:

  python -m pip install --upgrade tox

Visit Development page.

Installation from sources:

clone the sources:

git clone https://github.com/karpierz/deco3.git deco3

and run:

python -m pip install ./deco3

or on development mode:

python -m pip install --editable ./deco3

License

Copyright (c) 2025-2025 Adam Karpierz

Copyright (c) 2016 Alex Sherman

Licensed under the MIT License

https://opensource.org/license/mit

Please refer to the accompanying LICENSE file.

Authors

• Alex Sherman <[e-mail]>

• Adam Karpierz <[e-mail]>

Changelog

0.8.0 (2025-04-28)

• Add support for Python 3.14

• Drop support for Python 3.9 (due to compatibility issues).

• Update readthedocs’s python to version 3.13

• Update tox’s base_python to version 3.13

• Setup (dependencies) update.

0.7.0 (2025-04-27)

• 100% code linting.

• Add support for PyPy 3.10 and 3.11

• Add/improve support for Python >= 3.9, <= 3.13

• Drop support for Python <= 3.8

• Drop support for Python 2.

• Copyright year update.

• Tox configuration is now in native (toml) format (as part of

  pyproject.toml) and now based on tox >= 4.0.

• Setup update. Currently based on pyproject.toml.

• Source distribution (*.tar.gz now) is compliant with PEP-0625.

• Creating a fork of Alex Sherman’s deco package with a fix allowing to work with Python3 or higher and newest versions of pip/setuptools.

• Minor improvements and cleanup..

Above are changes of the original (v.0.6.3) deco:

0.6.3 (2025-04-24)

• Initial commit.