Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Decorators for running functions in Thread/ThreadPool/IOLoop

Project description

threaded

https://travis-ci.com/python-useful-helpers/threaded.svg?branch=master Azure DevOps builds https://coveralls.io/repos/github/python-useful-helpers/threaded/badge.svg?branch=master Documentation Status https://img.shields.io/pypi/v/threaded.svg https://img.shields.io/pypi/pyversions/threaded.svg https://img.shields.io/pypi/status/threaded.svg https://img.shields.io/github/license/python-useful-helpers/threaded.svg https://img.shields.io/badge/code%20style-black-000000.svg

threaded is a set of decorators, which wrap functions in:

  • concurrent.futures.ThreadPool
  • threading.Thread
  • asyncio.Task in Python 3.

Why? Because copy-paste of loop.create_task, threading.Thread and thread_pool.submit is boring, especially if target functions is used by this way only.

Pros:

Python 3.4
Python 3.5
Python 3.6
Python 3.7
PyPy3 3.5+

Note

For python 2.7/PyPy you can use versions 1.x.x

Decorators:

  • ThreadPooled - native concurrent.futures.ThreadPool.
  • threadpooled is alias for ThreadPooled.
  • Threaded - wrap in threading.Thread.
  • threaded is alias for Threaded.
  • AsyncIOTask - wrap in asyncio.Task. Uses the same API, as ThreadPooled.
  • asynciotask is alias for AsyncIOTask.

Usage

ThreadPooled

Mostly it is required decorator: submit function to ThreadPoolExecutor on call.

Note

API quite differs between Python 3 and Python 2.7. See API section below.

threaded.ThreadPooled.configure(max_workers=3)

Note

By default, if executor is not configured - it configures with default parameters: max_workers=CPU_COUNT * 5

@threaded.ThreadPooled
def func():
    pass

concurrent.futures.wait([func()])

Python 3.5+ usage with asyncio:

Note

if loop_getter is not callable, loop_getter_need_context is ignored.

loop = asyncio.get_event_loop()
@threaded.ThreadPooled(loop_getter=loop, loop_getter_need_context=False)
def func():
    pass

loop.run_until_complete(asyncio.wait_for(func(), timeout))

Python 3.5+ usage with asyncio and loop extraction from call arguments:

loop_getter = lambda tgt_loop: tgt_loop
@threaded.ThreadPooled(loop_getter=loop_getter, loop_getter_need_context=True)  # loop_getter_need_context is required
def func(*args, **kwargs):
    pass

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait_for(func(loop), timeout))

During application shutdown, pool can be stopped (while it will be recreated automatically, if some component will request).

threaded.ThreadPooled.shutdown()

Threaded

Classic threading.Thread. Useful for running until close and self-closing threads without return.

Usage example:

@threaded.Threaded
def func(*args, **kwargs):
    pass

thread = func()
thread.start()
thread.join()

Without arguments, thread name will use pattern: 'Threaded: ' + func.__name__

Note

If func.__name__ is not accessible, str(hash(func)) will be used instead.

Override name can be don via corresponding argument:

@threaded.Threaded(name='Function in thread')
def func(*args, **kwargs):
    pass

Thread can be daemonized automatically:

@threaded.Threaded(daemon=True)
def func(*args, **kwargs):
    pass

Also, if no any addition manipulations expected before thread start, it can be started automatically before return:

@threaded.Threaded(started=True)
def func(*args, **kwargs):
    pass

AsyncIOTask

Wrap in asyncio.Task.

usage with asyncio:

@threaded.AsyncIOTask
def func():
    pass

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait_for(func(), timeout))

Provide event loop directly:

Note

if loop_getter is not callable, loop_getter_need_context is ignored.

loop = asyncio.get_event_loop()
@threaded.AsyncIOTask(loop_getter=loop)
def func():
    pass

loop.run_until_complete(asyncio.wait_for(func(), timeout))

Usage with loop extraction from call arguments:

loop_getter = lambda tgt_loop: tgt_loop
@threaded.AsyncIOTask(loop_getter=loop_getter, loop_getter_need_context=True)
def func(*args, **kwargs):
    pass

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait_for(func(loop), timeout))

Testing

The main test mechanism for the package threaded is using tox. Available environments can be collected via tox -l

CI systems

For code checking several CI systems is used in parallel:

  1. Travis CI: is used for checking: PEP8, pylint, bandit, installation possibility and unit tests. Also it’s publishes coverage on coveralls.
  2. coveralls: is used for coverage display.
  3. Azure CI: is used for functional tests on Windows.

CD system

Travis CI: is used for package delivery on PyPI.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for threaded, version 4.0.8
Filename, size File type Python version Upload date Hashes
Filename, size threaded-4.0.8-cp36-cp36m-manylinux1_i686.whl (627.7 kB) File type Wheel Python version cp36 Upload date Hashes View hashes
Filename, size threaded-4.0.8-cp36-cp36m-manylinux1_x86_64.whl (679.7 kB) File type Wheel Python version cp36 Upload date Hashes View hashes
Filename, size threaded-4.0.8-cp36-cp36m-win32.whl (162.1 kB) File type Wheel Python version cp36 Upload date Hashes View hashes
Filename, size threaded-4.0.8-cp36-cp36m-win_amd64.whl (189.5 kB) File type Wheel Python version cp36 Upload date Hashes View hashes
Filename, size threaded-4.0.8-cp37-cp37m-manylinux1_i686.whl (629.5 kB) File type Wheel Python version cp37 Upload date Hashes View hashes
Filename, size threaded-4.0.8-cp37-cp37m-manylinux1_x86_64.whl (680.3 kB) File type Wheel Python version cp37 Upload date Hashes View hashes
Filename, size threaded-4.0.8-cp37-cp37m-win32.whl (162.1 kB) File type Wheel Python version cp37 Upload date Hashes View hashes
Filename, size threaded-4.0.8-cp37-cp37m-win_amd64.whl (189.2 kB) File type Wheel Python version cp37 Upload date Hashes View hashes
Filename, size threaded-4.0.8-cp38-cp38-manylinux1_i686.whl (681.4 kB) File type Wheel Python version cp38 Upload date Hashes View hashes
Filename, size threaded-4.0.8-cp38-cp38-manylinux1_x86_64.whl (741.6 kB) File type Wheel Python version cp38 Upload date Hashes View hashes
Filename, size threaded-4.0.8-py3-none-any.whl (17.7 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size threaded-4.0.8.tar.gz (26.9 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page