Skip to main content

Thread batch with timeout - return values in dict

  1. import threadingbatch
  2. add the decorator @threadingbatch.thread_capture to your function
  3. add *args, **kwargs to your function
  4. create a list of lists with all function calls 
  5. call threadingbatch.start_all_threads
  6. get the results from threadingbatch.results
$ pip install threadingbatch
from kthread_sleep import sleep  # killing might not work with time.sleep ( pip install kthread-sleep )
import threadingbatch
import random


@threadingbatch.thread_capture  # The threaded function must be decorated
def test(
    testnumber, *args, **kwargs
):  # a threaded function must have *args, **kwargs and can't have the keyword argument _starttime

    print(f"start {testnumber}")
    sleep(1)
    v = random.randrange(1, 30)
    print(f"end {testnumber}")
    return v


flist = []
for ini, k in enumerate(range(20)):  # creating 20 function calls

    flist.append(
        [
            test,  # function
            (),  # args
            {"testnumber": ini},  # kwargs
            f"function_{str(ini)}",  # key in threadingbatch.results (must be unique and type str), the key can't have the name "done"
        ]
    )
flistt = threadingbatch.start_all_threads(
    flist,
    threadtlimit=5,  # number of simultaneously executed threads
    timeout=4,  # call Kthread.kill after n seconds
    sleepafterkill=0.02,  # sleep time after calling Kthread.kill
    sleepafterstart=0.02,  # sleep time after starting a thread
    ignore_exceptions=False,
    verbose=False,
)

while not threadingbatch.results[
    "done"
]:  # when all threads are done, threadingbatch.results['done'] changes to True

    pass
    sleep(0.1)

# output:
# start 0
# start 1
# start 2
# start 3
# start 4
# start 5
# end 0
# start 6
# end 1
# start 7
# end 2
# start 8
# end 3
# ....

print(threadingbatch.results)
# defaultdict(<function threadingbatch.<lambda>()>,
#             {'done': True,
#              'function_19': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 11, 'realstart': 1673158006.740858}),
#              'function_18': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 23, 'realstart': 1673158006.4810376}),
#              'function_17': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 14, 'realstart': 1673158005.2808566}),
#              'function_16': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 1, 'realstart': 1673158005.1208546}),
#              'function_15': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 4, 'realstart': 1673158005.0807495}),
#              'function_14': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 23, 'realstart': 1673158004.7609262}),
#              'function_13': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 15, 'realstart': 1673158004.7207859}),
#              'function_12': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 4, 'realstart': 1673158004.5210721}),
#              'function_11': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 15, 'realstart': 1673158002.9808593}),
#              'function_10': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 4, 'realstart': 1673158002.941067}),
#              'function_9': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 20, 'realstart': 1673158002.9007895}),
#              'function_8': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 29, 'realstart': 1673158002.420729}),
#              'function_7': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 6, 'realstart': 1673158002.3609126}),
#              'function_6': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 16, 'realstart': 1673158002.3207767}),
#              'function_5': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 1, 'realstart': 1673158000.5775528}),
#              'function_4': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 18, 'realstart': 1673158000.5409462}),
#              'function_3': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 23, 'realstart': 1673158000.5010219}),
#              'function_2': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 14, 'realstart': 1673158000.4609187}),
#              'function_1': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 10, 'realstart': 1673158000.420832}),
#              'function_0': defaultdict(<function threadingbatch.<lambda>()>,
#                          {'results': 12, 'realstart': 1673158000.370694})})

Metadata

Release files for threadingbatch 0.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for threadingbatch 0.10
File Size Uploaded Provenance Actions
threadingbatch-0.10.tar.gz 6.0 kB Without provenance Details for threadingbatch-0.10.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for threadingbatch 0.10
File Interpreter ABI Platform Actions
threadingbatch-0.10-py3-none-any.whl Python 3 none any Details for threadingbatch-0.10-py3-none-any.whl

Total release size: 13.3 kB

Release files / threadingbatch-0.10.tar.gz

Download URL threadingbatch-0.10.tar.gz
Size 6.0 kB
Tags Source
SHA-256 checksum
How to use checksums
c8c663a0711e8a045ae08f9f98dba12ee2d06e11d796baceec59b531ffccbcf8
BLAKE2b-256 checksum
How to use checksums
6bde49d92d81647a70e5930d1469ae70dc4db0f2b807e74874abafb067819c04
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.1 CPython/3.9.13

Release files / threadingbatch-0.10-py3-none-any.whl

Download URL threadingbatch-0.10-py3-none-any.whl
Size 7.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
0f8b3037663ef0701317a246dc17ec16dbb9fa7d8113498a14e33912d6562726
BLAKE2b-256 checksum
How to use checksums
bb1eecc7e9a0c0102e278ddfea97dd6ee195712a9d7a5707343433169d86623f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.1 CPython/3.9.13

Security for threadingbatch 0.10

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.10 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page