Thread batch with timeout - return values in dict
- import threadingbatch
- add the decorator @threadingbatch.thread_capture to your function
- add *args, **kwargs to your function
- create a list of lists with all function calls
- call threadingbatch.start_all_threads
- get the results from threadingbatch.results
$ pip install threadingbatch
from kthread_sleep import sleep # killing might not work with time.sleep ( pip install kthread-sleep )
import threadingbatch
import random
@threadingbatch.thread_capture # The threaded function must be decorated
def test(
testnumber, *args, **kwargs
): # a threaded function must have *args, **kwargs and can't have the keyword argument _starttime
print(f"start {testnumber}")
sleep(1)
v = random.randrange(1, 30)
print(f"end {testnumber}")
return v
flist = []
for ini, k in enumerate(range(20)): # creating 20 function calls
flist.append(
[
test, # function
(), # args
{"testnumber": ini}, # kwargs
f"function_{str(ini)}", # key in threadingbatch.results (must be unique and type str), the key can't have the name "done"
]
)
flistt = threadingbatch.start_all_threads(
flist,
threadtlimit=5, # number of simultaneously executed threads
timeout=4, # call Kthread.kill after n seconds
sleepafterkill=0.02, # sleep time after calling Kthread.kill
sleepafterstart=0.02, # sleep time after starting a thread
ignore_exceptions=False,
verbose=False,
)
while not threadingbatch.results[
"done"
]: # when all threads are done, threadingbatch.results['done'] changes to True
pass
sleep(0.1)
# output:
# start 0
# start 1
# start 2
# start 3
# start 4
# start 5
# end 0
# start 6
# end 1
# start 7
# end 2
# start 8
# end 3
# ....
print(threadingbatch.results)
# defaultdict(<function threadingbatch.<lambda>()>,
# {'done': True,
# 'function_19': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 11, 'realstart': 1673158006.740858}),
# 'function_18': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 23, 'realstart': 1673158006.4810376}),
# 'function_17': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 14, 'realstart': 1673158005.2808566}),
# 'function_16': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 1, 'realstart': 1673158005.1208546}),
# 'function_15': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 4, 'realstart': 1673158005.0807495}),
# 'function_14': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 23, 'realstart': 1673158004.7609262}),
# 'function_13': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 15, 'realstart': 1673158004.7207859}),
# 'function_12': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 4, 'realstart': 1673158004.5210721}),
# 'function_11': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 15, 'realstart': 1673158002.9808593}),
# 'function_10': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 4, 'realstart': 1673158002.941067}),
# 'function_9': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 20, 'realstart': 1673158002.9007895}),
# 'function_8': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 29, 'realstart': 1673158002.420729}),
# 'function_7': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 6, 'realstart': 1673158002.3609126}),
# 'function_6': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 16, 'realstart': 1673158002.3207767}),
# 'function_5': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 1, 'realstart': 1673158000.5775528}),
# 'function_4': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 18, 'realstart': 1673158000.5409462}),
# 'function_3': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 23, 'realstart': 1673158000.5010219}),
# 'function_2': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 14, 'realstart': 1673158000.4609187}),
# 'function_1': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 10, 'realstart': 1673158000.420832}),
# 'function_0': defaultdict(<function threadingbatch.<lambda>()>,
# {'results': 12, 'realstart': 1673158000.370694})})
Metadata
Release files for threadingbatch 0.10
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| threadingbatch-0.10.tar.gz | 6.0 kB | Without provenance | Details for threadingbatch-0.10.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| threadingbatch-0.10-py3-none-any.whl | Python 3 | none | any | Details for threadingbatch-0.10-py3-none-any.whl |
Total release size: 13.3 kB
Release files / threadingbatch-0.10.tar.gz
| Download URL | threadingbatch-0.10.tar.gz |
|---|---|
| Size | 6.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c8c663a0711e8a045ae08f9f98dba12ee2d06e11d796baceec59b531ffccbcf8
|
|
BLAKE2b-256 checksum How to use checksums |
6bde49d92d81647a70e5930d1469ae70dc4db0f2b807e74874abafb067819c04
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.9.13
|
Release files / threadingbatch-0.10-py3-none-any.whl
| Download URL | threadingbatch-0.10-py3-none-any.whl |
|---|---|
| Size | 7.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0f8b3037663ef0701317a246dc17ec16dbb9fa7d8113498a14e33912d6562726
|
|
BLAKE2b-256 checksum How to use checksums |
bb1eecc7e9a0c0102e278ddfea97dd6ee195712a9d7a5707343433169d86623f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.9.13
|
Security for threadingbatch 0.10
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.