Resume your progress in any python loop
Project description
Resu
[Resu]me your progress in any Python loop by automatically creating checkpoints without changing your workflow.
Requirements
Installation
pip install resu
Examples
Example 1:
import time
from resu import Checkpoint
def process(x):
time.sleep(1)
return x + 1
c = Checkpoint()
c.insert(range(1000))
c.record(process)
# 0%|▏ | 24/1000 [00:05<16:40, 1.01s/it]
# ^C KeyboardInterrupt (id: 2) has been caught...
# Saving progress to checkpoint file `./1652598207.ckpt` before terminating the program gracefully...
- You can resume the same loop by calling the
p.resume
method and passing the checkpoint file path to it before running the program again:
# ...
c.resume('1652598207.ckpt')
c.record(process)
# Resuming from `1652598207.ckpt`... Skipped 24 completed enteries.
# 0%| | 2/1000 [00:02<16:40, 1.00s/it]
Example 2:
import time
import requests
from resu import Checkpoint
def process(x, url, cooldown):
headers = {'Accept': 'application/json', 'Content-Type': 'application/json'}
resp = requests.post(url, headers=headers, data=x)
time.sleep(cooldown)
return resp.text
c = Checkpoint(ckpt_file='/my/custom/ckpt/name.ckpt')
c.insert('customers_data.json')
results = c.record(
process,
url='https://reqbin.com/echo/post/json',
cooldown=1,
checkpoint_every=5)
# 0%| | 0/11 [00:00<?, ?it/s]
# 8%|▊ | 1/11 [00:02<00:11, 1.10s/it]
# 17%|█▏ | 2/11 [00:03<00:25, 1.10s/it]
# 27%|██▋ | 3/11 [00:07<00:20, 1.10s/it]
# 35%|███▍ | 4/11 [00:09<00:18, 1.10s/it]
# Saving progress to checkpoint file: `/my/custom/ckpt/name.ckpt`...
# 38%|███▊ | 5/11 [00:11<00:17, 1.10s/it]
# 52%|████▏ | 6/11 [00:12<00:16, 1.10s/it]
# 69%|██████▉ | 7/11 [00:19<00:08, 1.10s/it]
# ...
- Let's say your program failed for some reason, you can easily resume like described in
Example 1
.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
resu-0.2.2.tar.gz
(6.8 kB
view details)
Built Distribution
File details
Details for the file resu-0.2.2.tar.gz
.
File metadata
- Download URL: resu-0.2.2.tar.gz
- Upload date:
- Size: 6.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: python-requests/2.28.1
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a4ba5989fd6b864852da92b7cf725a692ae420b663eb1dbb8a62a979fff1a43c |
|
MD5 | f0457247c67fa16e4a9853c65f6fa630 |
|
BLAKE2b-256 | 2de64cf61d75ea32875f121d31dc776d56bcf4de4952a07000e7913b9fdea17d |
File details
Details for the file resu-0.2.2-py2.py3-none-any.whl
.
File metadata
- Download URL: resu-0.2.2-py2.py3-none-any.whl
- Upload date:
- Size: 5.3 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: python-requests/2.28.1
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ad16f5be2c79bfd34c1709a4bb9419e6a265c4de1d056e5b89aae607c9482a2f |
|
MD5 | b50f935609111ff631b016ebed1446d7 |
|
BLAKE2b-256 | fbad725b5b9bb3d9c2c862fb368a6245788e8b0236af4301b2603b5477156dff |