Resume your progress in any python loop
Project description
Resu
[Resu]me your progress in any Python loop by automatically creating checkpoints without changing your workflow.
Requirements
Installation
pip install resu
Examples
Example 1:
import time
from resu import Checkpoint
def process(x):
time.sleep(1)
return x + 1
c = Checkpoint()
c.insert(range(1000))
c.record(process)
# 0%|▏ | 24/1000 [00:05<16:40, 1.01s/it]
# ^C KeyboardInterrupt (id: 2) has been caught...
# Saving progress to checkpoint file `./1652598207.ckpt` before terminating the program gracefully...
- You can resume the same loop by calling the
p.resume
method and passing the checkpoint file path to it before running the program again:
# ...
c.resume('1652598207.ckpt')
c.record(process)
# Resuming from `1652598207.ckpt`... Skipped 24 completed enteries.
# 0%| | 2/1000 [00:02<16:40, 1.00s/it]
Example 2:
import time
import requests
from resu import Checkpoint
def process(x, url, cooldown):
headers = {'Accept': 'application/json', 'Content-Type': 'application/json'}
resp = requests.post(url, headers=headers, data=x)
time.sleep(cooldown)
return resp.text
c = Checkpoint(ckpt_file='/my/custom/ckpt/name.ckpt')
c.insert('customers_data.json')
results = c.record(
process,
url='https://reqbin.com/echo/post/json',
cooldown=1,
checkpoint_every=5)
# 0%| | 0/11 [00:00<?, ?it/s]
# 8%|▊ | 1/11 [00:02<00:11, 1.10s/it]
# 17%|█▏ | 2/11 [00:03<00:25, 1.10s/it]
# 27%|██▋ | 3/11 [00:07<00:20, 1.10s/it]
# 35%|███▍ | 4/11 [00:09<00:18, 1.10s/it]
# Saving progress to checkpoint file: `/my/custom/ckpt/name.ckpt`...
# 38%|███▊ | 5/11 [00:11<00:17, 1.10s/it]
# 52%|████▏ | 6/11 [00:12<00:16, 1.10s/it]
# 69%|██████▉ | 7/11 [00:19<00:08, 1.10s/it]
# ...
- Let's say your program failed for some reason, you can easily resume like described in
Example 1
.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
resu-0.2.2.tar.gz
(6.8 kB
view hashes)