Skip to main content

adding tiny resume function, for csv reading iterator

Project description

cssresumable

Adding tiny resume function for your csv reading iterator.

examples

00main.py

import sys
import os
import random
from io import StringIO
from csvresumable import DictReader

data = """\
id,name
1,foo
2,bar
3,boo
"""

random.seed(int(os.environ.get("SEED", "0")))
for row in DictReader(StringIO(data)):
    n = random.random()
    if n < 0.5:
        print("fail {}".format(n), file=sys.stderr)
        sys.exit(0)
    print("\t", row["name"])
print("finished")

resuming until success.

SEED=0 python 00main.py
fail 0.420571580830845
	 foo
	 bar
SEED=1 RESUME=1 python 00main.py
fail 0.13436424411240122
SEED=2 RESUME=1 python 00main.py
	 boo
finished
SEED=5 python 00main.py
	 foo
	 bar
	 boo
finished

or setting with resume option (run this script, such as python 01main.py --resume)

--- 00main.py	2018-06-16 04:19:51.573578443 +0900
+++ 01main.py	2018-06-16 04:20:04.760554103 +0900
@@ -4,17 +4,23 @@
 from io import StringIO
 from csvresumable import DictReader

-data = """\
+data = """
 id,name
 1,foo
 2,bar
 3,boo
 """

+import argparse
+parser = argparse.ArgumentParser()
+parser.add_argument("--seed", default=0, type=int)
+parser.add_argument("--resume", action="store_true")
+args = parser.parse_args()
+
 random.seed(int(os.environ.get("SEED", "0")))
-for row in DictReader(StringIO(data)):
+for row in DictReader(StringIO(data), resume=args.resume):
     n = random.random()
-    if n < 0.5:
+    if n > 0.8:
         print("fail {}".format(n), file=sys.stderr)
         sys.exit(0)
     print("\t", row["name"])

with multi files

from csvresumable import DictReader

files = ["a.csv", "b.csv"]
for f in files:
    r = DictReader(f)
    for row in r:
        do_something(row)

more

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for csvresumable, version 0.0.3.post2
Filename, size File type Python version Upload date Hashes
Filename, size csvresumable-0.0.3.post2-py2.py3-none-any.whl (6.4 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size csvresumable-0.0.3.post2.tar.gz (4.8 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page