Streaming Column Printer
Project description
Scolp
Introduction
Scolp is Streaming Column Printer for Python 3.6 or later.
Scolp let you easily pretty-print masses of tabular data in a streaming fashion - each value is printed when available, without waiting for end of data. It is perfect for apps that need to print progress reports in columns.
Main features:
-
Auto-adjusting column width according to the largest value so far or column header width.
-
Control verbosity of printing by:
- printing
1
of eachn
rows - printing no more than
1
row pern
seconds
- printing
-
Control format of printed values by:
- setting global defaults
- setting defaults per variable type (
int
,float
,str
,datetime
) - setting explicit formatting per column
-
Control alignment of printed values:
- left
- right
- center
- auto: numbers to the right, strings or other types to the left.
-
Control cosmetics of columns (initial width, padding fill char, alignment, and more..) by:
- setting global defaults
- setting explicit formatting per column
-
Control column title printing style:
- Inline in each row
- As headers, repeating each n rows
-
Easily print row count or time since execution started without need to keep track of those values yourself.
Examples
Example 1
Lets start with a simple country statistics output using default settings:
import scolp
scolper = scolp.Scolp()
scolper.config.add_columns("country", "population (mil)", "capital city", "life expectancy (female)",
"life expectancy (male)", "fertility rate")
scolper.print("Netherlands", 16.81, "Amsterdam", 83, 79, 1.5,
"China", 1350.0, "Beijing", 76, 72, 1.8,
"Israel", 7.71, "Jerusalem", 84, 80, 2.7,
"Nigeria")
scolper.print(174.51)
Output:
(Note how column width is auto adjusting, line breaks are printed automatically after last column, and each value is printed immediately without waiting for end of row)
country |population (mil)|capital city|life expectancy (female)|life expectancy (male)|fertility rate
--------|----------------|------------|------------------------|----------------------|--------------
Netherlands| 16.810|Amsterdam | 83| 79| 1.500
country |population (mil)|capital city|life expectancy (female)|life expectancy (male)|fertility rate
-----------|----------------|------------|------------------------|----------------------|--------------
China | 1,350.000|Beijing | 76| 72| 1.800
Israel | 7.710|Jerusalem | 84| 80| 2.700
Nigeria | 174.510|
Example 2
Lets build a program that find prime numbers. We will print the count of primes we found so far and the last prime found.
import datetime, scolp
def is_prime(num):
return 2 in [num, 2 ** num % num]
scolper = scolp.Scolp()
scolper.config.add_columns("time", "elapsed", "inspected_count", "prime_count", "last", "progress %")
scolper.config.output_each_n_seconds = 1
prime_count = 0
last_prime = None
i = 9_999_800
target_count = 30
while prime_count < target_count:
if is_prime(i):
last_prime = i
prime_count += 1
progress = prime_count / target_count * 100
scolper.print(datetime.datetime.now(), scolper.elapsed_since_init(),
scolper.row_index + 1, prime_count, last_prime, progress)
i += 1
Output:
(Note how the header repeats, the column width auto-expanding and the numbers are aligned to the right)
time |elapsed |inspected_count|prime_count|last |progress %
--------|--------|---------------|-----------|--------|----------
2019-06-05 11:49:31.271191|0:00:00 | 1| 0|None | 0.000
time |elapsed |inspected_count|prime_count|last |progress %
--------------------------|--------|---------------|-----------|--------|----------
2019-06-05 11:49:32.306225|0:00:01 | 27| 1|9,999,823| 3.333
2019-06-05 11:49:33.325694|0:00:02 | 53| 1|9,999,823| 3.333
2019-06-05 11:49:34.341678|0:00:03 | 79| 3|9,999,877| 10.000
2019-06-05 11:49:35.378966|0:00:04 | 105| 6|9,999,901| 20.000
2019-06-05 11:49:36.399298|0:00:05 | 131| 8|9,999,929| 26.667
2019-06-05 11:49:37.415522|0:00:06 | 157| 11|9,999,943| 36.667
2019-06-05 11:49:38.450551|0:00:07 | 183| 13|9,999,973| 43.333
2019-06-05 11:49:39.478987|0:00:08 | 209| 14|9,999,991| 46.667
2019-06-05 11:49:40.485409|0:00:09 | 233| 15|10,000,019| 50.000
time |elapsed |inspected_count|prime_count|last |progress %
--------------------------|--------|---------------|-----------|----------|----------
2019-06-05 11:49:41.508298|0:00:10 | 259| 15|10,000,019| 50.000
2019-06-05 11:49:42.543115|0:00:11 | 283| 16|10,000,079| 53.333
2019-06-05 11:49:43.555733|0:00:12 | 306| 17|10,000,103| 56.667
2019-06-05 11:49:44.572379|0:00:13 | 328| 18|10,000,121| 60.000
2019-06-05 11:49:45.574066|0:00:14 | 349| 20|10,000,141| 66.667
2019-06-05 11:49:46.583462|0:00:15 | 372| 21|10,000,169| 70.000
2019-06-05 11:49:47.594724|0:00:16 | 396| 22|10,000,189| 73.333
2019-06-05 11:49:48.639124|0:00:17 | 420| 22|10,000,189| 73.333
2019-06-05 11:49:49.661211|0:00:18 | 441| 24|10,000,229| 80.000
2019-06-05 11:49:50.691037|0:00:19 | 463| 27|10,000,261| 90.000
time |elapsed |inspected_count|prime_count|last |progress %
--------------------------|--------|---------------|-----------|----------|----------
2019-06-05 11:49:51.721844|0:00:20 | 487| 28|10,000,271| 93.333
2019-06-05 11:49:52.733437|0:00:22 | 510| 29|10,000,303| 96.667
2019-06-05 11:49:53.750463|0:00:23 | 534| 29|10,000,303| 96.667
Example 3
Now, lets change the code of the previous example to add a bit of custom formatting:
scolper = scolp.Scolp()
scolper.config.add_column("time", width=20)
scolper.config.add_columns("elapsed",
"inspected_count",
"prime_count")
scolper.config.add_column("last", width=11)
scolper.config.add_column("progress", fmt="{:.1%}")
scolper.config.output_each_n_seconds = 1
scolper.config.header_repeat_row_count_first = 0
scolper.config.default_column.column_separator = " "
scolper.config.default_column.type_to_format[datetime.datetime] = "{:%Y-%m-%d %H:%M:%S}"
prime_count = 0
last_prime = None
i = 9_999_800
target_count = 30
while prime_count < target_count:
if is_prime(i):
last_prime = i
prime_count += 1
progress = prime_count / target_count
scolper.print(datetime.datetime.now(), scolper.elapsed_since_init(),
scolper.row_index + 1, prime_count, last_prime, progress)
i += 1
Output:
time elapsed inspected_count prime_count last progress
-------------------- -------- --------------- ----------- ----------- --------
2019-06-05 11:54:46 0:00:00 1 0 None 0.0%
2019-06-05 11:54:47 0:00:01 23 0 None 0.0%
2019-06-05 11:54:48 0:00:02 45 1 9,999,823 3.3%
2019-06-05 11:54:49 0:00:03 67 2 9,999,863 6.7%
2019-06-05 11:54:50 0:00:04 90 5 9,999,889 16.7%
2019-06-05 11:54:51 0:00:05 115 7 9,999,907 23.3%
2019-06-05 11:54:52 0:00:06 139 10 9,999,937 33.3%
2019-06-05 11:54:53 0:00:07 164 11 9,999,943 36.7%
2019-06-05 11:54:54 0:00:08 188 13 9,999,973 43.3%
2019-06-05 11:54:55 0:00:09 212 14 9,999,991 46.7%
time elapsed inspected_count prime_count last progress
-------------------- -------- --------------- ----------- ----------- --------
2019-06-05 11:54:56 0:00:10 237 15 10,000,019 50.0%
2019-06-05 11:54:57 0:00:11 261 15 10,000,019 50.0%
2019-06-05 11:54:58 0:00:12 284 16 10,000,079 53.3%
2019-06-05 11:54:59 0:00:13 308 17 10,000,103 56.7%
2019-06-05 11:55:00 0:00:14 331 18 10,000,121 60.0%
2019-06-05 11:55:01 0:00:15 355 20 10,000,141 66.7%
2019-06-05 11:55:02 0:00:16 379 21 10,000,169 70.0%
2019-06-05 11:55:03 0:00:17 403 22 10,000,189 73.3%
2019-06-05 11:55:04 0:00:18 426 23 10,000,223 76.7%
2019-06-05 11:55:05 0:00:20 448 25 10,000,247 83.3%
time elapsed inspected_count prime_count last progress
-------------------- -------- --------------- ----------- ----------- --------
2019-06-05 11:55:06 0:00:21 471 27 10,000,261 90.0%
2019-06-05 11:55:07 0:00:22 493 28 10,000,271 93.3%
2019-06-05 11:55:08 0:00:23 516 29 10,000,303 96.7%
2019-06-05 11:55:09 0:00:24 539 29 10,000,303 96.7%
Example 4
Lets build an HTTP big-file downloader.
import datetime, urllib3, scolp
url = "http://speedtest.tele2.net/100MB.zip"
path = "downloaded.tmp"
chunk_size_bytes = 1000
scolp_cfg = scolp.Config()
scolp_cfg.add_column("time", fmt="{:%H:%M:%S}")
scolp_cfg.add_column("elapsed")
scolp_cfg.add_column("downloaded", width=16, fmt="{:,} B")
scolp_cfg.add_column("speed", width=14, pad_align=scolp.Alignment.RIGHT, type_to_format={float: "{:,.1f} kB/s"})
scolp_cfg.output_each_n_seconds = 1
scolp_cfg.title_mode = scolp.TitleMode.INLINE
scolp_cfg.default_column.column_separator = " | "
scolper = scolp.Scolp(scolp_cfg)
http = urllib3.PoolManager()
r = http.request('GET', url, preload_content=False)
dl_bytes = 0
def progress_update():
elapsed_sec = scolper.elapsed_since_init().total_seconds()
speed_kbps = dl_bytes / elapsed_sec / 1000 if elapsed_sec > 0 else "unknown"
scolper.print(datetime.datetime.now(), scolper.elapsed_since_init(), dl_bytes, speed_kbps)
with open(path, 'wb') as out:
while True:
data = r.read(chunk_size_bytes)
if not data:
break
out.write(data)
dl_bytes += len(data)
progress_update()
scolper.force_print_next_row()
progress_update()
r.release_conn()
Output:
time=14:30:11 | elapsed=0:00:00 | downloaded= 1,000 B | speed= unknown
time=14:30:12 | elapsed=0:00:01 | downloaded= 801,000 B | speed= 801.0 kB/s
time=14:30:13 | elapsed=0:00:02 | downloaded= 1,743,000 B | speed= 871.5 kB/s
time=14:30:14 | elapsed=0:00:03 | downloaded= 2,758,000 B | speed= 919.3 kB/s
time=14:30:15 | elapsed=0:00:04 | downloaded= 3,779,000 B | speed= 944.8 kB/s
time=14:30:16 | elapsed=0:00:05 | downloaded= 4,794,000 B | speed= 958.8 kB/s
time=14:30:17 | elapsed=0:00:06 | downloaded= 5,809,000 B | speed= 968.2 kB/s
time=14:30:18 | elapsed=0:00:07 | downloaded= 6,824,000 B | speed= 974.9 kB/s
time=14:30:19 | elapsed=0:00:08 | downloaded= 7,839,000 B | speed= 979.9 kB/s
time=14:30:20 | elapsed=0:00:09 | downloaded= 8,857,000 B | speed= 984.1 kB/s
time=14:30:21 | elapsed=0:00:10 | downloaded= 9,799,000 B | speed= 979.9 kB/s
time=14:30:22 | elapsed=0:00:11 | downloaded= 10,814,000 B | speed= 983.1 kB/s
time=14:30:23 | elapsed=0:00:12 | downloaded= 11,838,000 B | speed= 986.5 kB/s
time=14:30:24 | elapsed=0:00:13 | downloaded= 12,855,000 B | speed= 988.8 kB/s
time=14:30:25 | elapsed=0:00:14 | downloaded= 13,870,000 B | speed= 990.7 kB/s
time=14:30:26 | elapsed=0:00:15 | downloaded= 14,891,000 B | speed= 992.7 kB/s
time=14:30:27 | elapsed=0:00:16 | downloaded= 15,906,000 B | speed= 994.1 kB/s
time=14:30:28 | elapsed=0:00:18 | downloaded= 25,600,000 B | speed= 1,422.2 kB/s
time=14:30:29 | elapsed=0:00:19 | downloaded= 37,146,000 B | speed= 1,955.1 kB/s
time=14:30:30 | elapsed=0:00:20 | downloaded= 47,847,000 B | speed= 2,392.3 kB/s
time=14:30:31 | elapsed=0:00:21 | downloaded= 60,962,000 B | speed= 2,903.0 kB/s
time=14:30:32 | elapsed=0:00:22 | downloaded= 72,931,000 B | speed= 3,315.0 kB/s
time=14:30:33 | elapsed=0:00:23 | downloaded= 85,094,000 B | speed= 3,699.7 kB/s
time=14:30:34 | elapsed=0:00:24 | downloaded= 104,857,600 B | speed= 4,369.1 kB/s
Requirements
Scolp has no 3rd party requirements other than Python 3.6 or later.
Getting Started
Scolp is available via PyPi and can be installed using:
pip install scolp
.
Todo
- Document public API of the library
- Support colors
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.