Streaming Column Printer
Project description
Scolp
Introduction
Scolp is Streaming Column Printer for Python 3.6 or later.
Scolp let you easily pretty-print masses of tabular data in a streaming fashion - each value is printed when available, without waiting for end of data. It is perfect for apps that need to print progress reports in columns.
Main features:
-
Auto-adjusting column width according to the largest value so far or column header width.
-
Control verbosity of printing by:
- printing
1
of eachn
rows - printing no more than
1
row pern
seconds
- printing
-
Control format of printed values by:
- setting global defaults
- setting defaults per variable type (
int
,float
,str
,datetime
) - setting explicit formatting per column
-
Control alignment of printed values:
- left
- right
- center
- auto: numbers to the right, strings or other types to the left.
-
Control cosmetics of columns (initial width, padding fill char, alignment, and more..) by:
- setting global defaults
- setting explicit formatting per column
-
Control column title printing style:
- Inline in each row
- As headers, repeating each n rows
-
Easily print row count or time since execution started without need to keep track of those values yourself.
Examples
Example 1
Lets start with a simple country statistics output using default settings:
import scolp
scolp_cfg = scolp.Config()
scolp_cfg.add_columns("country", "population (mil)", "capital city", "life expectancy (female)",
"life expectancy (male)", "fertility rate")
scolper = scolp.Scolp(scolp_cfg)
scolper.print("Netherlands", 16.81, "Amsterdam", 83, 79, 1.5,
"China", 1350.0, "Beijing", 76, 72, 1.8,
"Israel", 7.71, "Jerusalem", 84, 80, 2.7,
"Nigeria")
scolper.print(174.51)
Output:
(Note how column width is auto adjusting, line breaks are printed automatically after last column, and each value is printed immediately without waiting for end of row)
country |population (mil)|capital city|life expectancy (female)|life expectancy (male)|fertility rate
--------|----------------|------------|------------------------|----------------------|--------------
Netherlands| 16.810|Amsterdam | 83| 79| 1.500
country |population (mil)|capital city|life expectancy (female)|life expectancy (male)|fertility rate
-----------|----------------|------------|------------------------|----------------------|--------------
China | 1,350.000|Beijing | 76| 72| 1.800
Israel | 7.710|Jerusalem | 84| 80| 2.700
Nigeria | 174.510|
Example 2
Lets build a program that find prime numbers. We will print the count of primes we found so far and the last prime found.
import datetime, scolp
def is_prime(num):
return 2 in [num, 2 ** num % num]
# define columns
scolp_cfg = scolp.Config()
scolp_cfg.add_column("time")
scolp_cfg.add_column("elapsed")
scolp_cfg.add_column("inspected_count")
scolp_cfg.add_column("prime_count")
scolp_cfg.add_column("last")
scolp_cfg.add_column("progress %")
scolp_cfg.output_each_n_seconds = 1
scolper = scolp.Scolp(scolp_cfg)
prime_count = 0
last_prime = None
i = 9_999_800
target_count = 30
while prime_count < target_count:
if is_prime(i):
last_prime = i
prime_count += 1
progress = prime_count / target_count * 100
scolper.print(datetime.datetime.now(), scolper.elapsed_since_init(),
scolper.row_index + 1, prime_count, last_prime, progress)
i += 1
Output:
(Note how the header repeats, the column width auto-expanding and the numbers are aligned to the right)
time |elapsed |inspected_count|prime_count|last |progress %
--------|--------|---------------|-----------|--------|----------
2019-06-05 11:49:31.271191|0:00:00 | 1| 0|None | 0.000
time |elapsed |inspected_count|prime_count|last |progress %
--------------------------|--------|---------------|-----------|--------|----------
2019-06-05 11:49:32.306225|0:00:01 | 27| 1|9,999,823| 3.333
2019-06-05 11:49:33.325694|0:00:02 | 53| 1|9,999,823| 3.333
2019-06-05 11:49:34.341678|0:00:03 | 79| 3|9,999,877| 10.000
2019-06-05 11:49:35.378966|0:00:04 | 105| 6|9,999,901| 20.000
2019-06-05 11:49:36.399298|0:00:05 | 131| 8|9,999,929| 26.667
2019-06-05 11:49:37.415522|0:00:06 | 157| 11|9,999,943| 36.667
2019-06-05 11:49:38.450551|0:00:07 | 183| 13|9,999,973| 43.333
2019-06-05 11:49:39.478987|0:00:08 | 209| 14|9,999,991| 46.667
2019-06-05 11:49:40.485409|0:00:09 | 233| 15|10,000,019| 50.000
time |elapsed |inspected_count|prime_count|last |progress %
--------------------------|--------|---------------|-----------|----------|----------
2019-06-05 11:49:41.508298|0:00:10 | 259| 15|10,000,019| 50.000
2019-06-05 11:49:42.543115|0:00:11 | 283| 16|10,000,079| 53.333
2019-06-05 11:49:43.555733|0:00:12 | 306| 17|10,000,103| 56.667
2019-06-05 11:49:44.572379|0:00:13 | 328| 18|10,000,121| 60.000
2019-06-05 11:49:45.574066|0:00:14 | 349| 20|10,000,141| 66.667
2019-06-05 11:49:46.583462|0:00:15 | 372| 21|10,000,169| 70.000
2019-06-05 11:49:47.594724|0:00:16 | 396| 22|10,000,189| 73.333
2019-06-05 11:49:48.639124|0:00:17 | 420| 22|10,000,189| 73.333
2019-06-05 11:49:49.661211|0:00:18 | 441| 24|10,000,229| 80.000
2019-06-05 11:49:50.691037|0:00:19 | 463| 27|10,000,261| 90.000
time |elapsed |inspected_count|prime_count|last |progress %
--------------------------|--------|---------------|-----------|----------|----------
2019-06-05 11:49:51.721844|0:00:20 | 487| 28|10,000,271| 93.333
2019-06-05 11:49:52.733437|0:00:22 | 510| 29|10,000,303| 96.667
2019-06-05 11:49:53.750463|0:00:23 | 534| 29|10,000,303| 96.667
Example 3
Now, lets change the code of the previous example to add a bit of custom formatting:
scolp_cfg = scolp.Config()
scolp_cfg.add_column("time", width=20)
scolp_cfg.add_column("elapsed")
scolp_cfg.add_column("inspected_count")
scolp_cfg.add_column("prime_count")
scolp_cfg.add_column("last", width=11)
scolp_cfg.add_column("progress", fmt="{:.1%}")
scolp_cfg.output_each_n_seconds = 1
scolp_cfg.header_repeat_row_count_first = 0
scolp_cfg.default_column.column_separator = " "
scolp_cfg.default_column.type_to_format[datetime.datetime] = "{:%Y-%m-%d %H:%M:%S}"
scolper = scolp.Scolp(scolp_cfg)
prime_count = 0
last_prime = None
i = 9_999_800
target_count = 30
while prime_count < target_count:
if is_prime(i):
last_prime = i
prime_count += 1
progress = prime_count / target_count
scolper.print(datetime.datetime.now(), scolper.elapsed_since_init(),
scolper.row_index + 1, prime_count, last_prime, progress)
i += 1
Output:
time elapsed inspected_count prime_count last progress
-------------------- -------- --------------- ----------- ----------- --------
2019-06-05 11:54:46 0:00:00 1 0 None 0.0%
2019-06-05 11:54:47 0:00:01 23 0 None 0.0%
2019-06-05 11:54:48 0:00:02 45 1 9,999,823 3.3%
2019-06-05 11:54:49 0:00:03 67 2 9,999,863 6.7%
2019-06-05 11:54:50 0:00:04 90 5 9,999,889 16.7%
2019-06-05 11:54:51 0:00:05 115 7 9,999,907 23.3%
2019-06-05 11:54:52 0:00:06 139 10 9,999,937 33.3%
2019-06-05 11:54:53 0:00:07 164 11 9,999,943 36.7%
2019-06-05 11:54:54 0:00:08 188 13 9,999,973 43.3%
2019-06-05 11:54:55 0:00:09 212 14 9,999,991 46.7%
time elapsed inspected_count prime_count last progress
-------------------- -------- --------------- ----------- ----------- --------
2019-06-05 11:54:56 0:00:10 237 15 10,000,019 50.0%
2019-06-05 11:54:57 0:00:11 261 15 10,000,019 50.0%
2019-06-05 11:54:58 0:00:12 284 16 10,000,079 53.3%
2019-06-05 11:54:59 0:00:13 308 17 10,000,103 56.7%
2019-06-05 11:55:00 0:00:14 331 18 10,000,121 60.0%
2019-06-05 11:55:01 0:00:15 355 20 10,000,141 66.7%
2019-06-05 11:55:02 0:00:16 379 21 10,000,169 70.0%
2019-06-05 11:55:03 0:00:17 403 22 10,000,189 73.3%
2019-06-05 11:55:04 0:00:18 426 23 10,000,223 76.7%
2019-06-05 11:55:05 0:00:20 448 25 10,000,247 83.3%
time elapsed inspected_count prime_count last progress
-------------------- -------- --------------- ----------- ----------- --------
2019-06-05 11:55:06 0:00:21 471 27 10,000,261 90.0%
2019-06-05 11:55:07 0:00:22 493 28 10,000,271 93.3%
2019-06-05 11:55:08 0:00:23 516 29 10,000,303 96.7%
2019-06-05 11:55:09 0:00:24 539 29 10,000,303 96.7%
Example 4
Lets build an HTTP big-file downloader.
import datetime, urllib3, scolp
url = "http://speedtest.tele2.net/100MB.zip"
path = "downloaded.tmp"
chunk_size_bytes = 1000
scolp_cfg = scolp.Config()
scolp_cfg.add_column("time", fmt="{:%H:%M:%S}")
scolp_cfg.add_column("elapsed")
scolp_cfg.add_column("downloaded", width=16, fmt="{:,} B")
scolp_cfg.add_column("speed", width=14, pad_align=scolp.Alignment.RIGHT, type_to_format={float: "{:,.1f} kB/s"})
scolp_cfg.output_each_n_seconds = 1
scolp_cfg.title_mode = scolp.TitleMode.INLINE
scolp_cfg.default_column.column_separator = " | "
scolper = scolp.Scolp(scolp_cfg)
http = urllib3.PoolManager()
r = http.request('GET', url, preload_content=False)
dl_bytes = 0
def progress_update():
elapsed_sec = scolper.elapsed_since_init().total_seconds()
speed_kbps = dl_bytes / elapsed_sec / 1000 if elapsed_sec > 0 else "unknown"
scolper.print(datetime.datetime.now(), scolper.elapsed_since_init(), dl_bytes, speed_kbps)
with open(path, 'wb') as out:
while True:
data = r.read(chunk_size_bytes)
if not data:
break
out.write(data)
dl_bytes += len(data)
progress_update()
scolper.force_print_next_row()
progress_update()
r.release_conn()
Output:
time=14:30:11 | elapsed=0:00:00 | downloaded= 1,000 B | speed= unknown
time=14:30:12 | elapsed=0:00:01 | downloaded= 801,000 B | speed= 801.0 kB/s
time=14:30:13 | elapsed=0:00:02 | downloaded= 1,743,000 B | speed= 871.5 kB/s
time=14:30:14 | elapsed=0:00:03 | downloaded= 2,758,000 B | speed= 919.3 kB/s
time=14:30:15 | elapsed=0:00:04 | downloaded= 3,779,000 B | speed= 944.8 kB/s
time=14:30:16 | elapsed=0:00:05 | downloaded= 4,794,000 B | speed= 958.8 kB/s
time=14:30:17 | elapsed=0:00:06 | downloaded= 5,809,000 B | speed= 968.2 kB/s
time=14:30:18 | elapsed=0:00:07 | downloaded= 6,824,000 B | speed= 974.9 kB/s
time=14:30:19 | elapsed=0:00:08 | downloaded= 7,839,000 B | speed= 979.9 kB/s
time=14:30:20 | elapsed=0:00:09 | downloaded= 8,857,000 B | speed= 984.1 kB/s
time=14:30:21 | elapsed=0:00:10 | downloaded= 9,799,000 B | speed= 979.9 kB/s
time=14:30:22 | elapsed=0:00:11 | downloaded= 10,814,000 B | speed= 983.1 kB/s
time=14:30:23 | elapsed=0:00:12 | downloaded= 11,838,000 B | speed= 986.5 kB/s
time=14:30:24 | elapsed=0:00:13 | downloaded= 12,855,000 B | speed= 988.8 kB/s
time=14:30:25 | elapsed=0:00:14 | downloaded= 13,870,000 B | speed= 990.7 kB/s
time=14:30:26 | elapsed=0:00:15 | downloaded= 14,891,000 B | speed= 992.7 kB/s
time=14:30:27 | elapsed=0:00:16 | downloaded= 15,906,000 B | speed= 994.1 kB/s
time=14:30:28 | elapsed=0:00:18 | downloaded= 25,600,000 B | speed= 1,422.2 kB/s
time=14:30:29 | elapsed=0:00:19 | downloaded= 37,146,000 B | speed= 1,955.1 kB/s
time=14:30:30 | elapsed=0:00:20 | downloaded= 47,847,000 B | speed= 2,392.3 kB/s
time=14:30:31 | elapsed=0:00:21 | downloaded= 60,962,000 B | speed= 2,903.0 kB/s
time=14:30:32 | elapsed=0:00:22 | downloaded= 72,931,000 B | speed= 3,315.0 kB/s
time=14:30:33 | elapsed=0:00:23 | downloaded= 85,094,000 B | speed= 3,699.7 kB/s
time=14:30:34 | elapsed=0:00:24 | downloaded= 104,857,600 B | speed= 4,369.1 kB/s
Requirements
Scolp has no 3rd party requirements other than Python 3.6 or later.
Getting Started
Scolp is available via PyPi and can be installed using:
pip install scolp
.
Todo
- Document public API of the library
- Support colors
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.