Skip to main content

Small tool to track time of Python code blocks.

Project description

Coverage Maintenance PyPI version PyPI pyversions PyPI status

about-time

Small tool to track time of Python code blocks.

What does it do?

There are several times where we need to instrument and log code execution, to see where complex pipelines are spending the most time in.

A simple start = time.time() and end = time.time() - start does not cut it when we want to track several lines at the same time, and/or whole blocks with line granularity.

This tool measures the execution time of a block of code, and can now even count iterations and the throughput of them, always with a beautiful "human" representation.

How to use it?

There's three modes of operation: context manager, callable handler and iterator metrics.

1. Use it like a context manager:

from about_time import about_time

with about_time() as t_whole:
    with about_time() as t_1:
        func_1()
    with about_time() as t_2:
        func_2('params')

Then, get the timings like this:

print(f'func_1 time: {t_1.duration_human}')
print(f'func_2 time: {t_2.duration_human}')
print(f'total time: {t_whole.duration_human}')

There's also the duration property, which returns the actual float time in seconds.

secs = t_whole.duration

2. You can also use it like a callable handler:

t_1 = about_time(func_1)
t_2 = about_time(lambda: func_2('params'))

If you use the callable handler syntax, there will be a new field called result to get the outcome of the function!

results = t_1.result, t_2.result

Or you can mix and match both:

with about_time() as t_whole:
    t_1 = about_time(func_1)
    t_2 = about_time(lambda: func_2('params'))

3. And you can count and measure throughput:

Wrap your iterable and iterate it! Since it have duration, it can calculate the throughput of the whole block. Specially useful in generators, which do not have length, but you can use with any iterables:

def callback(t_func):
    logger.info('func: size=%d throughput=%s', t_func.count,
                                               t_func.throughput_human)
items = filter(...)
for item in about_time(callback, items):
    # use item any way you want.
    pass

How do I install it?

Just do in python env:

$ pip install about-time

Some nice features

Humans are first class citizens :)

I've considered two key concepts in designing the human friendly functions: 3.44s is more meaningful than 3.43584783784s, and 14.12us is much nicer than .0000141233333s. So saying it another way, I round values to two decimal places at most, and finds the smaller unit to represent it, minimizing values smaller than 1.

Note that it dynamically finds the best unit to represent the value, considering even the rounding been applied. So if a value is for example 0.999999, it would end up like 1000.0ms after rounded, but it is auto-upgraded to the next unit 1.0s.

The duration_human changes seamlessly from nanoseconds to hours. Values smaller than 60 seconds are rendered with two digits precision at most (zeros to the right of the decimal point are not shown), and starting from 1 minute, an "hours:minutes:seconds.M" milliseconds (with only one digit precision). Some examples directly from the unit tests:

duration (float seconds) duration_human
.00000000185 '1.85ns'
.000000999996 '1.0us'
.00001 '10.0us'
.0000156 '15.6us'
.01 '10.0ms'
.0141233333333 '14.12ms'
.1099999 '110.0ms'
.1599999 '160.0ms'
.8015 '801.5ms'
3.434999 '3.43s'
59.999 '0:01:00'
68.5 '0:01:08.5'
125.825 '0:02:05.8'
4488.395 '1:14:48.4'

The throughput_human has similar logic, and to human brain it is much trickier to figure out: If it took 1165263 seconds to handle 123 items, how fast it did? Even dividing to find out the time per item 9473 seconds don't mean much. Dividing by 3600 we get 2.63 hours per item, and the throughput is returned nicely as 0.38/h. The tool has per-second, per-minute and per-hour. Some examples:

duration (float seconds) number of elements throughput_human
1. 10 '10.0/s'
1. 2500 '2500.0/s'
2. 1 '30.0/m'
2. 10 '5.0/s'
1.981981981981982 11 '5.55/s'
100. 10 '6.0/m'
1600. 3 '6.75/h'
.99 1 '1.01/s'
1165263. 123 '0.38/h'

Accuracy

This tool supports all versions of python, but in pythons >= 3.3, the code uses the new time.perf_counter to gain from the higher resolution and smaller propagating of errors. In older versions, it uses time.time.

License

This software is licensed under the MIT License. See the LICENSE file in the top distribution directory for the full license text.

Nice huh?

Thanks for your interest!

I wish you have fun using this tool! :)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

about-time-2.0.1.tar.gz (5.3 kB view details)

Uploaded Source

Built Distribution

about_time-2.0.1-py3-none-any.whl (5.5 kB view details)

Uploaded Python 3

File details

Details for the file about-time-2.0.1.tar.gz.

File metadata

  • Download URL: about-time-2.0.1.tar.gz
  • Upload date:
  • Size: 5.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.1.0 requests-toolbelt/0.8.0 tqdm/4.25.0 CPython/3.7.0

File hashes

Hashes for about-time-2.0.1.tar.gz
Algorithm Hash digest
SHA256 08818a4d47e7ffc5a0b1a82f48638a8eafc2c7898df26defaeeac41b61adc497
MD5 f6d77e5ac21e5d3d9f413c3764159645
BLAKE2b-256 0ca52bb512eb30aab119b9181ca6d8d44a9ea01eab9029092801ad815e57591c

See more details on using hashes here.

File details

Details for the file about_time-2.0.1-py3-none-any.whl.

File metadata

  • Download URL: about_time-2.0.1-py3-none-any.whl
  • Upload date:
  • Size: 5.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.1.0 requests-toolbelt/0.8.0 tqdm/4.25.0 CPython/3.7.0

File hashes

Hashes for about_time-2.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 63e0337630950d4f0fd46b5eae72c310f70363ed516ce1824b3e55ca80d47b0d
MD5 728272e1aab3f111fb6900d752cfacbb
BLAKE2b-256 a5f0454cfffc872564c81f385ee4959f7cf6b867e4a25a460cb4ef52a3c7d3cd

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page