Sized Generators with Decorators
Project description
sized
Sized Generators with Decorators
Why?
- The
SizedGenerator
type provides a simple and robust way to keep track of iterations and max sizes for generators and iterators. - An issue with using normal Generators with long-running / multithread processes has been reporting progress.
- Here is an example of a standard
Generator
being wrapped withtqdm
:
from tqdm import tqdm
def gen():
n = 2
for _ in range(100):
n += (5 * 1 / n)
yield n
for x in tqdm(gen()):
pass
# Output has no progress bar:
> 100it [00:00, 1.00it/s]
- A solution would be to keep track of the total progress, but this gets messy if an iteration is interrupted by user actions and the progress bar needs to continue.
- Now with the
sized
decorator:
from sized import sized
@sized(100)
def s_gen():
n = 2
for _ in range(100):
n += (5 * 1 / n)
yield n
for x in tqdm(s_gen()):
pass
# Now with a progress bar!
> 100%|██████████| 100/100 [00:00<00:00, 1.00it/s]
SizedGenerator
will also track iterations called and reflect remaining length
gen_instance = s_gen()
len(gen_instance) -> 100
next(gen_instance)
next(gen_instance)
len(gen_instance) -> 98
Getting started
There are 2 main ways to create a SizedGenerator
1. @sized
decorator
from sized import sized
@sized(5)
def gen():
yield ...
2. SizedGenerator
constructor
from sized import SizedGenerator
gen = SizedGenerator((x ** 2 for x in range(10)), 10)
Additional Info
- The
size
argument can be either anint
,Sized
object, or a callable accepting a dictionary of arguments and keyword-arguments, returning anint
orSized
object.
@sized(15) # int
def gen():
for i in range(15):
yield i ** 2
ls = [1, 4, 5]
@sized(ls) # `Sized` objects will have len(obj) called automatically
def gen():
for i in ls:
yield i ** 2
@sized(lambda x: x['some_arg']) # Callable using keyword argument, returning `Sized`
def gen(arg = None):
for i in arg:
yield i ** 2
@sized(lambda x: 2 * len(x['1'])) # Callable using positional argument, returning `int`
def gen(arg_0, arg_1):
for _ in range(2):
for i in arg_1:
yield i ** 2 + arg_0
License
The code in this project is released under the MIT License.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
sized-0.1.0.tar.gz
(5.2 kB
view hashes)
Built Distribution
sized-0.1.0-py3-none-any.whl
(4.9 kB
view hashes)