Skip to main content

Nesting generators using greenlets

Project description

Greengen

Greengen is a python package that allows implementing generators without yielding items all the way up. Instead, any inner function called from a greengen generator, can yield items directly to the outer generator.

Installation

pip install greengen

Usage

For example, suppose we want to implement a function that performs some heavy business logic, and generates logs. We want to handle the logs the moment they are created. But our function has to call a deep stack of helper functions, which in their turn call more functions, and logs may be written anywhere inside any function. The implementation with regular generators will be utterly annoying:

import time


def do_business_logic_and_yield_logs(some_input):
    yield from _helper_function(some_input)
    yield from _more_helper_function(some_input)


def _helper_function(some_input):
    for i in range(some_input):
        logs_and_result = _inner_helper_function(i)
        # Notice the enormous effort needed in order to retrieve the last result (using StopIteration etc.)
        while True:
            try:
                yield next(logs_and_result)  # This is a log
            except StopIteration as e:
                result = e.value  # This is the result
                yield log('Result for {} is {}'.format(i, result))
                break


def _more_helper_function(some_input):
    yield from _helper_function(some_input * 100)
    yield from _helper_function(some_input ** 2)


def _inner_helper_function(some_input):
    yield log('Started calculating')
    result = 2 ** some_input  # (Or whatever other heavy calculation)
    yield log('Finished calculating')
    return result  # Will be raised as StopIteration


def log(stuff):
    return {'message': str(stuff), 'timestamp': time.time()}


def main():
    for l in do_business_logic_and_yield_logs(42):
        # Consume the logs however we want
        print('{}: {}'.format(l['timestamp'], l['message']))

Using Greengen, this example can be simplified into the following:

import time
import greengen

@greengen.greengen
def do_business_logic_and_yield_logs(some_input):
    # Notice how we don't need the "yield from" anymore
    _helper_function(some_input)
    _more_helper_function(some_input)


def _helper_function(some_input):
    for i in range(some_input):
        # Notice how easy it is to retrieve the result now
        result = _inner_helper_function(i)
        log('Result for {} is {}'.format(i, result))


def _more_helper_function(some_input):
    _helper_function(some_input * 100)
    _helper_function(some_input ** 2)


def _inner_helper_function(some_input):
    log('Started calculating')
    result = 2 ** some_input  # (Or whatever other heavy calculation)
    log('Finished calculating')
    return result


def log(stuff):
    # This is the only place in the entire code where we need to be aware of the fact that we are inside a generator.
    # This will directly yield the log as the next item in the outer generator ("do_business_logic_and_yield_logs")
    greengen.yield_({'message': str(stuff), 'timestamp': time.time()})


def main():
    for l in do_business_logic_and_yield_logs(42):
        # Consume the logs however we want
        print('{}: {}'.format(l['timestamp'], l['message']))

Contributing

  1. Fork it!
  2. Create your feature branch: git checkout -b my-new-feature
  3. Commit your changes: git commit -am 'Add some feature'
  4. Push to the branch: git push origin my-new-feature
  5. Submit a pull request :D

History

TODO: Write history

Credits

TODO: Write credits

License

TODO: Write license

Project details


Release history Release notifications

This version

1.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for greengen, version 1.0.1
Filename, size File type Python version Upload date Hashes
Filename, size greengen-1.0.1-py2-none-any.whl (3.8 kB) File type Wheel Python version py2 Upload date Hashes View hashes
Filename, size greengen-1.0.1-py3-none-any.whl (3.8 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size greengen-1.0.1.tar.gz (3.2 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page