Skip to main content

Updated polling utility with many configurable options

Project description

Build Status PyPI PyPI HitCount

polling2

Never write another polling function again!

Polling2 is a powerful python utility used to wait for a function to return a certain expected condition.

Some possible uses cases include:

  • Wait for API response to return with code 200
  • Wait for a file to exist (or not exist)
  • Wait for a thread lock on a resource to expire

Polling2 is handy for getting rid of all that duplicated polling-code. Often, applications require retrying until the correct response is returned. Why re-implement the ability to poll again and again? Use Polling2!

Polling2 is a fork of the original polling. It was forked when the original maintainer failed to respond to issues or PRs.

Polling2 is ++under active development++. Would you like to see a particular feature? Ask and thou shall recieve.

Installation

pip install polling2

Development installation

# install lib, but use system links from the repo into sitepackages.
python setup.py develop
# install test dependenices.
python setup.py test
# run the tests
pytest tests

Examples

Example: Poll every minute until a url returns 200 status code

import requests
polling2.poll(
    lambda: requests.get('http://google.com').status_code == 200,
    step=60,
    poll_forever=True)

If you are creating a new cloud provider instance (e.g. waiting for an EC2 instance to come online), you can continue to poll despite getting ConnectionErrors:

import requests
polling2.poll(
    lambda: requests.get('your.instance.ip').status_code == 200,
    step=60,
    ignore_exceptions=(requests.exceptions.ConnectionError,),
    poll_forever=True)

Example: Poll for a file to exist

# This call will wait until the file exists, checking every 0.1 seconds and stopping after 3 seconds have elapsed
file_handle = polling2.poll(
    lambda: open('/tmp/myfile.txt'),
    ignore_exceptions=(IOError,),
    timeout=3,
    step=0.1)

# Polling will return the value of your polling function, so you can now interact with it
file_handle.close()

Example: Polling for Selenium WebDriver elements

from selenium import webdriver
driver = webdriver.Firefox()

driver.get('http://google.com')
search_box = polling2.poll(
    lambda: driver.find_element_by_id('search'),
    step=0.5,
    timeout=7)

search_box.send_keys('python polling')

Example: Using the polling timeout exception

# An exception will be raised by the polling function on timeout (or the maximum number of calls is exceeded).
# This exception will have a 'values' attribute. This is a queue with all values that did not meet the condition.
# You can access them in the except block.

import random
try:
    polling2.poll(lambda: random.choice([0, (), False]), step=0.5, timeout=1)
except polling2.TimeoutException, te:
    while not te.values.empty():
        # Print all of the values that did not meet the exception
        print te.values.get()

Example: Using a custom checker

# is_truthy() is the default checker for the parameter check_success. But, it's easy to create a custom
# checker function, that tests whether the value returned by the target is the expected value.

# Here the target is going to return None, which the custom checker, created by is_value(None)
# will return True for.
polling2.poll(target=lambda: None, step=0.1, max_tries=1, check_success=polling2.is_value(None))
# Or another example, where we can test that False is returned by the target.
polling2.poll(target=lambda: False, step=0.1, max_tries=1, check_success=polling2.is_value(False))

Example: Using a custom condition callback function

import requests

def is_correct_response(response):
    """Check that the response returned 'success'"""
    return response == 'success'

polling2.poll(
    lambda: requests.put('http://mysite.com/api/user', data={'username': 'Jill'},
    check_success=is_correct_response,
    step=1,
    timeout=10)

Example: Logging the return values from the target function.

import logging
import requests

def is_correct_response(response):
    """Check that the response returned 'success'"""
    return response == 'success'

polling2.poll(
    lambda: requests.put('http://mysite.com/api/user', data={'username': 'Jill'},
    check_success=is_correct_response,
    step=1,
    timeout=10,
    log=logging.DEBUG)

This will log the string representation of response object to python's logging module at the debug level. A message like this will be sent to the log for each return value. You can change the level by providing a different value to the log parameter.

poll() calls check_success(<Response [200]>)

There is also an option to log the exceptions that are caught by ignore_exceptions. Note, the full-exception traceback will not be printed in the logs. Instead, the error and it's message (using %r formatting) will appear. In the following code snippet, the ValueError raised by the function raises_error() will be sent to the logger at the 'warning' level.

import polling2
import logging
import mock

# basicConfig should sent warning level messages to the stdout.
logging.basicConfig()

# Create a function that raises a ValueError, then a RuntimeError.
raises_error = mock.Mock(side_effect=[ValueError('a message'), RuntimeError])

try:
    polling2.poll(
        target=raises_error,
        step=0.1,
        max_tries=3,
        ignore_exceptions=(ValueError),  # Only ignore the ValueError.
        log_error=logging.WARNING  # Ignored errors should be passed to the logger at warning level.
    )
except RuntimeError as _e:
    print "Un-ignored %r" % _e``

Future extensions

  • Add poll_killer(). Specify a hard timeout so that if the function being polled blocks and doesn't return, poll_killer() will raise a timeout.
    • Add an option to do via multiprocessing.
    • Add an option to do via threading - probably the default option.
  • Add poll_chain(). Have reason to poll a bunch of functions in a row? poll_chain() allows you to chain a bunch of polling functions together.
  • Allow step to be specificed as 0, so that we can poll continously. (Perhaps it's best to write a poll_continous() method.)

Release notes

0.4.5

  • 'Begin poll(*)' message is logged when poll() is called. Hopefully this means the user doesn't feel the need to write a message before every call to poll() to indicate how long the poll() might take.

0.4.4

  • Add is_value() function. A function that allows a user to easily build a custom checker, like is_truthy(), but for any value.

0.4.3

  • Add log_error parameter to the poll signature. Enables logging of ignored exceptions.

0.4.2

  • Add log_value() decorator and log parameter to poll signature. Enables logging of return_values.

0.4.0

  • Fixed polling function from waiting another sleep whenever the max_tries value has reached zero.
  • Remove test-only dependencies from requirements to install the package.
  • No longer testing on python 2.6. Add support for travis testing on python 3.6 and pypy 3.5.
  • Creation of polling2, forked from polling as previous maintainer seems to be ignoring issues and pull-requests.
  • Remove *a, **k from poll signature. This allows Type errors to be raised if caller spells arguments into correctly, making bugs easier to find.

0.3.0

  • Support Python 3.4+

0.2.0

  • Allow users to access a "last" attribute on the exceptions. This should hold the last evaluated value, which is the more common use case than getting the first value.
  • Fix a bug that actually ran 1 more time than value specified by max_tries

0.1.0

  • First version

Contributors

  • Justin Iso (original creator)
  • Donal Mee

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for polling2, version 0.4.5
Filename, size File type Python version Upload date Hashes
Filename, size polling2-0.4.5-py2.py3-none-any.whl (8.4 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size polling2-0.4.5.tar.gz (8.9 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page