Skip to main content

Tiny aiohttp wrapper for http request gathering in sync mode

Project description

https://codecov.io/gh/night-crawler/async-fetcher/branch/master/graph/badge.svg https://travis-ci.org/night-crawler/async-fetcher.svg?branch=master https://img.shields.io/pypi/v/async-fetcher.svg PyPI Wheel Requirements Status Supported versions Supported implementations

Installation

pip install async-fetcher
# or
pip install -e git+https://github.com/night-crawler/async-fetcher.git@#egg=async-fetcher

Sample

af = AsyncFetch({
    'first': AsyncFetch.mk_task(build_url('request-info')),
    'second': AsyncFetch.mk_task('http://example.com/'),
    'fail': AsyncFetch.mk_task(build_url('404'))
})
responses = af.go()

mk_task static method can take this arguments:

:param fail_silently: bool, do not raise exceptions, default is False;
    for test purpose, do not use in production
:param num_retries: int, *optional*, default is -1; -1 - no retries; 0 - use AsyncFetch.num_retries
:param autodetect_content_type: if no `content-type` header was specified, set `content-type` as
    `application/json` for dict, and `text/html` otherwise; default is True
:param json_encoder: JSONEncoder, *optional*, JSON encoder for data serialization
    tries to use DRF's encoder, or default JSONEncoder from json package; default is JSONEncoder
:param url: str, *required*, url address
:param api_key: str, optional API key passed into HEADERS dict
:param data: dict, *optional*, request data. Default is None,
:param method: str, *optional*, HTTP request method. Default is True.
:param headers: dict, *optional*, optional HTTP headers
:param response_type: str, *optional*, HTTP response type
    (in fact it's just aiohttp's method name, i.e. text, or json); default is 'json'
:param language_code: str, set `accept-language` header
:param timeout: float, *optional*, time to wait for response in seconds before TimeoutError
:param query: dict, *optional*, url get arguments
:param do_not_wait: bool, *optional*, fail silently with no retries and empty resultset
af0 = AsyncFetch({})
tcp_connector = af0.get_tcp_connector()

af1 = AsyncFetch({
    '1': AsyncFetch.mk_task(build_url('request-info')),
    '2': AsyncFetch.mk_task(build_url('request-info')),
}, tcp_connector=tcp_connector)
responses = af1.go()

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for async-fetcher, version 0.3.6
Filename, size File type Python version Upload date Hashes
Filename, size async_fetcher-0.3.6-py3-none-any.whl (12.0 kB) File type Wheel Python version 3.6 Upload date Hashes View hashes
Filename, size async-fetcher-0.3.6.tar.gz (9.1 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page