aiovast 4.0.3

Python3 library to scale functions using asyncio

Vast

A utility to easily scale functionality

Table of Contents

• Requirements
• Installation
• Information
  • Vast
    • Vast Event Loop
  • VastSession
    • Vast Bulk Requests
      • Vast Bulk Get
      • Vast Bulk Post
      • Vast Bulk Put
      • Vast Bulk Delete
      • Vast Bulk Head
• Examples

Requirements

• Python3.6+

Installation

• Create a new virtual environment with python 3.6+

  • Install the aiovast library

  $ git clone https://www.github.com/tannerburns/aiovast
  $ cd aiovast
  $ pip3 install .
  

Information

Details about the aiovast utility

Vast

Main variables

• loop: asyncio.new_event_loop
• max_async_pool: int, default=32
• max_futures_pool: int, default=10000

Vast Event Loop

Main method

• run_in_eventloop
  • arg1: functionObject, Callable
    • the function to run in the event loop
  • arg2: listOfArgs, list
    • the list of arguments to be mapped to the function in the event loop
  • kwarg: report, default= False
    • returns results and statistics about the event loop runtime
  • kwarg: disable_progress_bar, default= False
    • disables the progress bar from printing while the event loop runs
  • kwarg: progress_bar_color, default= green
    • provide another color for the progress bar template

VastSession

Variables

• loop: asyncio.new_event_loop
• max_async_pool: int, default=32
• max_futures_pool: int, default=10000
• self.session: requests.session

Vast Bulk Requests

A function that can handle any method requests will accept     

• bulk_requests
  • arg1: listOfCalls, list
    • format: [ [[method: string, url: string], options: dictionary], [[method: string, url: string], options: dictionary], .. ]

Function calls for single method types

Vast Bulk Get

• bulk_get_requests
  • arg1: listOfCalls, list
    • format: [ [[url: string], options: dictionary], [[url: string], options: dictionary], .. ]

Vast Bulk Post

• bulk_post_requests
  • arg1: listOfCalls, list
    • format: [ [[url: string], options: dictionary], [[url: string], options: dictionary], .. ]

Vast Bulk Put

• bulk_put_requests
  • arg1: listOfCalls, list
    • format: [ [[url: string], options: dictionary], [[url: string], options: dictionary], .. ]

Vast Bulk Delete

• bulk_delete_requests
  • arg1: listOfCalls, list
    • format: [ [[url: string], options: dictionary], [[url: string], options: dictionary], .. ]

Vast Bulk Head

• bulk_head_requests
  • arg1: listOfCalls, list
    • format: [ [[url: string], options: dictionary], [[url: string], options: dictionary], .. ]

Examples

#Basic add example
def add(x, y): return x + y

if __name__ == '__main__':
    rets = [add(x, y) for x in range(0, 5) for y in range(5, 10)]

#Example bulk add using aiovast class
from aiovast import Vast

def add(x, y): return x + y

if __name__ == '__main__':
    aiovast = Vast()
    args = [[[x, y]] for x in range(0, 5) for y in range(5, 10)]
    rets = aiovast.run_in_eventloop(add, args)

#Example using Vast context manager
from aiovast import Vast

def add(x, y): return x + y

if __name__ == '__main__':
    args = [[[x, y]] for x in range(0, 5) for y in range(5, 10)]
    with Vast() as aiovast:
        rets = aiovast.run_in_eventloop(add, args)

#Example bulk add using decorator
from aiovast.decorators import vast_loop

@aiovast_loop(max_async_pool=16)
def add_in_bulk(x, y):
    return x+y

if __name__ == '__main__':
    args = [[[x, y]] for x in range(0, 5) for y in range(5, 10)]
    rets = add_in_bulk(args)

#Vast session for sending bulk requests
from aiovast.requests import VastSession

session = VastSession(max_async_pool=4)
calls = [
    (['get', 'https://www.google.com'], {'headers': {'User-Agent':'custom'}}),
    (['post', 'https://www.github.com'], )
]
responses = session.bulk_requests(calls)