Skip to main content

Python3 library to scale functions using asyncio

Project description

Vast

MIT badge Version badge RepoSize badge Python3.6 badge Platform badge

A utility to easily scale functionality

Table of Contents


Requirements

  • Python3.6+

Installation

  • Create a new virtual environment with python 3.6+

    • Install the aiovast library
    $ git clone https://www.github.com/tannerburns/aiovast
    $ cd aiovast
    $ pip3 install .
    

Information

Details about the aiovast utility

Vast

Main variables
  • loop: asyncio.new_event_loop
  • max_async_pool: int, default=32
  • max_futures_pool: int, default=10000

Vast Event Loop

Main method
  • run_in_eventloop
    • arg1: functionObject, Callable
      • the function to run in the event loop
    • arg2: listOfArgs, list
      • the list of arguments to be mapped to the function in the event loop
    • kwarg: report, default= False
      • returns results and statistics about the event loop runtime
    • kwarg: disable_progress_bar, default= False
      • disables the progress bar from printing while the event loop runs
    • kwarg: progress_bar_color, default= green
      • provide another color for the progress bar template

VastSession

Variables
  • loop: asyncio.new_event_loop
  • max_async_pool: int, default=32
  • max_futures_pool: int, default=10000
  • self.session: requests.session

Vast Bulk Requests

A function that can handle any method requests will accept     
  • bulk_requests
    • arg1: listOfCalls, list
      • format: [ [[method: string, url: string], options: dictionary], [[method: string, url: string], options: dictionary], .. ]

Function calls for single method types

Vast Bulk Get
  • bulk_get_requests
    • arg1: listOfCalls, list
      • format: [ [[url: string], options: dictionary], [[url: string], options: dictionary], .. ]

Vast Bulk Post
  • bulk_post_requests
    • arg1: listOfCalls, list
      • format: [ [[url: string], options: dictionary], [[url: string], options: dictionary], .. ]

Vast Bulk Put
  • bulk_put_requests
    • arg1: listOfCalls, list
      • format: [ [[url: string], options: dictionary], [[url: string], options: dictionary], .. ]

Vast Bulk Delete
  • bulk_delete_requests
    • arg1: listOfCalls, list
      • format: [ [[url: string], options: dictionary], [[url: string], options: dictionary], .. ]

Vast Bulk Head
  • bulk_head_requests
    • arg1: listOfCalls, list
      • format: [ [[url: string], options: dictionary], [[url: string], options: dictionary], .. ]

Examples

#Basic add example
def add(x, y): return x + y

if __name__ == '__main__':
    rets = [add(x, y) for x in range(0, 5) for y in range(5, 10)]
#Example bulk add using aiovast class
from aiovast import Vast

def add(x, y): return x + y

if __name__ == '__main__':
    aiovast = Vast()
    args = [[[x, y]] for x in range(0, 5) for y in range(5, 10)]
    rets = aiovast.run_in_eventloop(add, args)
#Example using Vast context manager
from aiovast import Vast

def add(x, y): return x + y

if __name__ == '__main__':
    args = [[[x, y]] for x in range(0, 5) for y in range(5, 10)]
    with Vast() as aiovast:
        rets = aiovast.run_in_eventloop(add, args)
#Example bulk add using decorator
from aiovast.decorators import vast_loop

@aiovast_loop(max_async_pool=16)
def add_in_bulk(x, y):
    return x+y

if __name__ == '__main__':
    args = [[[x, y]] for x in range(0, 5) for y in range(5, 10)]
    rets = add_in_bulk(args)
#Vast session for sending bulk requests
from aiovast.requests import VastSession

session = VastSession(max_async_pool=4)
calls = [
    (['get', 'https://www.google.com'], {'headers': {'User-Agent':'custom'}}),
    (['post', 'https://www.github.com'], )
]
responses = session.bulk_requests(calls)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for aiovast, version 4.0.3
Filename, size File type Python version Upload date Hashes
Filename, size aiovast-4.0.3.tar.gz (6.0 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page