Skip to main content

pyvisauto - a vision-based automation tool

Project description


pyvisauto is a Python visual automation tool. Inspired by Sikuli, pyvisauto provides Python-native easy-to-use abstractions for complex interactions with on-screen visual elements by wrapping OpenCV (specifically opencv-contrib-python-headless), pyautogui, pytesseract, and numpy.

Features include:

  • OpenCV and numpy-driven image matching of on-screen elements
  • TesseractOCR support
  • Methods to find an image match (find), find all matches (find_all), check if a match exists (exists), wait until an image match occurs (wait), and wait for an image match to disappear (wait_vanish)
  • Methods to click and hover over regions and matches (click and hover, respectively) with random x and y coordinates within the region
  • Sub-region and cached matching for faster performance
  • Method to save screenshots of matches and regions to a file (screenshot)


pyvisauto has been tested on Python 3.7. The opencv-contrib-python-headless package limits availability to Python 2.7 and 3.4 ~ 3.7. While pyvisauto should be compatible with Python 3, Python 3.8 is currently not supported.

Installation and Usage

  1. Install OS-specific dependencies:
    • Windows: No extra dependencies needed
    • Linux: python3-xlib
    • OSX: pyobjc-core and pyobjc, in that order
  2. Install pyvisauto using pip: pip install pyvisauto
  3. Import pyvisauto: import pyvisauto
  4. Read the Quick Start and API docs

Quick Start

  • Define a full-screen region and assign it to r:

    r = pyvisauto.Region()
  • Define a region with the upper-left corner at x: 50px and y: 100px, with a width of 500px and height of 300px and assign it to r:

    r = pyvisauto.Region(50, 100, 500, 300)
  • Find the image asset1.png in the defined region, with a similarity score of 0.9:

    match1 = r.find('asset1.png', 0.8)
  • If there has been no visual changes in the defined region, subsequent find actions can be expedited by passing in cached=True:

    match2 = r.find('asset2.png', 0.9, cached=True)
  • find_all and exists can be used in a similar fashion as find.

  • Hover over a random point in the first returned match:

  • Click a random point in the second returned match:
  • One can use wait and vanish to wait for on-screen changes, detected by the presence or disappearance of an image on-screen, respectively:

    r.wait('wait_asset1.png', 30, 0.8)

    The above code will wait for wait_asset1.png in the previously defined region r, with a minimum similarity score of 0.8, waiting a maximum of 30 seconds before throwing a FindFailed exception. vanish, on the other hand, throws a VanishFailed exception. Both exceptions are defined in the pyvisauto module.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pyvisauto, version 1.0.3
Filename, size File type Python version Upload date Hashes
Filename, size pyvisauto-1.0.3-py3-none-any.whl (22.1 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size pyvisauto-1.0.3.tar.gz (6.3 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page