Skip to main content

GUI automation tool

Project description

guibot Build Status Documentation Status Language grade: Python codecov

A tool for GUI automation using a variety of computer vision and display control backends.

Introduction

In order to do GUI automation you usually need to solve two problems: first, you need to have a way to control and interact with the interface and platform you are automating and second, you need to be able to locate the objects you are interested in on the screen. Guibot helps you do both.

To interact with GUIs, Guibot provides the controller module which contains a common interface for different display backends, with methods to move the mouse, take screenshots, type characters and so on. The backend to use will depend on how your interface is accessible, but the ones currently supported are listed below.

To locate an element on the screen, you will need an image representing the screen, a target and a finder - the finder looks for the target on the image, and returns the coordinates to the region where that target appears. Finders, like display controllers, are wrappers around different backends that Guibot supports (see below), while targets are a representation of the object you are looking for, such as images, text, or patterns.

Finally, to bridge the gap between controlling the GUI and finding elements, the region module is provided. It represents a subregion of a screen, and contains methods to locate targets in this region and interact with the graphical interface using the chosen backends.

Supported backends

Supported Computer Vision (CV) backends are based on

  • OpenCV
    • Template matching
    • Contour matching
    • Feature matching
    • Haar cascade matching
    • Template-feature and mixed matching
  • Tesseract OCR
    • Text matching through pytesseract, tesserocr, or OpenCV's bindings
  • PyTorch
    • R-CNN matching through Faster R-CNN or Mask R-CNN
  • autopy
    • AutoPy matching

Supported Display Controller (DC) backends are based on

Resources

Homepage: http://guibot.org

Documentation: http://guibot.readthedocs.io

Installation: https://github.com/intra2net/guibot/wiki/Packaging

Issue tracking: https://github.com/intra2net/guibot/issues

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

guibot-0.41.1.tar.gz (2.8 kB view details)

Uploaded Source

Built Distribution

guibot-0.41.1-py3-none-any.whl (83.4 kB view details)

Uploaded Python 3

File details

Details for the file guibot-0.41.1.tar.gz.

File metadata

  • Download URL: guibot-0.41.1.tar.gz
  • Upload date:
  • Size: 2.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.6.1 requests/2.24.0 setuptools/50.3.2 requests-toolbelt/0.9.1 tqdm/4.51.0 CPython/3.5.7

File hashes

Hashes for guibot-0.41.1.tar.gz
Algorithm Hash digest
SHA256 6d4ae0b284b235e9008b3e2c0c7b8b99cc6809b8d8b26cb247165f2e8af04b90
MD5 17bc6c63efa31a99686b5858b246bd0a
BLAKE2b-256 4ab2294fb353c60a1f99d382a5d499af5b71239492cbe53f05d1f7055a4d3aa8

See more details on using hashes here.

File details

Details for the file guibot-0.41.1-py3-none-any.whl.

File metadata

  • Download URL: guibot-0.41.1-py3-none-any.whl
  • Upload date:
  • Size: 83.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.15.0 pkginfo/1.6.1 requests/2.24.0 setuptools/50.3.2 requests-toolbelt/0.9.1 tqdm/4.51.0 CPython/3.5.7

File hashes

Hashes for guibot-0.41.1-py3-none-any.whl
Algorithm Hash digest
SHA256 0e6691aa4ba1f17cd744a45523a1d9f1a3c7ea4fdb23d15a6b089b5ee6b4cf99
MD5 d6d5abeec819c72f663b478323ae8ab3
BLAKE2b-256 29ec3ff4fc4ead63375bed442b42bced486ab1d85239d7c6ab7e71730f8e34b5

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page