Skip to main content
Python Software Foundation 20th Year Anniversary Fundraiser  Donate today!

GUI automation tool

Project description

guibot Build Status Documentation Status Language grade: Python codecov

A tool for GUI automation using a variety of computer vision and display control backends.


In order to do GUI automation you usually need to solve two problems: first, you need to have a way to control and interact with the interface and platform you are automating and second, you need to be able to locate the objects you are interested in on the screen. Guibot helps you do both.

To interact with GUIs, Guibot provides the controller module which contains a common interface for different display backends, with methods to move the mouse, take screenshots, type characters and so on. The backend to use will depend on how your interface is accessible, but the ones currently supported are listed below.

To locate an element on the screen, you will need an image representing the screen, a target and a finder - the finder looks for the target on the image, and returns the coordinates to the region where that target appears. Finders, like display controllers, are wrappers around different backends that Guibot supports (see below), while targets are a representation of the object you are looking for, such as images, text, or patterns.

Finally, to bridge the gap between controlling the GUI and finding elements, the region module is provided. It represents a subregion of a screen, and contains methods to locate targets in this region and interact with the graphical interface using the chosen backends.

Supported backends

Supported Computer Vision (CV) backends are based on

  • OpenCV
    • Template matching
    • Contour matching
    • Feature matching
    • Haar cascade matching
    • Template-feature and mixed matching
  • Tesseract OCR
    • Text matching through pytesseract, tesserocr, or OpenCV's bindings
  • PyTorch
    • R-CNN matching through Faster R-CNN or Mask R-CNN
  • autopy
    • AutoPy matching

Supported Display Controller (DC) backends are based on





Issue tracking:

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for guibot, version 0.41.1
Filename, size File type Python version Upload date Hashes
Filename, size guibot-0.41.1-py3-none-any.whl (83.4 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size guibot-0.41.1.tar.gz (2.8 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page