shishito 3.1.16

Python module for selenium webdriver test execution

Shishito

Shishito is module for web and mobile application functional testing using Selenium Webdriver & Python. It runs tests using included libraries and generates nice test results output.

Documentation - http://shishito.readthedocs.org/en/latest/index.html (hosted on Read the Docs)

Features

• runs python Selenium Webdriver tests via PyTest
• easy configuration for local and remote (BrowserStack, Appium, ..) test execution
• contains useful test libraries
• generates HTML test results report (with screenshots for failed tests)
• designed to be used as a module (by multiple projects if needed)

Pre-requisities

Install Python moodules from requirements.txt

pip install -r requirements.txt

Webdriver drivers need to be setup (ChromeDriver, InternetExplorerDriver etc.)

Quick Start

1. clone Shishito repository. git clone [e-mail]:salsita/shishito.git
2. add shishito directory into PYTHONPATH environment variable
3. clone sample test project repository https://github.com/salsita/shishito-sample-project git clone [e-mail]:salsita/shishito-sample-project.git
4. if you want to use BrowserStack for running your tests, replace "bs_username", "bs_password" values with your credentials in shishito-sample-project/config/server_config.properties or pass it to runner python file as command line argument using flag --browserstack username:token
5. if you want to use Saucelabs for running your tests, add your credentials to saucelabs variable in shishito-sample-project/config/server_config.properties or pass it to runner python file as command line argument using flag --saucelabs username:token
6. set your preferred browser settings in shishito-sample-project/config/web/(browserstack|local).properties or for mobile apps in shishito-sample-project/config/mobile/appium.properties
7. run google_test_runner.py in sample project folder!

If you use local driver, you should now observe browser being started and tests running. There are information about progress shown in console output. Once testing is finished, HTML report can be found in:

shishito-sample-project/results folder # HTML report
shishito-sample-project/results_archive folder # zipped HTML report

Continuous Integration

Using Shishito with Continuous Integration solution, such as Jenkins, is easy! All you need to do is clone Shishito repo and add it into the PYTHONPATH.

Example script below (Jenkins "execute shell" build step):

#!/bin/bash
######################
# clone Shishito  #
######################

cd $WORKSPACE
git clone [e-mail]:salsita/shishito.git

######################
# VARIABLES          #
######################

export PYTHONPATH=${PYTHONPATH}:/$WORKSPACE/shishito

######################
# SCRIPT             #
######################

python google_test_runner.py

Command line options

--platform web         # define platform on which run tests (currently supported: web, mobile, generic)
--environmnet local    # define environment in which run tests (currently supported: local, browserstack, appium, remote)
--test_directory tests # define directory where to lookup for tests (project_root + test_directory)

# supported platform/environment combinations:
#   generic/local
#   generic/remote
#   web/local
#   web/browserstack
#   web/remote
#   mobile/appium (can run on local/remote appium server or on saucelabs)
#   node_webkit/node_webkit

--smoke # runs only tests with fixture "@pytest.mark.smoke"

--browserstack testuser1:p84asd21d15asd454 # authenticate on BrowserStack using user "testuser1" and token "p84asd21d15asd454"
--saucelabs testuser1:p84asd21d15asd454 # authenticate on Saucelabs using user "testuser1" and token "p84asd21d15asd454"
--test_rail [e-mail]:1AVFS51AS # authenticate on TestRail using user email "[e-mail]" and password "1AVFS51AS"

If no arguments are specified, Shishito, by default, searches for settings combinations in (server|local).properties files and runs tests according to them.

Configuration files

server_config.properties

• default configuration file with test variables
• changes to variables should be maintained in VCS; so that configuration can be reused for automated test execution

# modules
test_platform=web
test_environment=local

# test dir
test_directory=tests

# General
base_url=http://www.google.com
environment_configuration=Chrome

• test_platform - on which platform run tests (web, mobile)
• test_environment - in which environment run tests (local, browserstack, appium)
• test_directory - in which directory lookup for tests
• base_url - url that will be loaded by default upon start of each test
• environment_configuration - which configuration use from .properties file (used when tests are run without runner)
• remote_driver_url - remote driver hub. Selenium server needs to be running on this url.

local_config.properties

• if variable local_execution=True, script will look first search local config for test variables
• in case variables are not found, it will fall back to values in default server_config.properties
• changes to this file should not be maintained in VCS (they serve only for local test execution)

<platform>/<environment>.properties

• contains combinations, for which the tests should be executed
• e.g. browser and resolution for local web browser

conftest.py

• helper file that defines command line arguments, provides fixtures and other information for Shishito runner

Configuration

Shishito can be configured with command lines arguments and config files. Some configuration values are also added as arguments to PyTest (depends on test environment). Configuration values are looked up according to these priorities:

1. pytest.config
2. command line arguments
3. local configuration file (if enabled: local_execution=True)
4. server cofiguration file

Node-webkit configuration

Shishito is able to run tests against node-webkit applications. Current implementation does not allow tester to specify based URL, just to run application from URL directly specified within application. Creating of webdriver driver object is done by specific chromedriver which has to be placed in same directory as node-webkit application. Chromedriver will search for node-webkit binaries and start the application. Binaries have to have specific names otherwise chromedriver won't find them.
Node-webkit binary must have name:

• For Linux: nw
• For Windows: nw.exe
• For OS X: node-webkit.app

TEMPORARY SCREENSHOT ON FAILURE FUNCTIONALITY

Due to issue in Node-webkit chromedriver; there is added temporary screenshot on failure functionality using pyscreenshot module. This functionality takes screenshot of whole desktop not only node-webkit application window. This issue should be fixed in chromedriver 2.15. There is alpha version of node-webkit chromedriver v2.15. This functionality is going to be removed once issue is fixed.

Note: Ubuntu: It is necessary to install also python-imaging sudo apt-get install python-imaging

Troubleshooting for Node-webkit platform

If you see exception similar to one below

   raise WebDriverException(
       \"\'\" + os.path.basename(self.path) + \"\' executable needs to be \
           available in the path. Please look at \
           http://docs.seleniumhq.org/download/#thirdPartyDrivers \
           and read up at \
\>                   http://code.google.com/p/selenium/wiki/ChromeDriver")
E    WebDriverException: Message: 'chromedriver' executable needs to be available in the path. Please look at http://docs.seleniumhq.org/download/#thirdPartyDrivers and read up at http://code.google.com/p/selenium/wiki/ChromeDriver

/usr/local/lib/python2.7/dist-packages/selenium/webdriver/chrome/service.py:70: WebDriverException

You need to check Chromedriver file access rights mainly in Linux or OS X, Windows should be ok.

Test Management Support

Shishito support upload of test results to TestRail test management app. Following properties in server/local config have to be filled:

• test_rail - credentials for test rail. Can be also left empty and passed via cmd argument (see above)
• test_rail_url - URL of test rail instance. (example: https://mycompany.testrail.net)
• test_rail_project_id - ID of TestRail project (example: 1)
• test_rail_section_id - ID of TestRail test section (example: 2)
• test_rail_test_plan_id - ID of TestRail test plan (example: 5)
• test_rail_suite_id - ID of TestRail test suite (example: 1)

For further information, see TestRail API documentation http://docs.gurock.com/testrail-api2/start.

History

3.1.16 (2021-06-08)

• Logging options configuration

3.1.14 (2021-01-05)

• Hotfix Google Options

3.1.13 (2020-03-24)

• Fix bug in Test Rail API - handling skipped tests
• Add subsections to Test Rail

3.1.11 (2020-02-20)

• Update Test Rail API

3.1.10 (2019-08-12)

• Update the HTML report to contain even errors collected during PyTest collection phase

3.1.9 (2019-07-18)

• Minor fix of shi script not updating newly added packages to requirements.txt

3.1.8 (2019-07-03)

• Fix of HTML Test Report not showing errors in the setup phase
• Disable escaping of special characters error message in Test Report

3.1.7 (2019-07-01)

• support download_path for chromedriver 75+
• let download_path to be optional

3.1.6 (2019-06-18)

• support download_path and browser extension

3.1.5 (2019-05-22)

• Minor fixes

3.1.4 (2019-05-21)

• Fix Flaky & HTML report issue

3.1.3 (2019-03-07)

• extend appium Browserstack capabilities

3.1.2 (2019-01-14)

• support for environmental variables in config files

3.1.1 (2019-01-09)

• Fix problem with multiple drivers on Mobile devices
• Change how test names are obtained for Browserstack

3.1.0 (2018-11-27)

• Improve generated report

3.0.7 (2018-10-25)

• Hotfix - wrong config parsing

3.0.6 (2018-10-25)

• Add logging options to webdriver

3.0.5 (2018-09-10)

• Activate venv in shi executable script

3.0.4 (2018-08-15)

• Update the exacutable "shi" script to work in actual directory
• Add README and HISTORY files

3.0.3 (2018-08-14)

• Enable "shi" executable script to work with shishito installed with pip install shishito