pytest-axe 1.1.6

pytest plugin for axe-selenium-python

pytest-axe
==========

pytest-axe provides a variety of features to simplify accessibility testing using ``axe-selenium-python``.

.. image:: https://img.shields.io/badge/license-MPL%202.0-blue.svg
:target: https://github.com/mozilla-services/pytest-axe/blob/master/LICENSE.txt
:alt: License
.. image:: https://img.shields.io/pypi/v/pytest-axe.svg
:target: https://pypi.org/project/pytest-axe/
:alt: PyPI
.. image:: https://travis-ci.org/mozilla-services/pytest-axe.svg
:target: https://travis-ci.org/mozilla-services/pytest-axe
:alt: Travis
.. image:: https://img.shields.io/github/issues-raw/mozilla-services/pytest-axe.svg
:target: https://github.com/mozilla-services/pytest-axe/issues
:alt: Issues
.. image:: https://api.dependabot.com/badges/status?host=github&repo=mozilla-services/axe-selenium-python
:target: https://dependabot.com
:alt: Dependabot
.. image:: https://coveralls.io/repos/github/mozilla-services/pytest-axe/badge.svg?branch=master
:target: https://coveralls.io/github/mozilla-services/pytest-axe?branch=master

Requirements
------------

You will need the following prerequisites in order to use pytest-axe:

- Python 2.7 or 3.6
- axe-selenium-python >= 2.1.1
- The appropriate driver for the browser you intend to use, downloaded and added to your path, e.g. geckodriver for Firefox:

- `geckodriver <https://github.com/mozilla/geckodriver/releases>`_ downloaded and `added to your PATH <https://stackoverflow.com/questions/40208051/selenium-using-python-geckodriver-executable-needs-to-be-in-path#answer-40208762>`_

Optional
--------

- pytest-selenium >= 3.0.0

**pytest-selenium is required to use the axe pytest fixture.**

- PyPOM >= 1.3.0

**PyPOM is required to use the run_axe helper method.**

Installation
------------

To install pytest-axe:

.. code-block:: bash

$ pip install pytest-axe

Usage
------

``pytest-axe`` adds a command-line option for specifying whether or not to run accessibility tests.

Running pytest with ``--axe`` will run only tests marked as accessibility, i.e. ``@pytest.mark.accessibility``.

The absence of this command line option will run only tests **not** marked as accessibility.

``pytest-axe`` also includes a pytest fixture to handle set up and tear down.

The following example will run aXe-core against the entire page, and check for violations of any impact level.

.. code-block:: python

import pytest

@pytest.mark.nondestructive
def test_header_accessibility(axe):
violations = axe.run()
assert len(violations) == 0, axe.report(violations)

You can also customize your accessibility tests by defining ``context``, ``options``, or ``impact``.

.. code-block:: python

import pytest

@pytest.mark.nondestructive
def test_header_accessibility(axe):
violations = axe.run('header', None, 'critical')
assert len(violations) == 0, axe.report(violations)

**The above example runs aXe against only the content within the** ``<header>`` **tag, and filters for violations labeled** ``critical``.

For more information on ``context`` and ``options``, view the `aXe
documentation here <https://github.com/dequelabs/axe-core/blob/master/doc/API.md#parameters-axerun>`_.

The third parameter, ``impact``, allows you to filter violations by their impact
level. The options are ``'critical'``, ``'serious'`` and ``'minor'``, with the
default value set to ``None``.

This will filter violations for the impact level specified, and **all violations with a higher impact level**.

The axe fixture uses ``base_url`` defined in the pytest command or in a config file.

.. code-block:: bash

$ pytest --base-url http://www.mozilla.com --driver Firefox test_accessibility.py

Parameterized Tests and Expected Failures
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
As of version 1.1.0, ``pytest-axe`` provides helper methods to generate individual tests for
each accessibility rule.

Generating test cases for each accessibility rule enables the ability to
mark specific **rules** as expected failures. To enable test parameterization,
parameterized tests should be defined in separate classes, with xfailed rules
set in the class-level `params` dictionary.


Example
*******

.. code-block:: python
import pytest
from selenium import webdriver
from pytest_axe.pytest_axe import PytestAxe as Axe
from pytest_axe.parametrize_tests import *

class TestHomePageAccessibility(object):
params = {
# Used by pytest-axe to generate tests and configure xfails
"color-contrast": "Reason: GitHub issue #5014 https://github.com/mozilla-services/screenshots/issues/5014",
"html-has-lang": "Reason: GitHub issue #5015 https://github.com/mozilla-services/screenshots/issues/5015",
"landmark-one-main": "Reason: GitHub issue #5016 https://github.com/mozilla-services/screenshots/issues/5016",
"link-name": "Reason: GitHub issue #5017 https://github.com/mozilla-services/screenshots/issues/5017",
"meta-viewport": "Reason: GitHub issue #5018 https://github.com/mozilla-services/screenshots/issues/5018",
"region": "Reason: GitHub issue #5016 https://github.com/mozilla-services/screenshots/issues/5016",
}

@pytest.mark.accessibility
def test_home_page_accessibility(self, rule):
"""Run accessibility audits on the home page of Screenshots."""
driver = webdriver.Firefox()
driver.get("https://firefox.com")
axe = Axe(driver)
axe.inject()

results = axe.run_single_rule(rule)
assert len(results) == 0, home_page.report(results)

PyPOM Example
^^^^^^^^^^^^^^^^^^^^^

**These examples are dependent on the use of** `PyPOM <https://github.com/mozilla/PyPOM>`_ **, and assumes any** ``Page`` **object has a** ``selenium`` **object attribute.**

.. code-block:: python
from pytest_axe.pytest_axe import run_axe


@pytest.mark.accessibility
def test_login_page_accessibility(login_page):
"""Test login page for accessibility violations."""
run_axe(login_page)

And with custom run options:

.. code-block:: python
from pytest_axe.pytest_axe import run_axe


@pytest.mark.accessibility
def test_login_page_accessibility(login_page):
"""Test login page header for critical accessibility violations."""
run_axe(login_page, "header", None, "critical")

Recording Results
^^^^^^^^^^^^^^^^^^^

``pytest-axe`` checks for an environment variable ``ACCESSIBILITY_REPORTING``.

To enable writing the aXe JSON results to file, set ``ACCESSIBILITY_REPORTING=true``.

Currently, this will write the JSON files to the root of your test directory, with the page title and a timestamp as the file name.


Resources
---------

- `Issue Tracker <http://github.com/mozilla-services/pytest-axe/issues>`_
- `Code <http://github.com/mozilla-services/pytest-axe/>`_
- `axe-selenium-python <https://github.com/mozilla-services/axe-selenium-python>`_

CHANGELOG
----------

Version 1.1.6
^^^^^^^^^^^^^
** Breaking Changes **
- Renaming of class methods:
- ``run`` has been renamed to ``run_and_return_violations``
- ``analyze`` has been renamed to ``run_test_and write_violations_to_file``
- references to the parent class (``Axe`` inherited from **axe-selenium-python**) method ``execute`` have been updated to ``run`` to match changes in axe-selenium-python==2.1.5.

Version 1.1.3
^^^^^^^^^^^^^
- Modified the way that tests are generated. Rather than configuring xfails in conftest.py, they are defined as class-level variables.

Version 1.1.0
^^^^^^^^^^^^^
- Added the ability to generate individual test cases for each accessibility rule, which enables xfailing specific accessibility rules.

Version 1.0.0
^^^^^^^^^^^^^^
- Transferred functions and methods from ``axe_selenium_python``.
- ``run_axe`` helper method, to simplify accessibility testing for test suites using PyPOM.
- ``run()`` method, which injects the aXe JavaScript, runs aXe against the page, filters the results based on a specified ``impact`` level, and returns a dictionary of only violations, with the ``ruleId`` as the key.
- ``impact_included``, used by ``run()`` to filter checks by a specificed impact level.
- Environment variable ``ACCESSIBILITY_REPORTING`` to enable recording results.
- ``analyze()``, which calls ``run()`` and writes the JSON results to file, if enabled using the environment variable.

Version 0.2.0
^^^^^^^^^^^^^^
- Added a command line argument to filter tests based on the presence or absence of a pytest accessibility marker.