Write Selenium Tests

Use axe DevTools Java Selenium

This axe DevTools configuration does not inherently bind to any assertion library. This allows for simple accessibility testing as well as full test customization and use with custom assertions.

Prerequisites

In order to use axe DevTools Java Selenium and produce results with it, you must have already created a test file with axe DevTools and have your choice of Selenium WebDriver imported & initialized. If you haven't completed this step, first read this guide on how to do so.

Run a Scan

With this configuration, running a basic accessibility scan requires only three lines of code. In your test file, replace <URL> with the desired web address of the page you want to test. The scan results are saved in the results variable

webdriver.get("<URL>");
Results results = axeSelenium.run(axedriver);
webdriver.quit();

Using the Scan Results

Once you've run the scan, you may want to do something with the results. The simplest option is to print the results to the console.

System.out.print(results);

You can also use the results to check for accessibility violations. The simplest way to do this is with a statement like this:

if (!results.violationFree()) { 
    //do something, like throw an exception
}

For more information on how to utilize the results object to write custom tests, see the page on using the results.

Sample Test File

This test file works with the same building blocks covered in import and initialize as well as the writing tests guide on this page.

import com.deque.html.axedevtools.selenium.*;
import com.deque.html.axedevtools.selenium.results.*;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

public class Example {
    public static void main(final String[] args) {
        AxeSelenium axeselenium = new AxeSelenium();
        WebDriver webdriver = new ChromeDriver();
        AxeDriver axedriver = new AxeDriver(wd);

        webdriver.get("<URL>");

        Results results = axeselenium.run(axedriver);

        webdriver.quit();

        if (!results.violationFree()) {
            int violationCount = results.getViolations().size();
            System.out.printf("Found %d violations!\n", violationCount);
            System.exit(1);
        }

        System.out.println("No violations found!");
    }
}

In this example, Chrome and ChromeDriver are used. This can be substituted to any other selenium browser driver and browser. If this test file finds any accessibility issues, it prints out the number of found violations and exits with status code 1 (error). To use this test file in your own environment, you will have to add a URL to the line

webdriver.get("<URL>");

Additional Reference

In addition to the basic usage outlined above, axe DevTools Java Selenium provides several chaining methods to modify the way the scan is run. These methods can modify the scan's scope and ruleset to any desired configuration. By default, the entire page is scanned and checked for accessibility violations according to WCAG 2.0 level AA guidelines. An important note is that subsequent calls to the run method will overwrite any of the chain methods' preferences below. Changes made to scope or rules do not persist to multiple scans using these chan methods.

Scoping

By default, the entire page is scanned. However, axe DevTools supports the ability to scan specific portions of a page with its scoping chain methods. These scoping methods effectively change the "surface area" of the page to be scanned to the desired target area(s). Additionally, an ignore violations method is available. "Scoping" the page with this method doesn't reduce the area of the scan, but it will ignore violations of a specified type within the specified area and still scan the area for other potential violations. All of these methods work based on CSS selector areas. The area of each CSS selector on a given page can be determined by using the "inspect element" widget of your browser's developer tools.

Scoping Methods

As explained above, these methods effectively change the surface area of the page to be scanned. There are two methods in this category, an "inclusive" method and an "exclusive" method. They can be mutually chained with each other to create multiple enclaves and exclaves of scanning surface area.

Scoping - Including

The including scoping chain method alters the scan to only run within the page area specified by the CSS selector(s) passed to it. There are several ways to use the including method: single scope, multiple scope, compound scope, and IFrame scoping.

This method can be used with a single selector as follows:

axeselenium.including("#selector1").run(axedriver);

This scan will only run within the scope of selector 1

You can set up a scan with multiple scopes:

axeselenium.including("#selector1").including("#selector2").run(axedriver);

This call will scan within the scope of selector 1 and selector 2. They can either be comma separated or declared in a separate including clause.

Compound scoping can be accomplished with the following syntax:

axeselenium.including("#selector1 .selector2").run(axedriver);

This compound scoping results in only elements with selector 2 within selector 1 being scanned. This is set up by not comma separating the selectors.

In order to scope within an IFrame, a List<String> can be passed. The scope to be scanned within the IFrame is passed in as an additional parameter.

axeselenium.including(Arrays.asList(new String[] {"#frame1", "#selector1"})).run(axedriver);

This scan runs on elements within selector 1, which is inside of frame 1.

Scoping within multiple IFrames works the same way as standard multiple selection. Simply call including again.

axeselenium.including(Arrays.asList(new String[] {"#frame1", "#selector1"}))
    .including(Arrays.asList(new String[] {"#frame2", "#selector2"})).run(axedriver);

This scan runs only inside of selector 1 within frame 1, and selector 2 inside frame 2.

It is also possible to scope within nested IFrames:

axeselenium.including(Arrays.asList(new String[] {"#frame1", "#frame2", "#selector1"})).run(axedriver);

This scan runs within the scope of selector 1 inside of frame 2, which is itself inside frame 1

Scoping - Excluding

In addition to the including chain method, this API has an excluding method. This method alters the scan so that only elements outside of the specified scope will be scanned. Its setup and usage is identical to the including method, so refer to the above section for more details and examples of how it can be used. One important detail to remember is that the including and excluding methods can be chained with each other to create complex scopes much more simply than by only using include or exclude.

This example shows one such way to pair the within and excluding methods:

axeselenium.including("#selector1").excluding("#selector2").run(axedriver);

This scan runs only on the part of the page that is within selector 1 that is not also within selector 2.

Scoping - Ignoring Violations

The final scoping related chain method is the ignoring results method. This method doesn't alter the overall scope of the scan, but will ignore violations of a specified type within a specified scope. It works by passing the selector(s) of the element(s) you want to ignore as an array, coupled with the rule they violate.

axeselenium.ignoring("[\".sidebar\", \"#branded-content\"]", "color-contrast").run(axedriver);

This scan will ignore color contrast violations within the sidebar section and the branded-content section.

Rules

There are three ways to modify your rule configuration for each scan. These different methods allow for different granularity when selecting a rules to test against. The least granular ruleset method selects a ruleset, or larger group of rules based on the common accessibility standards. The slightly more granular ruleset standards method, also known as tags, selects rules based on which areas of a larger accessibility standard they fall within. Finally, there are three options for configuration with one rule at a time. All together, these chain methods allow you to tune your scan to run precisely the rules you want.

For information on axe-core rules, visit the ruleset overview page.

Ruleset Selection

This method selects rules by their most aggregated association - a ruleset. A ruleset is a collection of all the rules which pertain to an industry wide accessibility standard. Rulesets for WCAG 2.0, 2.1, and 2.2; ADA Section 508; Trusted Tester v5; and EN 301 549. Additionally, best practice rules can be enabled with a boolean flag.

Rulesets are often comprised of multiple tags, which are rules grouped by more specific distinctions. For example, the wcag2 ruleset contains all rules tagged wcag2a WCAG 2.0 level A rules, wcag2aa for WCAG 2.0 level AA rules, and wcag2aaa for WCAG 2.0 level AAA rules.

Use this option when you want to select a ruleset unmodified.

axeselenium.forRuleset("wcag2.1").run(axedriver);

This scan will be run with only WCAG 2.1 rules.

To enable best practice rules, use the second form of forRuleset.

axeselenium.forRuleset("wcag2.1", true).run(axedriver);

Standard Selection

The accessibility standard, or tag, chain method allows slightly more granularity within axe-core's rulesets. For WCAG based rules, this method allows you to select only rules tagged single-A or double-A. This list breaks down the available tag names and their corresponding standards.

This option is particularly useful when only testing a specific portion a ruleset, for example, only the single A rules under WCAG 2.0.

While not defined explicitly as a part of any accessibility standard, we offer a number of rules we call "best practices." While not strictly necessary, testing with these rules will help your website be as equitable in use as possible.

A single standard can be selected as follows:

axeselenium.accordingTo("wcag2a").run(axedriver);

This scan will run with only rules tagged under WCAG 2.0 level A

Additionally, multiple tags can be specified to select multiple accessibility standards:

axeselenium.accordingTo("wcag2a", "section508").run(axedriver);

These scans will run both the rules tagged under WCAG 2.0 level A and ADA Section 508.

Rule Selection

Finally, you can alter your ruleset by a single rule. For information on the names of all of axe-core's rules and what each rule tests for, see the axe-core rule descriptions documentation.

These methods work well when your desired final group of rules is only differnet from an existing ruleset or standard/tag by a few individual rules. To build a completly customized ruleset, check out our custom rule options.

Checking Additional Rules

The checking chain method allows you to add additional rules to test with in addition to your standard ruleset.

This method can be used to check a single additional rule:

axeselenium.checking("label").run(axedriver);

where the scan is run with the default ruleset in addition to the "label" rule.

It can also be used to check multiple additional rules in two different ways:

axeselenium.checking("label", "tabindex").run(axedriver);
axeselenium.checking("label").checking("tabindex").run(axedriver);

where the scan is run with the default ruleset in addition to the "label" and "tabindex" rules.

It can also be used to add a single, or multiple, rules to a specified tag:

axeselenium.accordingTo("wcag2a").checking("tabindex").run(axedriver);

where the scan is run testing against WCAG 2.0 level A rules, with the addition of the "tabindex" rule.

Skipping Rules

Similar to the checking method, the skipping method changes the default rules to be used by the rule(s) specified. However, instead of adding it to the ruleset, the skipping method removes the specified rule(s) from those used to test the page. Its setup and usage is identical to the checking method, so refer to the above section for more details and examples on how to use it. One important detail to remember is that the checking and skipping methods can be chained with each other to create customized rulesets much more simply than by using only the ignoring or checking methods alone.

axeselenium.accordingTo("wcag2a").checking("tabindex").skipping("label").run(axedriver);

This example shows a scan testing against WCAG 2.0 level A rules, with the addition of the "tabindex" rule and the removal of the "label" rule.

Checking Only

The checking only method checks only the rule(s) specified. Any rule not explicitly specified will not be included in the scan.

This chain method can be used to specify a single rule:

axeselenium.checkingOnly("tabindex").run(axedriver);

where the scan is run with only the "tabindex" rule.

It can also be used to specify more than one rule in these two ways:

axeselenium.checkingOnly("label", "tabindex").run(axedriver);
axeselenium.checkingOnly("label").checkingOnly("tabindex").run(axedriver);

where the scan is run with only the "label" and "tabindex" rules.

Disabling iframe Testing

On volatile pages, iframes can cause issues. Axe needs to be injected into every frame and configured the same way in each frame. If iframes are added or removed while AxeSelenium::run is testing your page, it is possible for exceptions to be thrown or unexpected behavior to occur. As a failsafe, AxeSelenium exposes a method to disable all iframe interaction, AxeSelenium::disableIframeTesting. When enabled, axe-core will not be injected into iframes in the page, nor will it be run in said iframes. Only the top-level page will be checked.

It can be used like so:

axeselenium.disableIframeTesting().run(axedriver);

Usage Service

Gain insight into axe DevTools usage trends within your organization

The usage service can be configured through enviornment variables, or methods at runtime. When both are used, the values in the methods are used. Note: not all usage service data fields can be set through methods.

By default the usage-service is disabled and the default url results are sent to is https://usage.deque.com.

Environment variables

This method allows users to change specific values of the usage service via environment variables

Enable Tracking

This method allows users to opt in or out of sending data to the usage service

.enableTracking(boolean state)

Set Tracking Url

This method allows users to change where the usage metrics data are being sent to. Defaults to https://usage.deque.com

.setTrackingUrl(String url)

Set Distinct ID

This method allows users to change the distinct id being stored/used

.setDistinctId(String distinctId)

Next Steps

Once you are writing tests with axe DevTools, read more about using results to write more detailed custom tests, or read about using the reporter to generate reports of the accessibility scans.

Troubleshooting

If you experience issues setting up your accessibility testing, contact your Deque representative directly, reach us via our support desk, or send us an email. We're happy to help get your accessibility testing efforts off the ground.