Write Hamcrest Tests

Using Hamcrest Matchers with axe DevTools

Prerequisites

In order to write accessibility tests and produce results, you must have already created a test file with axe DevTools and your choice of Selenium WebDriver pre-imported & pre-initialized. If you haven't done this yet, read this guide on how to do so.

A set of matchers have been provided for accessibility testing through hamcrest using Selenium WebDriver. These matchers can be used in a unit testing framework, or in custom cucumber steps.

Choosing a scanning method

As seen in the import statements, there are two accessibility scanning methods. The first, isAxeClean, will throw an exception and stop the tests if any violations are found. The second, isAuditedForAccessibility, won't stop the tests or throw an exception regardless of the accessibility level of the scanned page.

Writing Tests

For the remaining examples, the isAxeClean method is shown for brevity, but both methods work interchangeably. With your WebDriver and AxeDriver objects instantiated and initialized creating a scan is as simple as adding these two lines of code to a test method. Replace $URL with the desired URL of the page you wish to scan.

webDriver.get("$URL");
assertThat(axedriver, isAxeClean());

When testing multiple pages, you can repeat the get, scan process without having to create new instances of the drivers. In order to scan different states of a page, you can manipulate the page using the webdriver API and then run a new scan.

Additional Reference

Several chaining methods are available to modify the way the page is scanned. These methods include exclusion/inclusion clauses or scoping and modifications of accessibility standards, accessibility rules, and accessibility rulesets. For brevity, all chain methods are coupled with isAxeClean(), but they are interchangeable with isAuditedForAccessibility().

Accessibility Standards

Deque offers a chain method, according to, which can directly specify what accessibility testing standards are used by a scan.

If you aren't sure which standard is for you, read the axe-core documentation on which rules are associated with which standard, or contact us through your Deque representative or our help desk. For a full list of which rules apply to each standard, view this page on Rule Descriptions

To use a single standard, simply add the according to chain method and pass in your desired standard

assertThat(axedriver, isAxeClean().accordingTo("wcag2a"));

In addition to using one testing standard, there are two ways to scan against two or more testing standards. Both work identically.

assertThat(axedriver, isAxeClean().accordingTo("wcag2a", "section508"));
assertThat(axedriver, isAxeClean().accordingTo("wcag2a").accordingTo("section508"));

Scoping

Deque offers two chained methods to assist in scoping scans. The within() method alters the scan to only scan the page within the bounds of the selector passed to it. The excluding() method works the opposite way. It alters the scan to ignore content within the selector passed into it. The below examples all show the within() method, but the excluding() method works identically.

Most simply, the assertion is modified with one selector

assertThat(axedriver, isAxeClean().within("#selector1")); 

Compound selectors are also possible. In the below example, only elements with selector 2 within selector 1 are scanned

assertThat(axedriver, isAxeClean().within("#selector1 .selector2"));

There are three ways to accomplish multiple selection, or selecting multiple scopes to test within. In all of these examples, elements within selector 1 and selector 2.

assertThat(axedriver, isAxeClean().within("#selector1, .selector2"));
assertThat(axedriver, isAxeClean().within("#selector1", ".selector2"));
assertThat(axedriver, isAxeClean().within("#selector1").within(".selector2"));

In order to scope within an IFrame, an IFrame selector must be declared. The scope to be scanned within the IFrame is passed in as an additional parameter.

assertThat(axedriver, isAxeClean().within(new IFrameSelector("#frame1", "#selector1")));

Scoping within multiple IFrames works the same way as standard multiple selection. Simply create two or more IFrame selectors separated by commas.

assertThat(axedriver, isAxeClean().within(new IFrameSelector("#frame1", "selector1"), 
        new IFrameSelector("#frame2", "#selector2")));

It is also possible to scope within nested IFrames. This example shows scoping a scan within selector 1 of an IFrame which is inside a second IFrame.

assertThat(axedriver, isAxeClean().within(
    new IFrameSelector("#frame1",
    new IFrameSelector("#frame2", "#selector1"))));

Finally, it is possible to set multiple scopes within a page as well as within an IFame. For this to work, an element selector object is required.

assertThat(axedriver, isAxeClean().within(
    new ElementSelector("#selector1"),
    new ElementSelector("#selector2"),
    new IFrameSelector("#frame", "selector3")));

Rule Selection

The API also offers the ability to modify which rules are used in a scan. There are three ways to modify which rules are used. These are checking additional rules outside of the standard ruleset for the scan, not checking rules within the standard ruleset for the scan, and checking only those rules explicitly specified.

For a complete list of the valid rule tags you can use with these methods and what they test for, read the documentation on rules

In order to check additional rules, use the checking method. To test a single additional rule, just pass the rule label into the checking method

assertThat(axedriver, isAxeClean().checking("label"));

Similar to the other chain methods, multiple additional rules can be checked with the checking method two ways:

assertThat(axedriver, isAxeClean().checking("label", "tabindex"));
assertThat(axedriver, isAxeClean().checking("label").checking("tabindex"));

The checking method can be chained with a rule selection method as well.

assertThat(axedriver, isAxeClean().accordingTo("wcag2a").checking("tabindex"));

The option also exists to omit rules from a scan. This is possible through the skipping method. It works identically to the checking method except that it leaves rules out instead of adding them.

assertThat(axedriver, isAxeClean().skipping("label"));

It can also be used to omit multiple rules.

assertThat(axedriver, isAxeClean().skipping("label", "tabindex"));
assertThat(axedriver, isAxeClean().skipping("label").skipping("tabindex));

Finally, it can be used with a rule selection method as well.

assertThat(axedriver, isAxeClean().accordingTo("wcag2a").skipping("#label"));

The last method is used when only specific rules are wanted. The checking only method modifies the scan to check only those rules explicitly passed into it. Additionally, this method will not chain with the accessibility standards selection method. To check only a single rule:

assertThat(axedriver, isAxeClean().checkingOnly("label"));

This method can also be used to check multiple rules in two ways.

assertThat(axedriver, isAxeClean().checkingOnly("label", "tabindex"));
assertThat(axedriver, isAxeClean().checkingOnly("label").checkingOnly("tabindex"));

For a complete list of the valid rule tags you can use with this methods and what they test for, read the documentation on rules.

Ignoring Violations

It is possible to configure the scan to ignore specific instances of violations. In order to do this, the ignoring method is used. It works by passing the selector(s) of the element(s) you want to ignore as an array, coupled with the rule they violate.

assertThat(axedriver, isAxeClean().ignoring("[\".sidebar\", \"#branded-content\"]", "color-contrast"));

Usage Service

Gain insight into axe DevTools usage trends within your organization

By default the usage-service is disabled and the default url is https://usage.deque.com.

Environment variables

This method allows users to change specific values of the usage service via environment variables

Next Steps

For further use of your axe DevTools scan results, like reporting, see the axe DevTools Java Selenium package documentation

Troubleshooting

If you experience issues setting up your accessibility testing, contact your Deque representative directly, reach us via our support desk, or send us an email. We're happy to help get your accessibility testing efforts off the ground.