Change the Accessibility (A11y) Threshold
How to change the a11y threshold to customize how your organization treats accessibility issues
This article discusses the accessibility (a11y) threshold and how to change the criteria axe Developer Hub uses for calculating the a11y threshold.
What Is the A11y Threshold?
The a11y threshold reflects your organization's tolerance for accessibility errors and is a number that is computed from two criteria:
- Whether the computation uses all accessibility issues found by axe Developer Hub during your test runs or only new issues discovered in the latest test run.
- Which impacts (critical, serious, moderate, and minor) will be included in the threshold calculation.
If you change any of the criteria for the a11y threshold, the change only affects calculations going forward and not past calculations; older a11y threshold values are not recalculated. An icon will appear next to the calculation to indicate that the configuration has changed, and if you hover your cursor over it, you will be reminded that the threshold has changed:
The a11y threshold allows you to customize axe Developer Hub for what matters to your organization:
- Are new issues your organization's main concern, or do you want to track all issues?
- What impacts matter to your organization?
For example, maybe you are only interested in mitigating critical issues, or maybe you are only interested in addressing both critical and serious issues. Maybe your organization is just starting out addressing accessibility issues, so you need to limit the impacts to avoid being overwhelmed. Later, as your organization's knowledge of mitigating accessibility issues increases, you can add serious impacts (or moderate or minor).
How Is the A11y Threshold Used?
The a11y threshold provides a quick gauge of the accessibility quality of your project and appears prominently throughout the axe Developer Hub site. Any value over zero exceeds the allowed accessibility threshold and indicates issues that need to be addressed. Over time you can track the accessibility threshold from commit to commit and release to release to develop a picture of the accessibility health of your project.
The following pages show the a11y threshold:
- Branches page
- Commits page
- Test run page
- Issues page
The following example shows how the calculated a11y threshold is shown in the left section of each panel (here, there are two issues over the threshold):
Clicking on the a11y threshold value (here, 2) causes the impacts to be added to the filter, so you can drill down to only the issues that caused the a11y threshold to be exceeded. (In this case, clicking on 2 added all impacts to the filter because all were set in the a11y configuration.)
How to Change
On your axe Account page click on the CONFIGURATION tab at the top of the page:
You will see the Configuration page. Select axe Developer Hub from the list on the left side of the page to see the a11y threshold configuration:
Individual account holders can change the a11y threshold criteria, but if you are part of an enterprise, you must be an enterprise administrator to change them.
Changing the a11y threshold as an enterprise administrator will change it for all users of the enterprise.
After you've changed your a11y threshold settings, click SAVE.
You cannot deselect critical from the list of impacts because critical issues result in blocked content for people with disabilities and will prevent them from accessing fundamental features or content.
Issues Included
The Issues Included value indicates whether to include in the threshold calculation
- all accessibility issues that meet the selected impacts or
- only the new ones.
Impacts Included
The impacts included value indicates which impact level or levels to include in the calculation.
In the present implementation, the a11y threshold settings are completely independent from the global accessibility settings (on the Global tab on the Configuration page) (but this may change in the future). Changing the settings in the Global tab will not change the a11y threshold (or vice versa).
Using the A11y Threshold With the GitHub Action
You can use the a11y threshold with the axe Developer Hub GitHub action to act as a gatekeeper to block, by default, pull requests that contain accessibility issues that exceed the a11y threshold. The action will add a comment to the pull request indicating the problem:
In the above example PR comment, the link directs you to the Issues page on the axe Developer Hub website to see the issues that failed this PR. You can also change the default behavior to only warn users of accessibility issues instead of blocking the PR from being merged.
How It Works
The GitHub action queries the axe Developer Hub server (via its REST endpoint) for the latest commit associated with this pull request (PR). The web service returns the number of accessibility issues as determined by the a11y threshold settings. By default, any number of accessibility errors above zero is considered a failure, which causes a comment such as above to be added to the pull request and the pull request to be blocked from being merged.
See Using the GitHub Action for more information.