Interpreting Report Data

Link to Interpreting Report Data copied to clipboard

The axe Auditor Executive Report is composed of several pre-defined worksheets and import-generated worksheets.

The four worksheet tabs of the Executive Report{width="600"}

The list below provides a brief description of the four worksheet tabs, each of which is linked to a separate page describing in greater detail its content and function:

  • Scope -- Basic identifying information about the assessment followed by a list of page descriptions with URLs for each page tested.
  • Dashboard -- A high level view of the assessment results, in the form of visual charts.
  • All Issues -- Generated when the assessment is finalized after the axe Auditor issue export file is imported, the All Issues worksheet will list every issue found on all of the pages that were selected for export from axe Auditor.
  • Executive Summary - A high level, but more detailed, view of the assessment results primarily in the form of data tables.
  • Issues by Page -- Issues are rolled up by Issue Type (Best Practice/User Agent/Accessibility Issues) and User Impact (Critical/Serious/Moderate/Minor) for each page under test.
  • Resources - A listing of WCAG 2.0 Success Criteria statements with W3C links; and Deque Checkpoint requirements with Deque University links to detailed information.

Information Icons

In addition to this documentation, you can take advantage of embedded comment help throughout the Executive Report. Within the Excel workbook itself, several sections display an "info" [i ] icon that, when clicked, display popup comments that provide textual descriptions to aid with your interpretation of the terminology, concepts, functions and data displayed in the report. These information buttons appear as white circles or squares containing a blue "i" against a blue background or as blue boxes containing a white "i" against a white background. The Excel keyboard shortcut to show or hide comments while in a cell is ALT + R, H.

Screen shot of the info icon button being clicked and the comment popup window text being displayed{width="700"}

Key Terms and Concepts

Web Content Accessibility Guidelines (WCAG)

The Web Content Accessibility Guidelines (WCAG) are part of a series of guidelines published by the Web Accessibility Initiative (WAI) of the World Wide Web Consortium (W3C), the main international standards organization for the World Wide Web.

::: {.note} The W3C (http://www.w3.org/TR/WCAG20/) describes WCAG this way:

"Web Content Accessibility Guidelines (WCAG) 2.0 covers a wide range of recommendations for making Web content more accessible. Following these guidelines will make content accessible to a wider range of people with disabilities, including blindness and low vision, deafness and hearing loss, learning disabilities, cognitive limitations, limited movement, speech disabilities, photosensitivity and combinations of these. Following these guidelines will also often make your Web content more usable to users in general." :::

WCAG Success Criteria

Success Criteria (SC) that are written as testable statements that are not technology-specific. For example:

  • SC 1.1.1: Non-text Content: All non-text content that is presented to the user has a text alternative that serves the equivalent purpose....
  • SC 2.1.1 Keyboard: All functionality of the content is operable through a keyboard interface without requiring specific timings for individual keystrokes....
  • SC 3.3.1 Error Identification: If an input error is automatically detected, the item that is in error is identified and the error is described to the user in text.

ALL Success Criteria are important to some set or sets of users with disabilities. However, the W3C recognized that the impact, effort, knowledge, and cost of meeting each SC are not the same. As a result, WCAG SC are divided into three groupings - Level A (lowest), Level AA and Level AAA (highest) - according to attributes such as: impact on users, effort to achieve, and impact on design and function.

This Executive Report is geared toward WCAG Level A and Level AA assessments.

WCAG 2.0 Level A

Level A conformance is the minimum level of WCAG conformance. Level A Success Criteria are essential to accessibility of web content, removing many of the largest barriers to accessibility that impact a large number of users. Techniques to satisfy Level A SC can be learned by web content developers / creators with the least investment in training and experience. Level A SC will have the least impact on the presentation, design, and/or function of the web content. However, Level A compliant web content will still have significant barriers to usability for many people.

WCAG 2.0 Level AA

For Level AA conformance, web content must satisfy all the Level A and Level AA Success Criteria. Level AA conformance is the most-often cited standard in international standards and in legal settlements. It ensures that web content is accessible to a broad array of users and will work with common assistive technologies. Techniques to satisfy Level AA SC may take more time to master (although this isn't strictly the case) and may have higher impact on presentation, design and / or function of the web content.

WCAG 2.0 Level AAA

Level AAA SC are the most specific, stringent requirements. Some are easy to comply with but may have impact on a limited set of users; others are difficult and expensive to achieve and maintain. While very few organizations set Level AAA as the standard to meet, it is worth taking a look at each of the AAA requirements to see which ones can be integrated into your accessibility standards at an acceptable level of effort and cost.

Deque Checkpoints

Deque Checkpoints are a further breakdown of each WCAG Success Criteria where failures of a Success Criteria can be logically separated, typically by content type. For example, SC 1.1.1 Non-text Criteria is further broken down into eight (8) Deque Checkpoints based on the type of image (active, informative, decorative, media, etc.).

The Deque Checkpoints are a proven methodology for accessibility requirements testing. The Checkpoint Requirements provide guidance to assist reviewers in achieving consistent and accurate test results during the accessibility assessment process. For more information on the Deque Checkpoints visit the Resources worksheet.

User Impact

Not all accessibility issues are equal. Some have a higher impact on accessibility (for example, not being able to operate a navigation menu without a mouse, or an image with no alt text) and others have a lower impact (for example, a screen reader user having to listen to "spacer.gif" every time they get to a spacer / formatting image, or a list that uses asterisk (*) instead of list markup to convey the list).

User Impact is a useful metric to use when prioritizing remediation efforts.

This Executive Report associates a default Impact level (Critical, Serious, Moderate, and Minor) with each Deque Checkpoint according to what Deque accessibility experts have determined is generally true for a particular type of accessibility issue. (The impact levels are described in more detail on the Dashboard page.) Assessors may use their judgment to change the default Impact. Both Critical and Serious issues are considered "High Risk" with greater effect on usability and greater vulnerability to legal action.

Checkpoint Status

For each Deque Checkpoint, an assessor will indicate the status of the Checkpoint (No Result, Pass, Fail, or Not Applicable). By default the Status is No Result for each Checkpoint, and the assessor changes the value as appropriate during the assessment.

[]{#bestpractice}Best Practice

An accessibility issue that does not fail a WCAG Success Criteria / Checkpoint Requirement but does impact user experience is labelled a Best Practice. Many times, Best Practices are small efforts that can greatly improve the user experience.

[]{#useragent}User Agent Issue

An accessibility issue that is deemed to be caused by a user agent (e.g. browser or Assistive Technology) bug - not a failure to code to WCAG standards -- is considered a User Agent Issue. For example, VoiceOver does not recognize the WCAG-approved headers + ID method of marking up a complex table. It may important to document the cause and impact of a user agent issue in case a complaint is received or to alert the project owner that an alternative approach that does not trigger known user agent issues should be considered.

Subtopics

[[pdoMenu]]

Is this page helpful?