Frequently Asked Questions

Android

Why does the app need extra permissions?

When setting up the application for the first time, you'll be prompted to give axe DevTools Mobile Analyzer permission for draw overlay and accessibility settings. Draw overlay allows the floating action button to follow you no matter the app you have open on your device. Accessibility settings allows the analyzer app to access view information within the application you are attempting to scan.

Can I set up automated testing without authentication?

We offer offline builds of our SDKs and Appium drivers for your automation pipeline that do not require any network requests to the axe DevTools Mobile service. As such, these build configurations are only available through Deque's Artifactory.

Please note this build is a subset of capabilities within axeDevTools Mobile for Android. To use the full set of capabilities, including pushing results to the dashboard, please use our standard SDKs or Appium drivers. Refer to the getting started guide.

Setup

While navigating the documentation, please note that not all of the features which require server interaction are available in the offline build.

Setup for Testing

In the application build.gradle file, add:

androidTestImplementation 'com.deque.android.no-auth:axe-devtools-android:8.1.0'

and

android {
    packagingOptions {
        exclude 'META-INF/DEPENDENCIES'
        exclude 'META-INF/AL2.0' 
        exclude 'META-INF/LGPL2.1'
    }
}

Espresso Test Example

Note that the axe.loginWithUsername(...) and axe.loginWithApiKey(...) are not available within this version of the library and should be removed from the setup code if you're converting from using the authenticated version.

Layout Agnostic

@RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {
    @Rule    
    @JvmField    
    var rule: ActivityScenarioRule<MainActivity> = ActivityScenarioRule(MainActivity::class.java)

    companion object {
        private val axe = AxeDevTools()
        init {
            axe.setOfflineLicenseKey("deque_provided_license_key_here")
        }
        axe.setInstrumenation(InstrumentationRegistry.getInstrumentation())
    }

    @Test    
    fun exampleTest() {
        onView(withText("Example Button Name")).perform(click())
    }

    @After    
    fun runAccessibilityScan() {
        val scan = axe.scan()
        val result = scan?.getSerializedResult()

        axe.tearDown()   
    }
}

Why are my results not appearing in Developer Hub?

If you don't see your results in Developer Hub, verify that you have done the following:

• Provide a valid Axe DevTools Mobile API key when initializing the Axe library in your tests. Visit Axe Account Settings to find your API Keys or Generate a new Axe DevTools Mobile API key.
• Provide a valid Project ID along with the API key when starting a testing session. A Project ID is not required to start a testing session, but it is required to send results to Developer Hub. When you Create a Project for Your Results, a Project ID is automatically generated.
• Call the uploadToDashboard function after each scan in your test to send results to Developer Hub.
• See a full example. Refer to the Android Sample Test Class and compare it with your implementation.

iOS

What’s a bundle identifier?

A bundle identifier is a unique identifier within Apple’s ecosystem for application identification. No two applications can have the same identifier. This includes a beta build or other variations of an application. axe DevTools Mobile uses the bundle identifier to connect and query the views’ accessibility information of the application under test.

What does the installed Runner Application have access to?

Only the application you’ve specified to communicate with, through adding the Bundle Identifier in the setup steps. Apple is really great at security, and uses a sandbox environment for each application you have installed on your device, no matter how that application was installed (Testflight, Xcode or App Store). The analyzer application utilizes UI Test functionality built into Xcode/Apple’s ecosystem. This is a closed-box communication to talk to an application through the bundle identifier you’ve specified in the Setup file.

Common Errors & Remediation

If you get an error / Test Failed while testing, this section will highlight the common error messages and how to resolve them. Select the diamond-shaped red ‘x’ icon, and Xcode will open the ‘Issue Navigator’ panel on the left to highlight the specific error’s message.

• Cannot request screenshot data because it does not exist: This is the error message you are likely to get on the first run. Run the test again to see if it resolves. If this error appears a second time, ensure you have the correct bundle identifier in place and that the application to test is open on the selected simulator/device.

• caught error: “couldNotVerifyUser”: Login with Deque failed. Check your API key has been added to the Setup file, and is valid for axe DevTools Mobile by visiting axe Account Settings.

Can I set up for automated testing without authentication?

We offer offline builds of our SDKs and Appium drivers for your automation pipeline that do not require any network requests to the axe DevTools Mobile service. As such, these build configurations are only available through Deque's Artifactory.

Please note this build is a subset of capabilities within axeDevToolsXCUI. To use the full set of capabilities, including pushing results to the dashboard, please use our axeDevToolsXCUI framework. Refer to the setup guide.

Setup

1. Import the framework in any file used for accessibility testing.

import axeDevToolsXCUI_noauth

2. Create an object within your testing class to hold onto the axe DevTools instance:

var axeDevTools: AxeDevTools?

3. Initialize the framework within the setUp or setUpWithError methods.

axeDevTools = AxeDevTools.startSession()

Full Setup Example

import axeDevToolsXCUI_noauth
import XCTest

class MyUITests: XCTestCase {

    var axeDevTools: AxeDevTools?

    override func setUpWithError() throws {
        axeDevTools = AxeDevTools.loginWithLicenseKey("deque_provided_license_key_here") // does this change??
    }
    ...
}

UI Testing

To begin testing, pass in any XCUIElement to the framework to run accessibility tests against it and its children.

let result = try axeDevTools.run(onElement: XCUIApplication())

Full Example

import XCTest
import axeDevToolsXCUI_noauth

final class XCUI_noAuthUITest: XCTestCase {
    var axeDevTools: AxeDevTools = AxeDevTools.startSession()

    override func setUpWithError() throws {
        continueAfterFailure = false
    }

    func testExample() throws {
        let app = XCUIApplication()
        app.launch()

        let result = try axeDevTools.run(onElement: app)
        // Do something with the result
    }
}

What's Next

We give you the data and tools to create a CI/CD workflow that helps your team. Here's a few suggestions for what to do with that result object:

1. Fail the build if failures were found. If you have a screen that has already been cleared of accessibility issues, you may want to ensure no new ones creep up in the development lifecycle by asserting the count is 0, and failing the pull-request status check if it's not.

// Add an assertion to fail the build if issues were found
XCTAssertTrue(result.failures.count > 0)

2. Save the results locally to create a scoped report of the issues found in the branch. Checkout creating a report with our Reporter CLI for CICD pipelines. This can be really helpful for release-candidate or beta branches to bring awareness for what potential roadblocks for customers using assistive technology may face.

Why are my results not appearing in Developer Hub?

If you don't see your results in Developer Hub, verify that you have done the following:

• Provide a valid Axe DevTools Mobile API key when initializing the Axe framework in your tests. Visit Axe Account Settings to find your API Keys or Generate a new Axe DevTools Mobile API key.
• Provide a valid Project ID along with the API key when starting a testing session. A Project ID is not required to start a testing session, but it is required to send results to Developer Hub. When you Create a Project for Your Results, a Project ID is automatically generated.
• Call the postResult function after each scan in your test to send results to Developer Hub.
• See a full example. Refer to the iOS Sample Test Class and compare it with your implementation.

Appium

Can I set up for automated testing without authentication?

You can run a scan with our Appium offline drivers, using a license key provided by Deque. This can be useful when working with cloud providers or when you need to run scans without making network requests to the axe DevTools Mobile service.

Installation

You can install the Appium offline drivers using a private npm package from Deque's Artifactory:

Android

appium driver install --source=npm @axe-devtools/axe-appium-uiautomator2-driver-offline

iOS

appium driver install --source=npm @axe-devtools/axe-appium-xcuitest-driver-offline

Alternatively, you can download it directly from Agora and install it locally. Refer to our documentation on Setup for Private Cloud and On Prem for more details.

License Key

You will need a license key from Deque to use this offline version of the Appium driver. Please send a request to helpdesk@deque.com or at support.deque.com.

The license key will be in string format and look something like this: eyJjb21wYW55TmFtZSI6Ik1vYmlsZSBUZWFtIiwiZXhwaXJlcyI6MTcyMzk5NDk1MDY2NX0=.+aHokyifCnw6peuAmAq75IGrTjVSpkRhhfBWnf92Hp0WV3FF5Qph/KFNr7ALzi6/3K7BcSMKnelqtnwrd6mMkQ==

It's highly recommended to add this license key to your environment variables for security.

Offline Scan

To run a scan, make sure that your license key is included in axeSettings.

const axeSettings = { 
  'licenseKey': 'YOUR_LICENSE_KEY_HERE'
};

const result = await driver.execute('mobile: axeScan', axeSettings);

In axeSettings, the following optional properties are available:

• ignoreRules (default: [])
• ignoreExperimental (default: false)

The following are full examples of offline scans with our Appium drivers, using the Mocha testing framework in JavaScript.

Android

const { remote } = require('webdriverio');
const assert = require('assert');

describe('AxeScan', () => {
    let driver;
    let axeSettings;

    before(async () => {
        axeSettings = {
            //  Your license key has been stored in an environment variable for security
            licenseKey: process.env.AXE_LICENSE_KEY
        };

        driver = await remote({
            hostname: 'localhost',
            port: 4723,
            capabilities: {
                platformName: 'Android',
                'appium:automationName': 'AxeUiAutomator2',
                'appium:deviceName': 'Android',
                'appium:appPackage': 'com.android.settings',
                'appium:appActivity': '.Settings',            
            },
            logLevel: 'silent'
        });
    });

    after(async () => {
        await driver.deleteSession();
    });

    it('scan settings screen', async () => {
        // run accessibility scan
        const result = await driver.execute('mobile: axeScan', axeSettings);

        // ensure no errors were encountered during the scan
        if (result.axeError) {
            assert.fail(`AxeScan failed with error: ${result.axeError}`);
        }

        const failCount = result.axeRuleResults.filter(rule => rule.status === 'FAIL').length;

        // assert that there are no accessibility violations
        assert.strictEqual(failCount, 0);
    });
});

iOS

const { remote } = require('webdriverio');
const assert = require('assert');

describe('AxeScan', () => {
    let driver;
    let axeSettings;

    before(async () => {
        axeSettings = {
            //  Your license key has been stored in an environment variable for security
            licenseKey: process.env.AXE_LICENSE_KEY
        };

        driver = await remote({
            hostname: 'localhost',
            port: 4723,
            capabilities: {
                platformName: 'iOS',
                'appium:automationName': 'AxeXCUITest',
                'appium:bundleId': 'com.apple.Maps',
                'appium:udid': '...', // xcrun simctl list | grep Booted         
            },
            logLevel: 'silent'
        });
    });

    after(async () => {
        await driver.deleteSession();
    });

    it('scan settings screen', async () => {
        // run accessibility scan
        const result = await driver.execute('mobile: axeScan', axeSettings);

        // ensure no errors were encountered during the scan
        if (result.axeError) {
            assert.fail(`AxeScan failed with error: ${result.axeError}`);
        }

        const failCount = result.axeRuleResults.filter(rule => rule.status === 'FAIL').length;

        // assert that there are no accessibility violations
        assert.strictEqual(failCount, 0);
    });
});

Why are my results not appearing in Developer Hub?

If you don't see your results in Developer Hub, verify that you have done the following:

• Make sure you have provided a valid Axe DevTools Mobile API key when starting a testing session and initiating a scan. Visit Axe Account Settings to find your API Keys or Generate a new Axe DevTools Mobile API key.
• Make sure you have provided a valid Project ID along with the API key when starting a testing session. A Project ID is not required to start a testing session, but it is required to send results to Developer Hub. When you Create a Project for Your Results, a Project ID is automatically generated.
• See full automation examples of Axe DevTools Mobile for Appium and compare with your implementation.