Frequently Asked Questions

Android

Why does the app need extra permissions?

When setting up the application for the first time, you'll be prompted to give axe DevTools Mobile Analyzer permission for draw overlay and accessibility settings. Draw overlay allows the floating action button to follow you no matter the app you have open on your device. Accessibility settings allows the analyzer app to access view information within the application you are attempting to scan.

Can I set up automated testing without authentication?

The optimized build for your automation pipeline does not require any network requests to the axe DevTools Mobile service. As such, these build configurations are only available through Deque's Artifactory.

Please note this build is a subset of capabilities within axeDevTools Mobile for Android. To use the full set of capabilities, including pushing results to the dashboard, please use our full framework. Refer to the getting started guide.

Setup

While navigating the documentation, please note that not all of the features which require server interaction are available in this optimized build.

Setup for Testing

In the application build.gradle file, add:

androidTestImplementation 'com.deque.android.no-auth:axe-devtools-android:6.2.0'

and

android {
    packagingOptions {
        exclude 'META-INF/DEPENDENCIES'
        exclude 'META-INF/AL2.0' 
        exclude 'META-INF/LGPL2.1'
    }
}

Espresso Test Example

Note that the axe.loginWithUsername(...) and axe.loginWithApiKey(...) are not available within this version of the library and should be removed from the setup code if you're converting from using the authenticated version.

Layout Agnostic

@RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {
    @Rule    
    @JvmField    
    var rule: ActivityScenarioRule<MainActivity> = ActivityScenarioRule(MainActivity::class.java)

    companion object {
        private val axe = AxeDevTools()
        axe.setInstrumenation(InstrumentationRegistry.getInstrumentation())
    }

    @Test    
    fun exampleTest() {
        onView(withText("Example Button Name")).perform(click())
    }

    @After    
    fun runAccessibilityScan() {
        val scan = axe.scan()
        val result = scan?.getSerializedResult()

        axe.tearDown()   
    }
}

XML (Deprecated)

@RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {
    @Rule    
    @JvmField    
    var rule: ActivityScenarioRule<MainActivity> = ActivityScenarioRule(MainActivity::class.java)

    companion object {
        private val axe = AxeDevTools()
    }

    @Test    
    fun exampleTest() {
        onView(withText("Example Button Name")).perform(click())
    }

    @After    
    fun runAccessibilityScan() {
        rule.scenario.onActivity {            
            val scan = axe.scan(it)
            val result = scan?.getSerializedResult()

            result?.axeRuleResults?.forEach { res ->     
                assertEquals(AxeStatus.PASS, res.status)
            }            
            axe.tearDown()
        }    
    }
}

Compose (Deprecated)

class ExampleInstrumentedComposeTest {
    @get:Rule
    val composeTestRule = createAndroidComposeRule<TestComposeActivity>()

    companion object {
        private val axeCompose = AxeDevToolsCompose()
    }

    @Before
    fun setup() {
        composeTestRule.activity.setContent { ComposableFunction() }
    }

    @Test
    fun runAccessibilityScan() {
        axeCompose.setComposeTestRule(composeTestRule)
        val result = axeCompose.scan()?.getSerializedResult()

        result?.axeRuleResults?.forEach { res ->     
            assertEquals(AxeStatus.PASS, res.status)
        }     
    }
}

iOS

What’s a bundle identifier?

A bundle identifier is a unique identifier within Apple’s ecosystem for application identification. No two applications can have the same identifier. This includes a beta build or other variations of an application. axe DevTools Mobile uses the bundle identifier to connect and query the views’ accessibility information of the application under test.

What does the installed Runner Application have access to?

Only the application you’ve specified to communicate with, through adding the Bundle Identifier in the setup steps. Apple is really great at security, and uses a sandbox environment for each application you have installed on your device, no matter how that application was installed (Testflight, Xcode or App Store). The analyzer application utilizes UI Test functionality built into Xcode/Apple’s ecosystem. This is a closed-box communication to talk to an application through the bundle identifier you’ve specified in the Setup file.

Common Errors & Remediation

If you get an error / Test Failed while testing, this section will highlight the common error messages and how to resolve them. Select the diamond-shaped red ‘x’ icon, and Xcode will open the ‘Issue Navigator’ panel on the left to highlight the specific error’s message.

• Cannot request screenshot data because it does not exist: This is the error message you are likely to get on the first run. Run the test again to see if it resolves. If this error appears a second time, ensure you have the correct bundle identifier in place and that the application to test is open on the selected simulator/device.

• caught error: “couldNotVerifyUser”: Login with Deque failed. Check your API key has been added to the Setup file, and is valid for axe DevTools Mobile by visiting axe Account Settings.

Can I set up for automated testing without authentication?

The optimized build for your automation pipeline does not require any network requests to the axe DevTools Mobile service. As such, these build configurations are only available through Deque's Artifactory.

Please note this build is a subset of capabilities within axeDevToolsXCUI. To use the full set of capabilities, including pushing results to the dashboard, please use our axeDevToolsXCUI framework. Refer to the setup guide.

Setup

1. Import the framework in any file used for accessibility testing.

import axeDevToolsXCUI_noauth

2. Create an object within your testing class to hold onto the axe DevTools instance:

var axeDevTools: AxeDevTools?

3. Initialize the framework within the setUp or setUpWithError methods.

axeDevTools = AxeDevTools.login()

Full Setup Example

import axeDevToolsXCUI_noauth
import XCTest

class MyUITests: XCTestCase {

    var axeDevTools: AxeDevTools?

    override func setUpWithError() throws {
        axeDevTools = AxeDevTools.login()
    }
    ...
}

UI Testing

To begin testing, pass in any XCUIElement to the framework to run accessibility tests against it and its children.

let result = try axeDevTools.run(onElement: XCUIApplication())

Full Example

import XCTest
import axeDevToolsXCUI_noauth

final class XCUI_noAuthUITest: XCTestCase {
    var axeDevTools: AxeDevTools = AxeDevTools.login()

    override func setUpWithError() throws {
        continueAfterFailure = false
    }

    func testExample() throws {
        let app = XCUIApplication()
        app.launch()

        let result = try axeDevTools.run(onElement: app)
        // Do something with the result
    }
}

What's Next

We give you the data and tools to create a CI/CD workflow that helps your team. Here's a few suggestions for what to do with that result object:

1. Fail the build if failures were found. If you have a screen that has already been cleared of accessibility issues, you may want to ensure no new ones creep up in the development lifecycle by asserting the count is 0, and failing the pull-request status check if it's not.

// Add an assertion to fail the build if issues were found
XCTAssertTrue(result.failures.count > 0)

2. Save the results locally to create a scoped report of the issues found in the branch. Checkout creating a report with our Reporter CLI for CICD pipelines. This can be really helpful for release-candidate or beta branches to bring awareness for what potential roadblocks for customers using assistive technology may face.