Running Tests in Parallel

Some testing frameworks run tests in parallel by default, which can affect your accessibility testing results if not accounted for. (You'll only see the last test run's accessibility results in Axe Developer Hub.) For example, although Playwright by default runs all tests in a single file sequentially, it creates workers to run individual test files in parallel. It's also possible that your organization runs its continuous integration pipeline in parallel, or you may want to run your test suite using parallel workers to speed up processing.

With parallel test runners, you need to make sure that each test runner is given the same non-null buildID because when the buildID property is null (the default), every test runner's results replace existing accessibility results for the same Git commit SHA, resulting in incomplete data in Axe Developer Hub.

When buildID is not null, parallel test runners with the same buildID can generate results that appear as a single test run in Axe Developer Hub. Each test runner should share the same non-null buildID string so that each test runner concatenates (rather than replaces) its results with the existing results for the same buildID and Git commit SHA.

To change the value of buildID, use:

• (JavaScript or TypeScript) The AxeConfiguration.buildID property
• (Java) The AxeWatcherOptions.setBuildId() method

You can usually obtain a usable buildID value from your continuous integration provider. For example, you can use these values:

• github.run_id

  Available within GitHub workflows. See github context in the GitHub documentation for more information.

• CIRCLE_PIPELINE_ID

  With CircleCI, the above value is available from your process's environment. See Built-in environment variables in the CircleCI documentation for more information.

  Other continuous integration environment variables you can use for buildID:

  Provider	Environment variable
  Bitbucket	BITBUCKET_BUILD_NUMBER
  GitLab	CI_PIPELINE_ID
  Jenkins	BUILD_NUMBER