Add task checks to automated accessibility tests

An automated accessibility test can reveal recurring problems and run again after changes. Its result, however, concerns the rules, pages and states examined. The statement “no errors reported” needs that scope to support a useful decision.

Move from the report to a concrete task

Consider a fictional project overview. Suppose a scan reports no errors for the initial state. A client now needs to select a project and download the approved plan. The manual test plan for this task includes the following:

The table is a testing assignment, rather than a report of tests already performed. Depending on the tool, automated checks can support parts of it. The initial report showing no issues does not alone establish whether users can understand and complete the entire task.

Combine automated and manual findings

W3C WAI explains that some accessibility checks cannot be automated and that tools can produce inaccurate results. Reported issues therefore also need assessment in their specific context. For keyboard operation, W3C additionally describes the need for users to be able to leave an area they can reach; any necessary special exit methods must be understandable.

Make the next test traceable

Record the tool and configuration, page states examined, manual task and observed result together. After a correction, the affected steps are checked again. This keeps clear what was examined and what remains open. A single scan or manual task does not provide a complete assessment of the product’s accessibility.

Sources

• W3C WAI: Evaluation Tools Overview

• W3C: Understanding No Keyboard Trap