When testing a web site for compliance with accessibility guidelines or regulations, it is hard to know when enough pages have been tested. This is a problem for anyone who wants to ensure a site is accessible.
Through a systematic analysis of different approaches to sampling a web site, the authors are working towards a way of determining the optimal number of pages to test for confidence that the evaluation is accurate. This paper is a report on early results from their research.
More details
The authors conclude that:
- Testing 13 specifically selected pages reveals a mean of 93 % of all unique accessibility problems, but there is wide variation among sites. For 68% of the websites, no new guideline violations were found after the first 13 pages
- A sample of 13 specific and 10 random pages is enough to find 99% of the unique accessibility problems in a website. For 92% percent of the websites, no new guideline violations were found after testing all 23 pages.
Links
Velleman, E. and van der Geest, T. (2013) Page Sample Size in Web Accessibility Testing: How Many Pages Is Enough. ACM Assets Conference’13, October, 2013, Bellevue, Washington, US doi: 10.1145/2513383.2513408