Incorporating Web Accessibility into a CI/CD Pipeline : A study of Open Source Evaluation Tools

Sammanfattning

With an ever growing digital landscape, ensuring that online content is accessible for everyone is a must. By adhering to guidelines like the Web Content Accessibility Guidelines (WCAG), developers can ensure their applications meet established standards in the field of accessibility. However, verifying compliance with these guidelines can be a tedious task. A promising solution is the use of Web Accessibility evaluation tools within a continuous integration (CI) and continuous deployment (CD) environment. This study explores a selection of open-source Web Accessibility evaluation tools and investigates how easily they can be integrated into a CI/CD environment. Further, the study examines the effectiveness of such tools in detecting violations of the WCAG within a CI/CD environment. The tools selected for this study are: Asqatasun, Deque Systems’ Axe-Core CLI, Squiz Labs’ HTML_CodeSniffer, and IBM’s Equal Access Accessibility Checker. In order to conduct our evaluation of the tools, a testing website with a specific number of WCAG violations was developed. The implementation details and code for this project are discussed in Chapter 3 and are available on GitHub. Given that the source code for this study is stored in a GitHub repository, GitHub Actions was chosen for testing the tools within a CI/CD environment. The tools performance was assessed based on metrics measuring the accuracy of error identification, error positioning, and error description. The results indicate that the selected Web Accessibility evaluation tools detect approximately 20% of all errors on a given web page on average. This finding suggests that while these tools can support development teams in identifying accessibility issues on web pages, manual testing is still required to ensure comprehensive Web Accessibility compliance.