Accessible Code Quality

Thus spake the Master Programmer:

Though a program be but three lines long, someday it will have to be maintained.

— Geoffrey James, The Tao of Programming

Every project that is more than a few lines long should implement automated testing to ensure code quality. This is especially true when it comes to the accessibility features. When a developer adds accessibility features to code, another developer may want to change that code months later and, in doing so, may accidentally remove those accessibility features.

In order to prevent this from happening in Enable, we have implemented the following automated testing frameworks inside of Enable:

v.Nu

Before testing anything else, it is important that the HTML of the project you are working on is valid. If a developer produces invalid HTML, a browser's accessibility API may not have the right information for screen readers and other assistive technology to work with the page correctly.

Enable uses v.Nu to check the HTML of all the pages within Enable. It does this by:

  1. Generating all the HTML for all the PHP pages on the site.
  2. Separating pages that initialize instantly (let's call this group "A") with ones that need a bit more processing time due to JavaScript use (let's call this group "B").
  3. Parsing the group A pages with v.Nu, using one call to v.Nu (since each call to the v.Nu command line tool would be a separate call to Java, which is expensive).
  4. Parsing the group B pages with v.Nu (each page requires a separate call to v.Nu, and thus Java).

Note that v.Nu requires Java in order to run. If this is a concern for your project, you may want to try using this Node based HTML validator instead (I have not used it yet, so your mileage may vary).

Using Axe-core and Pa11y CI for Accessibility Linting

Enable uses both Deque Labs' @axe-core/cli as well as Pa11y CI to do accessibility linting. Why two? Both are very good tools, but they don't test for the same things, and as Craig Abbott states in this excellent article that compares axe-core and pa11y, it's hard to compare the two. So why not just use both?

The problem with using axe-core compared to Pa11y-CI is that axe-core requires Chromedriver in order to work (axe-core will run pages in a headless version of Chrome to ensure the accessibility markup works, including any JavaScript generated markup). I have personally had problems with Chromedriver updates (here is one of the issues I had in the past). Pa11y, on the other hand, uses Puppeteer to launch Chrome and do its tests. You can read about how these two technologies differ, but from my experience, it seems Chromedriver updates are more likely to break things more often than Puppeteer updates. You have been warned.

Both tools go through all the Enable pages to check to see if color contrast is right, alt attributes are set, ARIA is marked up correctly, and so on. As axe-core explicitly states after execution, automated testing can only catch 20% to 50% of accessibility issues. Is there any way to improve upon that?

Unit Testing

Unit testing is the final tool in your automated testing toolkit that you should use in your project to ensure any accessibility feature you have just implemented stays within the project. For example, if you create a custom accessible listbox dropdown, you want to make sure that when keyboard users tab into the component and use the arrow keys, they can change the selected listbox value.

Enable currently uses Jest with Puppeteer to do unit tests. Usually, each test involves:

  1. Loading a page that contains component examples
  2. Querying the DOM on the page to make sure the components in question are coded correctly.
  3. Querying the current CSS style in the components to make sure it captures the visual requirements (and/or screen reader contents, when using visually-hidden CSS generated content)
  4. If needed, simulate a keyboard user manipulating the components to ensure the user-experience works correctly.
  5. After the component is manipulated, go through steps 2-5 again, if necessary.

Before we start

Our unit testing examples use ES6 modules. In order to support ES6 Modules in Jest, you need to do the following commands inside your project:

☜ Scroll to read full source ☞

                    
                
Figure 1. NPM commands to install in order to use ES6 Modules in Jest.

You should also put the following lines in your .babelrc:

☜ Scroll to read full source ☞

                    
                
Figure 2. What to put in .babelrc in order for Jest to transform the ES6 modules with babel.

A Simple Example: Having Screen Readers Read Strikethrough Text

Let's look at a simple example that just involves steps 1 through 3. If you look at the page for Exposing Style Information to Screen Readers, we use visually-hidden content inside of mark tags. We want to ensure that a new developer that contributes code to Enable never removes this screen reader only text by accident, so we create a jest test file, exposing-style-info-to-screen-readers.test.js, to ensure we can test that this CSS is in these examples. Let's walk through this file to show how it works.

Code Walkthrough of the Above Example

Below is the HTML of the above example. Use the dropdown to highlight each of the individual steps that make the example accessible.

☜ Scroll to read full source ☞

                    
                

A Simple Interactive Example: Switches/Toggles

This example is used to test Enable's switch component to ensure that it is keyboard accessible and that the HTML structure includes all the necessary accessibility features (e.g., the role="switch", a valid aria-checked attribute set, a proper label, etc.). Please go through the code walkthrough below for more details.

Code Walkthrough of the Above Example

Below is the HTML of the above example. Use the dropdown to highlight each of the individual steps that make the example accessible.

☜ Scroll to read full source ☞

                    
                

A More Complex Example: Testing Focus States on Multiple Pages

This is an example of a test that ensures all interactive elements on all the pages within Enable have a focus state (in our case, using a CSS `outline`). Note that we ignore iframe, video, and body tags in this test because of the tests giving false negatives (which we are actively looking into to fix).

Code Walkthrough of the Above Example

Below is the HTML of the above example. Use the dropdown to highlight each of the individual steps that make the example accessible.

☜ Scroll to read full source ☞

                    
                

Using GitHub Actions to Run the Tests in the CI/CD Pipeline

The best place to start is by reading the documentation inside the GitHub Actions: Prevent Merge page on the Calmcode website. Once you go through the set up of GitHub Actions on your project, the page describes writing a .yml file that contains the GitHub Actions steps that you'd like to enforce. The .yml file that we used for Enable looks like this (the highlighted section is the part we wrote to set up the unit testing and automated testing):

☜ Scroll to read full source ☞

                    
                

After you have set up your project with your own rules, do the last few steps to ensure your project is set up correctly:

  1. In your github project, go to settings, located as the last item in the main navigation of your github project (note that it may be under the ellipsis button named "Additional Navigation Items").
  2. Expand the "Actions" collapsable button in the menu of the left hand side of the settings page and click the "General" link. Inside the Actions permissions radio group, choose the "Allow all actions and reusable workflows" radio button and save.

Thanks to Alison Hall for her work in setting up Enable with Github Actions.

Further Reading

If you want to do some further reading, we recommend Writing Automated Tests for Accessibility by Marcie Sutton and Web Accessibility Testing by Kfir Zuberi. They are great places to start (it's where we started).