How to evaluate product accessibility in a few steps
Digital accessibility is still the area, which is underrated in the design world. Most of the time it stays out of the design scope because it seems unclear or just nice to have.
So if you aim to build an accessible product you should know how to conduct accessibility evaluation.
I want to share how it can point out the weak areas of the product, and moreover how simple accessibility improvements can enhance the overall product’s usability. In this article, I describe the step-by-step process of accessibility evaluation by the case of Preply.
Why accessibility is so important
Often, when we start a new project we think of how to deliver the best user experience. We focus on specific user profiles, but sometimes, forget about the variety of users who can use digital products. Whenever I thought about accessibility, I used to think only about color contrast, font size, and element sizes.
But I knew it was only the icing on the cake. So I took an accessibility course. The course included many home tasks and lectures with online webinars, during which I learned accessibility principles, but the game-changer for me was the final project. The idea was to choose a website or mobile app which is more or less popular, with a wide range of users and conduct an accessibility evaluation. So I chose Preply. It’s an online education platform for learning mainly foreign languages. The platform is very popular and it has about 5M monthly visitors. I’m a user of the platform and for me, it works well, but how is it going to be for the blind users? Or colorblindness?
The methodology of accessibility evaluation
You are probably quite familiar with the process of usability testing. But when we speak about accessibility, and you’ve never done such evaluation, it’s difficult to come with a solid approach. WCAG can be of help here. It is a guideline which describes recommendations of how to design and implement digital products within accessibility guidelines. So following WCAG recommendations, the methodology includes 2 parts.
- Semi-automated evaluation tools. The tools automatically check a specific page and show all low contrast elements, presence of alt-text, duplicating links, empty forms titles, and the hierarchy of headers.
- Manual evaluation by experienced reviewers. The emphasis on the reviewer’s experience uncovers a lot of issues because the real audience has very specific needs. For manual evaluation, I did qualitative research with users.
Research process
Build hypotheses
Before researching and writing a script you need to know what you are going to check. I wasn’t sure if the users actually heard anything about Preply, and I didn’t know about their experience. If you don’t do user screening before, I’d recommend you to validate basic things. So I formulated these hypotheses:
- Users with vision issues would be able to navigate and use the platform to search for a tutor and schedule a lesson.
- The searching and booking processes will not create any issues for blind users.
Create the script
Usually, scripts include 3 parts.
- Warm-up. In my case, it was a small interview. it is not only an ice-breaker it also helps explore your users, their experience, along with the context, so you can thoughtfully analyze the results.
- The task part. The most difficult part for users. I organized the timeline of the testing in a way to spend around 30 min for the task itself. So users still can focus on the tasks.
- Wrap-up. Closing part, when users can ask follow-up questions, add details, or clarify anything, and nicely closing the session.
Build the task
The task was simple as possible. It was a website overview, filtering tutors by morning classes and native speakers, and booking a class with a tutor.
Recruit users
The school recruited blind users and users with weak vision. But I’d like to test the website with color blind users as well. So I asked my friends if they knew anybody with colorblind vision. I was very surprised by how many people wrote to me. It means the number of colorblind users is underrated.
6 users were recruited for accessibility testing:
- 2 blind users
- 2 color blind users
- 1 user with weak vision
- 1 user with good vision
Do a final check
Sounds simple, right? And suddenly, the night before the usability test, I thought, what if I’ll try to use a website with a screen reader? A Screen reader is an assistive technology used in devices to read the interface and control it with a keyboard. This is actually how blind users were supposed to do the task.
After trying, panic took over me. I realized it’s not easy to use the website with a screen reader, it was very difficult, I was scared to close my eyes and try to navigate only with the keyboard and Screen Reader.
Later I acquired a new habit. Now every day I turn on a Screen reader for 15–20 min, just to keep learning and get familiar with the technology. It makes me understand users much better.
Do a semi-automated check
For semi-automated evaluation, I used the online tool WAVE, if you haven’t heard about it, I would encourage you to give it a try. I recommend doing a semi-automated check before qualitative research. It allows you to find out the weak areas of the product and focus on them during the manual review.
Do user interviews
I’ve done 6 usability testings with users who have vision impairment. Based on the profiles, I’ve created 3 proto-personas. You can check their profiles
Usability testing with accessibility focus was a very new experience for me. I was so grateful the participants accepted the invitations and shared their experience, so I learned many new things.
Executive summary
Here is the link to a detailed report with the visual representation.
You will find a lot of details there, but in this section, I’ll focus on the most obvious accessibility issues.
As you see, usability testing with blind users wasn’t successful. In general, the website accessibility is on a moderate level and has several major issues. Users with disabilities should be pro-level users having proficient experience in assistive technologies like screen readers. All users reported that the pages included too much content, which might not be relevant to them. Almost all participants reported small font sizes. The users stated that some parts of web pages were accessibility appropriate because it used WAI-ARIA, the Accessible Rich Internet Applications Suite making content easily accessible for keyboard inputs.
Here is the list of areas that need improvements:
- Tutor filters don’t support keyboard navigation, which makes them difficult to use.
- When booking a lesson, it doesn’t support keyboard navigation, which is a core feature of the website.
- Many elements have low-contrast, which decreases readability for readers with weak visions.
- Form controls lack of semantic labels and error state with description.
How to improve it
In the report, you will find a lot of details, but if you try to categorize them, you can see that most of them are repetitive, so the experience can be enhanced by a few simple steps:
- Add semantic labels or descriptions to form controls. Blind people usually use screen readers to navigate through forms. They use the Tab key to move through the form controls. The <label> elements are read for each form control, and if they are not represented, screen readers skip them.
- Make the filtering process support keyboard navigation on the tutors’ list page.
- Make the booking process and hour selection support keyboard navigation.
- Enhance color contrast for inputs and text.
- Increase font size for text across the website.
A deep dive into findings
In the next sections, I’d like to share more insights from the research. Hopefully, these examples will bring you more context about accessibility evaluation. I added screenshots with problematic areas and how it can be solved with minimum changes.
Tutor list
All users reported that the page has too much content. Especially compared with the other pages. Some users didn’t get the logic behind the floating windows and expected the calendar in the floating window to be interactive.
Color blind users. All of them mentioned the low color contrast of the website. The color coding for the calendar in the floating window was very confusing because they don’t understand the meaning of highlighted hours.
Blind users. The structure of content markup doesn’t seem to be logical for blind users. Some related info didn’t wrap into groups, so screen readers read it as 2 separate pieces. Some links are redundant, and some of them are not working. For blind users, it was difficult to use the filters as they don’t support the keyboard navigation.
Weak vision users. Weak vision users reported problems with filters, because of the small font size and low contrast. A lot of content overwhelmed users during the task.
Good vision users. Didn’t report a lot of issues, aside from small fonts and a lot of content.
Below you can find the draft proposal on how the page can be changed to follow accessibility guidelines.
Tutor page
The page is quite simple and easy to read until it goes to the calendar.
Blind users. The navigation of the calendar is not very user-friendly for keyboard users as they need to go through all hours specified in the calendar to choose the needed day and time. When it comes to booking, users can’t book the selected hours with a keyboard. Using screen readers when selecting hours it doesn’t allow the user to know if it’s available or unavailable to be booked.
Weak vision and Color blind users. Users with weak vision and with color blindness had issues with next/last week buttons, as the elements are not contrasted enough. Reported small font size.
Good vision users. Users with good vision also reported small fonts
Below you can find the draft proposal on how the page can be changed to follow accessibility guidelines.
Form request
All users weren’t able to change the currency on the page, because the header hides it when scrolling.
Blind users. After redirecting to the form request page the cursor focused on the title “Request”, which is great for blind users. But it was a bit difficult to understand the meaning of some form controls, because they miss form labels, so screen readers can’t read them. The slider for price range was marked as empty forms with links, so blind users were concerned about the meaning of these forms
Weak vision and Color blind users. Users with weak vision along with color blind users reported low contrast for forms and small font size.
Good vision users. No major issues for users with good vision.
Below you can find the draft proposal on how the page can be changed to follow accessibility guidelines.
The final note
As you might notice while reading the article, it’s not so difficult to make an accessible design. Sometimes you need to sacrifice simplicity over usability reasons. But it’s worth it! As designers, we are responsible for making the digital world accessible for everyone. Of course, not all depend on us, but we can do the first step to change the perception about accessibility!
In conclusion, I want to sum up the most important things I learned during the research.
- Do not try to follow dribble trends, just because they look nice, always check color contrasts and try to think about how users will use it.
- Build your design system bearing in mind accessibility, it will save a lot of time for you in the future! When the company finally decides to make the product accessible you spend less time!
- Always test your product with assistive technology, at least the main functionality must work! Keep a good habit to check that all forms, buttons, and inputs must have titles.
- Always try to test your product with users who have disabilities. It will give you much more insights than you expect!
The story about accessibility is not about hypothetical users, it’s about us, about users we meet and interact with every day.
I’d like to say a special thank you to the Projector school, and our curators Slava Shestopalov and Eugene Shykiriavyi for organizing such a course, their support, help, insightful webinars, and of course for sharing their experiences!
Thank you for reading! I’m Nadiia and I’m a product designer, periodically I publish case studies and materials from the world of the design industry. If you find the article interesting and want to support me, just share the article with others!
So to stay in touch follow me on Twitter
