SlideShare a Scribd company logo

Presentation: Why Usability Testing Should be Part of your Accessibility Testing Strategy

•Download as PPTX, PDF•
•
User Experience Center, Bentley University
User Experience Center, Bentley University

Pete McNally, Senior Consultant at User Experience Center receives the Best Paper award at the recent ICT 2017 ICT Accessibility Testing Symposium in Washington, DC.

1 of 47
Why Usability Testing Should be Part of your Accessibility Testing Strategy The 2017 ICT Accessibility Testing Symposium: Automated & Manual Testing, WCAG 2.1, and Beyond October 26, 2017 Peter McNally Email: pmcnally@Bentley.edu Twitter: @pmcnallyux User Experience Center Bentley University Waltham, MA 02452 USA
Table of Contents 2 Section Slide Opening 3 Intro to Usability Test 13 Real World Examples 19 Benefits of Usability Testing 26 Recommended Approaches 39 Moving Forward 43
3 Opening
Question Can anyone think of any examples of products or “things” from everyday life that you have access to, but are hard to use? 4
Doors 5 • Is this door accessible?
Doors 6 • Is this door accessible?
Doors 7 • Is this door accessible? • Is this door usable?
Doors 8 • Is this door accessible? • Is this door usable?
Doors - Video • Norman doors • https://www.youtube.com/watch?v= yY96hTb8WgI 9
Doors • What do doors have to do with usability testing or accessibility testing? • You can access door by using the handle or pressing the automatic opener button. • However, as we have seen the door may not be usable. 10
Design of Everyday Things by Don Norman 11
Goal • Discuss the benefits of usability testing in conjunction with accessibility testing and how they can work together. • Review some recent usability testing projects where we had people with disabilities participate. • Demonstrate how these findings can augment and support accessibility testing efforts. 12
13 Introduction to Usability Testing
Usability Testing Overview 14 When the process of refining is complete, the next step is to implement the changes live. • User Experience (UX) method. • Ask an individual to try and use something without instructions. • Can be performed on anything that a human has to use. • including digital products such as a web site or a mobile app.
Usability Testing Overview 15 When the process of refining is complete, the next step is to implement the changes live. • Most common one used in the UX industry follows a qualitative approach: • 5-12 participants tested over 1 or 2 days. • Each session lasts 30 to 90 minutes. • Participants given multiple tasks to accomplish during the session. • Tasks types may range between open ended and specific. • Typical to identify many common issues. • Provide design recommendations
Usability Testing Tasks 16 When the process of refining is complete, the next step is to implement the changes live. • Examples of usability testing tasks Completely Open Ended Example Product Type Example Task Electronic component mobile app Let’s say you are working on a project and need some information about a product related to what you described earlier. How would you go about learning more about it from the app?
Usability Testing Tasks 17 When the process of refining is complete, the next step is to implement the changes live. • Examples of usability testing tasks Less Open Ended Example Product Type Example Task Clothing web site You saw a friend’s daughter wearing this Summer Green Surf Floral Swimsuit and are considering getting one for your daughter. Please, find it and add it to your cart
Usability Testing Participants 18 When the process of refining is complete, the next step is to implement the changes live. • Each participant conducts a test session one-on-one with a moderator. • Participants are asked to think-aloud. • The goal is to see if people can use the product “cold” as w/ consumer web sites • Participants should have a desire, need or goal in using the product tested. • Anyone fitting this definition including people with disabilities can participate.
19 Real World Examples
User Experience Center (UXC) 20 • Consulting group • Provides user experience research, evaluation, and design services • Goal of understanding and improving the experiences our clients provide to their customers.
21 UXC Experience 75yrs Rich & diverse collective team experience in design and UX research A breadth of methodologies tailored to Client’s market needs to improve customer experience Integration with award- winning graduate program in Human Factors in Information Design
4 Projects - Participants 22 When the process of refining is complete, the next step is to implement the changes live. • Large well known US companies. • Requested participants with disabilities. Industry/ Disability Blind Low Vision Color Blind Motor Learning/ Cognitive Total Banking 4 1 3 2 10 Publishing 5 3 8 Chemical 4 1 1 4 10 Pharmaceutical 1 1 3 4 9 Total 14 6 1 6 10 37
Issues Found 23 When the process of refining is complete, the next step is to implement the changes live. • Found both usability and accessibility issues • 43% - Accessibility issues = that should have been found in a thorough accessibility review (WCAG 2.0). • 57% - Usability issues = not accessibility related, but have a negative impact on the experience. • Most interesting are usability issues that are more likely to be found in context of usability testing with people with disabilities
Issues Found 24 When the process of refining is complete, the next step is to implement the changes live. Issue Category Accessibility Usability Total Content/readability/Page Layout 12 26 38 Search 21 11 32 Forms/interactive pages 16 10 26 Process/Understanding Concepts 0 19 19 Navigation/Menus, Orientation 7 10 17 Registration/Log in/Captcha 5 7 12 Error handling/Required fields 5 4 9 General/Other 5 3 8 Information Architecture/Site Structure 0 4 4 Total 71 (43%) 94(57%) 165
Issues Found 25 When the process of refining is complete, the next step is to implement the changes live.
26 Benefits of Usability Testing
Benefits of Usability Testing 27 When the process of refining is complete, the next step is to implement the changes live. • Usability Testing tasks can focus on the end-to-end process (big picture) • Focused on user goals • May or may not line up with accessibility issues. • Accessible does not necessarily mean usable.
Focusing Resources 28 When the process of refining is complete, the next step is to implement the changes live. • Focusing design and development resources. • Ideally, all accessibility should be fixed. • Prioritize resources around what blocks users from accomplishing their goals.
Context of Use 29 When the process of refining is complete, the next step is to implement the changes live. • Discovery of problems in context of use. • Usability testing can identify problems that may not be identified during accessibility testing. • A real user may do things very differently than in any test case. • Talking to users may give you insights into their motivation and allow you to ask “why” questions.
Context of Use – Example 1 30 When the process of refining is complete, the next step is to implement the changes live. • Bank site with page for applicants with disabilities. • Assumed “Applicants with Disabilities” page would be customized for PWD • “Oh no, really?! …I thought I was going to go to a page that would have a screen reader friendly page or TTY number for people who are deaf…feels like I was tricked” • bank web site usability testing participant.
Context of Use – Example 1 31 When the process of refining is complete, the next step is to implement the changes live. • Bank site with page for applicants with disabilities.
Context of Use – Example 2 32 When the process of refining is complete, the next step is to implement the changes live. • Chemical Company Job Description • Some participants using screen readers missed some the job description because when they heard the “About <Company>” they assumed there were at the end of the job description and did not continue on • Therefore missing important information about company benefits
Context of Use – Example 2 33 When the process of refining is complete, the next step is to implement the changes live. • Chemical Company Job Description
Context of Use – Example 2 34 When the process of refining is complete, the next step is to implement the changes live. • Chemical Company Job Description. • Video clip with Joe.
Learning/Cognitive Disabilities 35 When the process of refining is complete, the next step is to implement the changes live. • Many issues that are considered usability related deal with learning and cognition. • Present more challenges for individuals with learning and cognitive disabilities. • Accessibility guidelines such WCAG 2.0 focus more on code based issues. • Design guidelines such as Jacob Nielsen’s usability heuristics and from the Center for Plain Language are a good start
Learning/Cognitive Disabilities 36 When the process of refining is complete, the next step is to implement the changes live. • Nielsen’s Heuristics: • Visibility of system status • The system should always keep users informed about what is going on, through appropriate feedback within reasonable time. • Recognition rather than recall • Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
Getting the Word Out 37 When the process of refining is complete, the next step is to implement the changes live. • Having stakeholders (clients, developers, managers, etc.) watch usability testing sessions puts a face on accessibility for some who were not aware (or didn’t care). • Inviting developers to watch usability testing can help them understand the importance of their impact for everyone.
Accessibility Program Jump Start 38 When the process of refining is complete, the next step is to implement the changes live. • Provides a good way to “jump start” an accessibility program. • A “kick in the pants”.
39 Recommended Approaches
Recommended Approaches 40 When the process of refining is complete, the next step is to implement the changes live. • Conduct accessibility reviews by using automatic and manual processes. • After the “big” issues resolved conduct usability testing with individuals with disabilities. • Can test in office, conference room (or specialized lab (but not necessary)). • Best if participants can bring in their own AT to usability test. • Tools such as Zoom offer promise of remote usability testing with screen reader users.
Recommended Approaches 41 When the process of refining is complete, the next step is to implement the changes live. • Approaches to recruiting participants • Don’t test with all experts • Participants should match your user/customer profile – domain knowledge. • Ideally part of your standard UX approach. • Work with your UX team or local disability organization. • Range of disabilities: visual, learning, motor, etc. Don’t forget older users…
Recommended Approaches 42 When the process of refining is complete, the next step is to implement the changes live. • Steve Krug – Rocket Surgery Made Easy • http://www.sen sible.com/rsme .html
43 Moving Forward
Moving Forward 44 When the process of refining is complete, the next step is to implement the changes live. • Typical to usability test after accessibility test, however… • Once coding starts UX issues are baked in. Expensive to change. • Accessibility should be integrated into entire design and development process. • Developing user personas and reviewing wireframes, e.g., asking how will someone only using the keyboard or a screen reader to access the primary navigation menu, etc.
Moving Forward 45 When the process of refining is complete, the next step is to implement the changes live. • Sarah Horton and Whitney Quesenbery – A Web for Everyone. • http://rosenfel dmedia.com/bo oks/a-web-for- everyone/
Moving Forward 46 When the process of refining is complete, the next step is to implement the changes live. • UX Personas with individuals with disabilities
Thank You! Questions? Peter McNally Email: pmcnally@Bentley.edu Twitter: @pmcnallyux User Experience Center Bentley University Waltham, MA 02452 USA

More Related Content

Presentation: Why Usability Testing Should be Part of your Accessibility Testing Strategy

  • 1. Why Usability Testing Should be Part of your Accessibility Testing Strategy The 2017 ICT Accessibility Testing Symposium: Automated & Manual Testing, WCAG 2.1, and Beyond October 26, 2017 Peter McNally Email: pmcnally@Bentley.edu Twitter: @pmcnallyux User Experience Center Bentley University Waltham, MA 02452 USA
  • 2. Table of Contents 2 Section Slide Opening 3 Intro to Usability Test 13 Real World Examples 19 Benefits of Usability Testing 26 Recommended Approaches 39 Moving Forward 43
  • 4. Question Can anyone think of any examples of products or “things” from everyday life that you have access to, but are hard to use? 4
  • 5. Doors 5 • Is this door accessible?
  • 6. Doors 6 • Is this door accessible?
  • 7. Doors 7 • Is this door accessible? • Is this door usable?
  • 8. Doors 8 • Is this door accessible? • Is this door usable?
  • 9. Doors - Video • Norman doors • https://www.youtube.com/watch?v= yY96hTb8WgI 9
  • 10. Doors • What do doors have to do with usability testing or accessibility testing? • You can access door by using the handle or pressing the automatic opener button. • However, as we have seen the door may not be usable. 10
  • 11. Design of Everyday Things by Don Norman 11
  • 12. Goal • Discuss the benefits of usability testing in conjunction with accessibility testing and how they can work together. • Review some recent usability testing projects where we had people with disabilities participate. • Demonstrate how these findings can augment and support accessibility testing efforts. 12
  • 14. Usability Testing Overview 14 When the process of refining is complete, the next step is to implement the changes live. • User Experience (UX) method. • Ask an individual to try and use something without instructions. • Can be performed on anything that a human has to use. • including digital products such as a web site or a mobile app.
  • 15. Usability Testing Overview 15 When the process of refining is complete, the next step is to implement the changes live. • Most common one used in the UX industry follows a qualitative approach: • 5-12 participants tested over 1 or 2 days. • Each session lasts 30 to 90 minutes. • Participants given multiple tasks to accomplish during the session. • Tasks types may range between open ended and specific. • Typical to identify many common issues. • Provide design recommendations
  • 16. Usability Testing Tasks 16 When the process of refining is complete, the next step is to implement the changes live. • Examples of usability testing tasks Completely Open Ended Example Product Type Example Task Electronic component mobile app Let’s say you are working on a project and need some information about a product related to what you described earlier. How would you go about learning more about it from the app?
  • 17. Usability Testing Tasks 17 When the process of refining is complete, the next step is to implement the changes live. • Examples of usability testing tasks Less Open Ended Example Product Type Example Task Clothing web site You saw a friend’s daughter wearing this Summer Green Surf Floral Swimsuit and are considering getting one for your daughter. Please, find it and add it to your cart
  • 18. Usability Testing Participants 18 When the process of refining is complete, the next step is to implement the changes live. • Each participant conducts a test session one-on-one with a moderator. • Participants are asked to think-aloud. • The goal is to see if people can use the product “cold” as w/ consumer web sites • Participants should have a desire, need or goal in using the product tested. • Anyone fitting this definition including people with disabilities can participate.
  • 20. User Experience Center (UXC) 20 • Consulting group • Provides user experience research, evaluation, and design services • Goal of understanding and improving the experiences our clients provide to their customers.
  • 21. 21 UXC Experience 75yrs Rich & diverse collective team experience in design and UX research A breadth of methodologies tailored to Client’s market needs to improve customer experience Integration with award- winning graduate program in Human Factors in Information Design
  • 22. 4 Projects - Participants 22 When the process of refining is complete, the next step is to implement the changes live. • Large well known US companies. • Requested participants with disabilities. Industry/ Disability Blind Low Vision Color Blind Motor Learning/ Cognitive Total Banking 4 1 3 2 10 Publishing 5 3 8 Chemical 4 1 1 4 10 Pharmaceutical 1 1 3 4 9 Total 14 6 1 6 10 37
  • 23. Issues Found 23 When the process of refining is complete, the next step is to implement the changes live. • Found both usability and accessibility issues • 43% - Accessibility issues = that should have been found in a thorough accessibility review (WCAG 2.0). • 57% - Usability issues = not accessibility related, but have a negative impact on the experience. • Most interesting are usability issues that are more likely to be found in context of usability testing with people with disabilities
  • 24. Issues Found 24 When the process of refining is complete, the next step is to implement the changes live. Issue Category Accessibility Usability Total Content/readability/Page Layout 12 26 38 Search 21 11 32 Forms/interactive pages 16 10 26 Process/Understanding Concepts 0 19 19 Navigation/Menus, Orientation 7 10 17 Registration/Log in/Captcha 5 7 12 Error handling/Required fields 5 4 9 General/Other 5 3 8 Information Architecture/Site Structure 0 4 4 Total 71 (43%) 94(57%) 165
  • 25. Issues Found 25 When the process of refining is complete, the next step is to implement the changes live.
  • 27. Benefits of Usability Testing 27 When the process of refining is complete, the next step is to implement the changes live. • Usability Testing tasks can focus on the end-to-end process (big picture) • Focused on user goals • May or may not line up with accessibility issues. • Accessible does not necessarily mean usable.
  • 28. Focusing Resources 28 When the process of refining is complete, the next step is to implement the changes live. • Focusing design and development resources. • Ideally, all accessibility should be fixed. • Prioritize resources around what blocks users from accomplishing their goals.
  • 29. Context of Use 29 When the process of refining is complete, the next step is to implement the changes live. • Discovery of problems in context of use. • Usability testing can identify problems that may not be identified during accessibility testing. • A real user may do things very differently than in any test case. • Talking to users may give you insights into their motivation and allow you to ask “why” questions.
  • 30. Context of Use – Example 1 30 When the process of refining is complete, the next step is to implement the changes live. • Bank site with page for applicants with disabilities. • Assumed “Applicants with Disabilities” page would be customized for PWD • “Oh no, really?! …I thought I was going to go to a page that would have a screen reader friendly page or TTY number for people who are deaf…feels like I was tricked” • bank web site usability testing participant.
  • 31. Context of Use – Example 1 31 When the process of refining is complete, the next step is to implement the changes live. • Bank site with page for applicants with disabilities.
  • 32. Context of Use – Example 2 32 When the process of refining is complete, the next step is to implement the changes live. • Chemical Company Job Description • Some participants using screen readers missed some the job description because when they heard the “About <Company>” they assumed there were at the end of the job description and did not continue on • Therefore missing important information about company benefits
  • 33. Context of Use – Example 2 33 When the process of refining is complete, the next step is to implement the changes live. • Chemical Company Job Description
  • 34. Context of Use – Example 2 34 When the process of refining is complete, the next step is to implement the changes live. • Chemical Company Job Description. • Video clip with Joe.
  • 35. Learning/Cognitive Disabilities 35 When the process of refining is complete, the next step is to implement the changes live. • Many issues that are considered usability related deal with learning and cognition. • Present more challenges for individuals with learning and cognitive disabilities. • Accessibility guidelines such WCAG 2.0 focus more on code based issues. • Design guidelines such as Jacob Nielsen’s usability heuristics and from the Center for Plain Language are a good start
  • 36. Learning/Cognitive Disabilities 36 When the process of refining is complete, the next step is to implement the changes live. • Nielsen’s Heuristics: • Visibility of system status • The system should always keep users informed about what is going on, through appropriate feedback within reasonable time. • Recognition rather than recall • Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
  • 37. Getting the Word Out 37 When the process of refining is complete, the next step is to implement the changes live. • Having stakeholders (clients, developers, managers, etc.) watch usability testing sessions puts a face on accessibility for some who were not aware (or didn’t care). • Inviting developers to watch usability testing can help them understand the importance of their impact for everyone.
  • 38. Accessibility Program Jump Start 38 When the process of refining is complete, the next step is to implement the changes live. • Provides a good way to “jump start” an accessibility program. • A “kick in the pants”.
  • 40. Recommended Approaches 40 When the process of refining is complete, the next step is to implement the changes live. • Conduct accessibility reviews by using automatic and manual processes. • After the “big” issues resolved conduct usability testing with individuals with disabilities. • Can test in office, conference room (or specialized lab (but not necessary)). • Best if participants can bring in their own AT to usability test. • Tools such as Zoom offer promise of remote usability testing with screen reader users.
  • 41. Recommended Approaches 41 When the process of refining is complete, the next step is to implement the changes live. • Approaches to recruiting participants • Don’t test with all experts • Participants should match your user/customer profile – domain knowledge. • Ideally part of your standard UX approach. • Work with your UX team or local disability organization. • Range of disabilities: visual, learning, motor, etc. Don’t forget older users…
  • 42. Recommended Approaches 42 When the process of refining is complete, the next step is to implement the changes live. • Steve Krug – Rocket Surgery Made Easy • http://www.sen sible.com/rsme .html
  • 44. Moving Forward 44 When the process of refining is complete, the next step is to implement the changes live. • Typical to usability test after accessibility test, however… • Once coding starts UX issues are baked in. Expensive to change. • Accessibility should be integrated into entire design and development process. • Developing user personas and reviewing wireframes, e.g., asking how will someone only using the keyboard or a screen reader to access the primary navigation menu, etc.
  • 45. Moving Forward 45 When the process of refining is complete, the next step is to implement the changes live. • Sarah Horton and Whitney Quesenbery – A Web for Everyone. • http://rosenfel dmedia.com/bo oks/a-web-for- everyone/
  • 46. Moving Forward 46 When the process of refining is complete, the next step is to implement the changes live. • UX Personas with individuals with disabilities
  • 47. Thank You! Questions? Peter McNally Email: pmcnally@Bentley.edu Twitter: @pmcnallyux User Experience Center Bentley University Waltham, MA 02452 USA