Don't Fear the User

Editor's Notes

1. L and J
2. L: What exactly is usability and usability testing? J: Why do usability testing? - Usability testing will benefit you in your work L: Learn how to conduct usability tests - specifically what’s called guerilla usability testing - as well as how to use the results to improve the design of your websites J: Then we’ll finish up with some Q&A.
3. Usability testing refers to evaluating a product or service by testing it with representative users. The goal is to identify any usability problems and assess participant's satisfaction with the product.
4. But what do we mean by “Usability”. Usability refers to how easy something is to use. It consists of a combination of 6 factors: And usability testing enables you to assess these factors in combination: Intuitive Design, Ease of Learning, Efficiency of Use, Memorability, Error frequency and severity Subjective satisfaction
5. If something has an intuitive design, then understanding the architecture and navigation of the site is nearly effortless for users. That means your information is structured well. Your labels are clearly-worded and all your design elements stand out and relative priority is appropriate and obvious.
6. This is how fast a user who has never seen the user interface before can accomplish basic tasks. When someone uses your system for the first time, even never having seen it before, they can get done what they are trying to do.There’s not a steep learning curve. The available functionality on a page makes sense even without a deep understanding of the system.
7. This refers to how fast an experienced user can accomplish tasks. You don’t want expert users bogged down by needless steps, excessive clicking and searching or by pop up explanations of features that they’ve used a hundred times. They can get in, accomplish their task, and get out.
8. Memorability means that after visiting the site, a user can remember enough to use it proficiently in future visits. To what degree can a visitor who hasn’t been to your site or used your product in a long time remember what to do when they go back?
9. How often users make errors while using the system, how serious the errors are, and how users recover from those errors. No system is perfect. Any site or application will have technical or usability errors creep up now and then. And no user is perfect. Everyone makes mistakes sometimes, even if just accidentally clicking the wrong thing. Making sure you have a safety net that includes things like an undo option where feasible and clear and easy-to-understand error messages with obvious calls-to-action will mitigate frustration.
10. Subjective satisfaction just means the user likes using the system. A streamlined and functional chore is still a chore. User experience is fundamentally emotional because we’re humans, and whether we like it or not, we perceive everything through an emotional lens. You’re doing the hard work up front to make sure that the user feels powerful and capable.
11. If you’re designing a system that works really well and is easy to use, but it’s not something anyone needs or wants, then, well, you’ve got another issue entirely.
12. It is however, a critical component of user experience. This is Dan Willis’s UX umbrella and it shows how usability is just a piece of what accounts for the overall user experience.
13. Aside from the fact that we’re telling you you should
14. L: This is how long you have to demonstrate that your website is worth staying on. According to Usability Expert Jakob Nielsen: On the Web, usability is a necessary condition for survival. If a web site is difficult to use, people leave. If the homepage fails to clearly state what a company or program offers and what users can do on the site, people leave. If users get lost on a web site, they leave. If a web site's information is hard to read or doesn't answer users' key questions, they leave. Note a pattern here? There's no such thing as a user reading a web site manual or otherwise spending much time trying to figure out an interface. There are plenty of other web sites available; leaving is the first line of defense when users encounter a difficulty.
15. L: In fact, do you know most frequent user behavior on websites? You guessed it...leaving.
16. Stunning good looks and even fast load times are not enough to make a site people actually want to use and will stay on.
17. L: Even if you are a usability expert, if you are not building a website for people exactly like you, you can only get so far without talking to actual users you are not your users/
18. L: I know some people can have an aversion when they first think about testing I can test my website by myself; why would I have to test with other users I get that but am going to talk about how usability testing really can be worth your while DISCOVER - Discover which parts of your web site are difficult for users. DIAGNOSE - Understand why users do what they do (Google Analytics) VALIDATE - Ensure that the website changes you made do in fact improve users’ experience, and don’t diminish it. Even if you are a usability expert, you can only get so far without talking to actual users Finding out why can be difficult. And while web analytics are helpful for a lot of things, understanding why your website is not performing well (whether it’s not converting users)
19. L: Usability testing is excellent at helping you discover which aspects of your website are difficult or problematic for your users. Again, you are not your user. What you think should be easy, frequently isn’t.
20. Usability testing can help you understand why users are behaving the way they are on your website While website analytics tools like Google Analytics are excellent for showing you where on your website you have a high bounce or drop-off rate, but can’t tell you why or how to fix the problem. Let’s say you have designed a Wordpress site for a retailer that has launched an online store in the last year. Analytics show that customers are bailing in the checkout process once shipping fees have been calculated. How would you aim to improve the company’s conversion rate? Without doing user testing, you don’t know whether the root of the problem is the amount of the shipping fees, concerns about the security of the order form, or confusion about the form itself. [REPLACE WITH MORE RELEVANT EXAMPLE?]
21. If you do diagnose a usability problem and you make some design tweaks to try to fix, usability testing enables you to validate that the changes are effective before releasing. What this all amounts to is saving time and money. Small iterations, testing, and validating is better than investing a lot of time and money only to find that your website is not doing its job and you have confused and unhappy users, low conversion rates, and therefore unhappy clients and managers. And usability testing can also give you a professional edge - you can demonstrate that your designs are evidence-based, that you are truly utilizing a user-centered design process and when it’s time to launch, your site will be more likely to be successful because you’ve identified and addressed the most important usability problems up front.
22. There are a variety of user research and testing methods
23. Of which, Guerilla usability testing is one. Guerilla usability testing is a qualitative technique, It’s also behavioral, so rather than asking people what they think they would do, you’re observing what they do. It’s really good at telling you about the underlying behaviors of your users and giving you insight into why they’re doing them.
24. Why it’s powerful… You do not need a large number of people In fact, research suggests that as few as five people can help you identify 85% of a website’s usability problems.
25. If there’s one key requirement of user testing it’s having users actually test your product or prototype. You’re looking to find users using whatever means necessary. Go where they are, like a coffee shop or the library. So guerilla usability testing is a more informal method of testing. You don’t need a lab or special equipment to do this. And you don’t need to recruit participants or schedule appointments.
26. When you’re testing something you’ve made, you’ll want to look for patterns of behavior. Individuals are different, of course, but it’s when you see different people all do the same thing that you know something’s up. This means you need to run your test with at least 5, and probably no more than 10 people.
27. So how long should this take? The breakdown for this sort of basic testing is something like this: 3 - 4 hours of prep work: finding users, writing down your testing goals, and writing out your script. Another 3 - 5 hours in the field talking to your users and watching them do the tasks. 4 - 6 hours afterwards analyzing the results and prepping presentation. So you’re really just looking at 10 - 15 hours total.
28. You’re going to want to use this when you’re validating your design prior to implementation. It’s really a part of the development process, so you’ll want this to be a regular part of your release cycle. You’ll want to use this on high-risk areas of your product where it’s critical to get it right or you have new concepts or interactions that your users haven’t used before.
29. So let’s go through some steps to show how you could do it yourself.
30. First, and this may come as a shock, you’ll want to think a little about your actual users. How old are they? Where might they live or work? Do during their free time? Where would be a good place to find them to talk to them (if you wanted to do such a crazy thing)?
31. Next, you’re going to determine your research question. So take some time and decide on one single question you want to answer with your test. Again, make sure it’s something that’s critical to your project’s workflow.
32. Now that you know what you want to know… your known unknown, if you will… you’ll want to start planning your test. That means writing a script so you’re being consistent with each participant. A script has 3 main parts: The intro and disclaimer The questions Your exit questions Using natural language, you’re going to write down a task that you’d like to see your users try out. Then give the user questions and tasks, such as, “I’d like you to start on this page and try to find an in-network provider.”
33. You’ll want to start by introducing yourself, explaining the test, and making sure the participant knows what they’re getting into.
34. First, you’ll start by welcoming the participant, saying who you are, and thanking them for their time. Be sure to introduce yourself in a natural way. Think through how you’d normally greet people in a situation like this and write that down. This is going to get you into the mindset of writing a script in a way that feels normal to you so you’ll have an easier time sounding normal while reading through the rest of it. Then you’ll move on to the disclaimer. So, while we’re not doing formal research, it’s still important to make sure your users fully understand exactly what they’re getting into. So be sure to tell them exactly how their information will be used in reporting. For example, if you might want to record a part of the test and replay it for stakeholders later (which I would recommend), then let them know that here. Be sure to get the user’s clear consent before you get started. Having the user sign a consent form is the best way to do this. This page shows an example consent form from Duke Web Services.
35. After you have their permission, you’ll want to move on to explaining what it is you’re going to be doing. So you’ll want to set up your test by providing necessary context. For example, “Today you’re going to use a website to try to find a healthcare provider.” Tell them that you’ll have some questions or tasks for them, and that this information is what you’re going to be using to make decisions about the final site or application you’re going to be making. You’ll also want to let them know that you’re testing the application and NOT them. I also like to state that the participant shouldn’t worry about hurting my feelings. I’m just the tester today and I’ll take the results back to the team and we’ll see how we can make the project better. It’s also a good idea at this point to explain that as they’re going through the tasks it’s good to think out loud so that we can get feedback about what they’re thinking as they’re doing things. Finally, if you have any incentives, go ahead and review those here. For example, free coffee or cookies or—if you’re a big spender—gift cards.
36. When you start asking questions, go ahead and start with some basic demographics. Serves a dual purpose: Gives you a sense of who your participants are and what sorts of users they represent. It also helps make your participants feel more comfortable because they’re the sorts of things that are easy to answer.
37. Finally! Writing the actual questions and tasks! When writing your questions and tasks it’s important to stay focused and brief. Remember, the point of this exercise is to quickly speak with several users to try to discern patterns. To do this, you want to ask more people fewer, more focused questions. Getting people in and out quickly is going to get you more data that you can use to compare and find the patterns you’re looking for. For your tasks, you’re simply asking the participant to show you how they would accomplish a goal. For example, “show me how you would make an appointment at a clinic on this site.”
38. At then end you’ll wrap up with your exit questions. These are just some basic follow-ups that serve as a catch-all. You’ve seen questions like these before at the ends of surveys. I usually just ask the following two: “Is there anything else you’d like to say about the site or application?” “Is there anything you’d like to say about the test itself?”
39. Now that you’ve got your script, it’s time to go out and talk to users and watch them use your site or application. Watch as they do so and take good notes. Remind them to speak out loud as they think through what they’re doing.
40. So here’s what you’ll need to get started. First, go to wherever your users normally are if you can. So if you think you can find people like your users at the library or at a coffee shop, go there. If you can’t, then try to find people as similar as possible and do your best. You’ll want to make sure you have your Script, a Sign, a Computer, and Something to take notes on. This is what I like to call the 2Ss, 1C, and another S. You can remember it with the handy acronym “SSCS.”
41. Before you get started with things, just make sure the participant is comfortable so they aren’t distracted during the test. One tip is to make sure you’re sitting with them instead of standing over them. If you have a laptop, bring a mouse so they can choose whether they want to use that or the trackpad. If you need to pause to take notes, just mention that’s what you’re doing as you do so. “Give me a couple more seconds to finish taking my notes….”
42. Let’s touch on reading from a script. Here’s an example script from Duke Web Services. It really is just a 2 or 3 page Word document. Using a script helps you keep your tests--and thus its results--as consistent as possible. But because you’re not going to be publishing these in a peer-reviewed journal, exactness isn’t critical. At the same time, while going off script isn’t forbidden, it does mean you’ll have to use your judgement. I don’t recommend going off-script though unless you feel like you need to track something down. Remember, you’re just trying to find patterns of behavior so you can make your project better for your users.
43. Once you’ve conducted 5 - 10 tests it’s time to go back and compare your results. Be sure to make note of anything you didn’t expect or any unexpected patterns of behavior. This is where you’ll be really glad you took good notes!
44. So when you’re analyzing your data, here are some things to look for. Difficulty clicking targets. Maybe they were too small on mobile and the user had to click them a couple of times but didn’t mention it out loud.
45. Participants clicking things that don’t actually do anything. Yes, this will happen.
46. People doing unexpected things. This is the good stuff. For links to things that aren’t in the expected workflow for my prototype I will usually create a page that lets the user know the page doesn’t exist and prompts them to use their browser back button.
47. Similarly, people being surprised by where a link takes them.
48. Difficulty locating desired information. This happens more than I care to admit and it will never cease to surprise me every time a participant fumbles around missing the clearly-labeled, giant orange button that’s right in front of them.
49. When presenting your findings you’ll want to talk about the behaviors you saw and the conclusions you drew of course, but nothing has an impact on those you’re presenting to like direct user quotes, images, and videos. Being able to let the user or their actions speak for themselves can be truly enlightening to reluctant stakeholders. “Here is something a real user actually said when they were trying to do this,” is really hard to argue with. Imagine if that’s 2 or 3 users. This is where you’re glad you thought ahead to this point and got specific permission for the types of reporting artifacts you’re using. Great! Now you’re done! What’s next, you ask? It’s almost 5 on Friday. Margarita time!
50. No! You do it again! User testing should be a key part of the development process. That means doing it as much as you have bandwidth for. Having the ability to refine your project before it’s implemented will save you time and money, which is truly important for under-resourced teams where projects can live on for years after they go live before they’re touched again (but no-one here would know anything about that).
51. L: Finally, let’s just go over some quick tips and tricks that will help you out as you do your testing.
52. Just grab some people in your own office and test your test before you get started. This will get you talking through your script and help you see where you might need to change things up. For example, this is great at revealing that you might be showing the participant where the answer to a later question is ahead of time, which would bias your test. It’ll also help you work the awkwardness out of your script reading.
53. We touched on this a little before, but it’s important to remember that it’s not just about what the participant says, but also what they do. Note their actions and pay attention to how they’re doing things. Often users can think they’re accomplishing their task when they aren’t. Pay attention to reactions and expressions, because behavior doesn’t always match what people say. A user could be cruising along through a flow, get caught up on a radio button that takes a few clicks to activate, but they don’t consciously think much of it.
54. Try not to bias people with your own reactions. “What? That GIANT ORANGE BUTTON IS RIGHT IN FRONT OF YOU!”
55. So, in conclusion: user testing doesn’t have to be difficult or intimidating. Just get out and get people using what you’re making and watch and learn as they do it. You will be amazed, we promise.