Design by Numbers: A Data-Driven UX Process
•Download as PPTX, PDF•
4 likes•970 views
This document discusses a data-driven UX process that uses both qualitative and quantitative metrics. It presents the HEART framework for defining UX goals and metrics around Happiness, Engagement, Adoption, Retention, and Task Success. An example is provided of applying this process to an advanced search feature, where initial user interviews and testing revealed discrepancies between stated importance and actual usage. Prototypes were developed and tested, with metrics like SUS scores and error rates measured before and after to evaluate improvements. The process emphasizes continuously defining and refining goals, collecting both baseline and comparative metrics over time, and using data to inform iterative design changes.
Report
Share
Design by Numbers: A Data-Driven UX Process
- 1. Design by Numbers A Data-Driven UX Process Brian Rimel @brianrimel UX Consultant, OpenSource Connections
- 2. User-Centered Design Internal enterprise applications Access to users
- 3. Why Data? Balancing the qualitative and quantitative You can’t always trust your users Limited data doesn’t tell the whole story
- 4. The HEART Framework Src: https://library.gv.com/how-to-choose-the-right-ux-metrics-for-your-product-5f46359ab5be
- 5. PULSE Metrics Page views, Uptime, Latency, Seven-day active users, Earnings
- 7. HEART Metrics Happiness, Engagement, Adoption, Retention, Task Success
- 8. Happiness Satisfaction or Delight System Usability Scale, Net Promoter Score
- 9. Engagement Level of involvement Number of visits per user per week
- 10. Adoption New users/uses of a feature Number of accounts created in the last 7 days
- 11. Retention Rate at which existing users return Percentage of seven-day active users that are still active 30 days later
- 12. Task Success Traditional behavior metrics for efficiency, effectiveness, and error rate. Percentage of completion errors for a given task
- 14. Goals Signals Metrics Happiness The user feels the welcome wizard is easy to use Level of user satisfaction Mean SUS Score Engagement - - - Adoption - - - Retention - - - Task Success The welcome wizard should be as simple as possible The number of errors during the process Rate of error per step Example: Welcome Wizard
- 15. Goals should be SMART Specific, Measurable, Attainable, Realistic, Time-Based
- 16. Normalize the Data What does an increase in total active users tell us?
- 18. Initial Metrics Gathering Existing metrics influence feature priority Kano Survey for feature-level satisfaction
- 19. Kano Survey src: http://uxmag.com/articles/leveraging-the-kano-model-for-optimal-results Feature Must-be One- Dimensional Attractive Unimportan t Undesired Advanced Search 87% 8% 4% 1% 0%
- 20. Prioritizing of Features 1. 2. 3. Advanced Search 4. 5. 6. 7. 8. 9. 10. From Kano Survey: 87% Must-be feature From Usage Statistics: 22% Engagement/Week Why the discrepancy?
- 21. Goals Signals Metrics Happiness The user feels comfortable using advanced search Level of confidence SUS Survey Engagement The features enable consistent searching Number of advanced searches Searches per day per user Adoption - - - Retention - - - Task Success The advanced search process is easily understood User enters a query, but does not complete the search Percentage of Abandoned Searches Advanced Search: Goals & Metrics
- 22. User Interview & Testing Identify discrepancy between stated importance and usage metrics Establish baseline metrics Measure satisfaction - SUS Survey
- 23. System Usability Scale (SUS) src: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
- 24. Review Findings Metric Initial Testing Mean SUS Score 56 Error Rate / Step 21% Okay, so where is the problem? Let’s map it!
- 27. User Testing of Prototype Continue measuring baseline metrics A/B Testing Follow-up SUS Survey
- 28. Results & Recommendations Great! But, what does this mean? Context critical to interpretation Metric Initial Testing Prototype Testing Mean SUS Score 56 73 Error Rate / Step 21% 12%
- 30. Long-Term Metrics Tracking Engagement, Adoption, Retention, and Task Success over time Periodic usability testing of full application
- 32. Tools
- 33. Kibana Dashboard
- 34. “Extremely satisfied is like extremely edible.” - Jared Spool
- 35. Key Takeaways • Collaboratively define SMART goals • Revisit and challenge goals • Continuously monitor metrics over time • Balance quantitative and qualitative measures
- 36. Questions
- 37. References • Google HEART Metrics Study http://static.googleusercontent.com/media/research.google.com/e n//pubs/archive/36299.pdf • Kano Survey http://uxmag.com/articles/leveraging-the-kano-model-for-optimal- results • SUS Survey https://www.usability.gov/how-to-and-tools/methods/system- usability-scale.html