SlideShare a Scribd company logo

Supercharge Your Corporate Dashboards With UX Analytics

•
•
UserZoom
UserZoom

This document discusses supercharging corporate dashboards with UserZoom. It begins with polls to understand the audience. It then discusses the current state of user research, including common organizational models and goals to grow the practice. It defines three types of dashboards - individual research engagements, product scorecards, and executive dashboards. Product scorecards provide consistent reporting of UX measures to product teams. Executive dashboards do the same for leadership. The document outlines a vision for UserZoom to be a single place for all dashboard data and describes plans to provide this capability starting in 2017.

1 of 57
Download to read offline
Supercharge your Corporate Dashboards Andrea Peer, Ph.D. | Director of Onboarding | January 18, 2017
Quick Housekeeping • Chat box is available if you have questions • There will be time for Q&A at the end • We will be recording the webinar for future viewing Additional information about these feature can be found in the UserZoom Support Center
Focus Today Current State of the Union Individual Research Engagements 3 Types of Reporting Andrea Peer apeer@userzoom.com +1 (719) 661-8440 The UserZoom Vision Wrap Up Executive Dashboards Product Scorecards
Let’s start with a poll What is your role?
Let’s start with a poll Do you use UserZoom today?
Let’s start with a poll Do you have a single place where UX data is presented to stakeholders AND those stakeholders can access is later?
What is the current state of the union for user research?
Current State of the Union What should we design/build? UX research answers one of two questions. Did we design it right?
Current State of the Union Portfolio Concept Prototype Alpha Beta Prod • Strategic focus • Big picture • Business risk • Innovation • R&D • Longitudinal • Quasi strategic but have to get to tactical • Design iterations focused • More tactical focus to feed dev • Feel the need for speed • Small group of loyal customers focused • Major risks identified • The gatekeepers • Larger group of loyal customers or target users/regions • Monitor risks • Continuous improvement • Looking for gaps • Intercept strategy • Marketing integration UX research falls into one or more of these areas on the PDLC.
Current State of the Union UX research usually looks like one of these models in organizations. Shared Service for Org • One central UX group that serves the organization • Research requests usually come ad hoc from prod teams. • Sometimes UX team will determine their own work based on strategic needs. UX Research Prod Team Prod Team UX Research Prod Team Deeply Embedded with Dev • Many UX people/groups that may collaborate but they all serve different parts of the organization and can work independently • Research requests come on a cadence that matches the development cadence. Usually more tactical.
Current State of the Union UX research usually looks like one of these models in organizations. Shared Service with Embedded • One central UX group that serves the leadership and support individual UX researchers embedded with product teams. • Research requests are more strategic in nature focused on big organization wide questions. • Central UX may act as consult to product UX. Central UX Leadership Team Product UX Prod Team Designers
Current State of the Union Irrespective of where in the PDLC you do research and what organization model you follow for UX research, we are all trying to do the same thing….. Grow UX research practice within the organization. Influence Impact Authority Budget Headcount
Current State of the Union Guiding principles to grow UX research practice within the organization. • Have a vision / value prop statement • Establish a consistent UX measurement (ie: Universal Measures) • Establish a consistent way to deliver UX data • Penetrate into process (PDLC) • Be the behavioral data powerhouse (cadence & rigor) • Integrate with the experience data stack • Make your awesome work known!
What do I mean with I say “Dashboard?”
3 Types of Dashboards Individual Research Engagements Product Scorecard Executive Dashboards Single user research engagement. Sharing the findings with a small set of internal stakeholders. Light reporting, more about take the data and run into dev. Next level – Have a consistent UX measure that you report on to help stakeholders know what to prioritize with the findings. A consistent way to show UX research data to the product team that is always accessible to all members of the product team and reports on items that are important to that product team. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data in one view. A consistent way to show UX research data to leadership that is always accessible and reports on items that are important to established business KPIs. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data and business KPI data in one view.
UserZoom and Dashboards The UserZoom Vision… All of your UX dashboards in one spot for any stakeholder to access, explore, and make data-driven decisions.
UserZoom and Dashboards The UserZoom Vision… All of your UX dashboards in one spot for any stakeholder to access, explore, and make data-driven decisions. Step 1: In 2017, UserZoom made a significant architecture Investment Step 2: Dec 2017, provide MVP solution to clients to gain the value from the architecture investment. à Tableau Web Data Connector Step 3: 2018, make this value known in the UserZoom Manager. Step 4: Date TBD, Voyage into the BI space with APIs.
3 Types of Dashboards Individual Research Engagements Product Scorecard Executive Dashboards Single user research engagement. Sharing the findings with a small set of internal stakeholders. Light reporting, more about take the data and run into dev. Next level – Have a consistent UX measure that you report on to help stakeholders know what to prioritize with the findings. Tie to analytics. A consistent way to show UX research data to the product team that is always accessible to all members of the product team and reports on items that are important to that product team. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data in one view. A consistent way to show UX research data to leadership that is always accessible and reports on items that are important to established business KPIs. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data and business KPI data in one view.
UserZoom vs Dashboard 2017 • Support one place to find data across research engagements • Modify chart formats or colors to align with brand • Combine individual research results with other data (ie: web analytics) • Show pivots • Show qualitative video results quickly • Fast project summaries and reporting • Results reported from one project only
UserZoom vs Dashboard 2017 Research Engagements need to show pivots Example 1 • Segment factor 1 • Segment factor 2 Example 2 • Task 1 with design A • Task 1 with design B Example 3 • Study 1 – Pre • Study 2 – Post Dashboard Experience 2017
UserZoom Tableau Web Data Connector Research Engagements need to show pivots Example 2 • Task 1 with design A • Task 1 with design B Example 3 • Study 1 – Pre • Study 2 – Post
UserZoom Tableau Web Data Connector Research Engagements need to show pivots Example 1 • Segment factor 1 • Segment factor 2
UserZoom Tableau Web Data Connector References https://community.tableau.com/community/developers/web-data-connectors https://www.tableau.com/about/blog/2015/8/connect-just-about-any-web-data-new-web-data-connector-42246
3 Types of Dashboards Individual Research Engagements Product Scorecard Executive Dashboards Single user research engagement. Sharing the findings with a small set of internal stakeholders. Light reporting, more about take the data and run into dev. Next level – Have a consistent UX measure that you report on to help stakeholders know what to prioritize with the findings. A consistent way to show UX research data to the product team that is always accessible to all members of the product team and reports on items that are important to that product team. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data in one view. A consistent way to show UX research data to leadership that is always accessible and reports on items that are important to established business KPIs. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data and business KPI data in one view.
Let’s start with a story
The Story – The Matrix The Organizational Structure Me 5 Researchers 10 Designers 3 Web Analytics Product Team 1 Product Team 2 Senior Leadership Team User Experience & Product Analytics Team VP Marketing VP Product VP Engineering Product Team 17 Engineering Team 1 Engineering Team 17 Product Teams PO Team 1 PO Team 2 PO Team 17 Data Science Team Data Science Manager Data Science 1 Data Science 10
The Experience Stack Concept Dev Beta Production What do we design? Did we design it right? UX research Marketing research Customer data Cost of production/develop ment Quality measures Sales Transactions Product usage data Customer surveys Call center data Experimentals Intercept surveysCTAs
We want to make data-driven product decisions and We have a ton of data from everywhere!
The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
1 Place Where all Data Can Live and Be Accessed by All Stakeholders For the Matrix Product 1 Product 2 Product 3 Product 4 Product 5 Product Name Product description and success statements Product Name Experience Data
The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
Categorize Data by Exploratory and Critical For the Matrix Critical Transactions Exploratory Analytics Critical transactions are customer behaviors that directly contribute to the vision, mission and return on investment for the company. Exploratory analytics are queries that can aid in design and business decisions but have not yet been recognized on the critical transaction paths. Realized Opportunity Total Opportunity % of Opportunity Realized Findings here are very good to highlight areas for design improvement
The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
Establish the Intake and Reporting Process For the Matrix • Form to submit requests • Backlog is visible to all • Product Owners work with UX team to build their desired product scorecard • UX interviews senior level leadership to build executive dashboards • All UX research shown on the site in a consistent format • Scorecard • Report • Tagged Videos • Tagged Transcripts
The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
3 Types of Dashboards Individual Research Engagements Product Scorecard Executive Dashboards Single user research engagement. Sharing the findings with a small set of internal stakeholders. Light reporting, more about take the data and run into dev. Next level – Have a consistent UX measure that you report on to help stakeholders know what to prioritize with the findings. A consistent way to show UX research data to the product team that is always accessible to all members of the product team and reports on items that are important to that product team. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data in one view. A consistent way to show UX research data to leadership that is always accessible and reports on items that are important to established business KPIs. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data and business KPI data in one view.
Product Scorecard Components • Product Owners work with UX team to build their desired product scorecard • Some components they had to have • Critical Analytics, Business KPIs, eXperience Score • Some components they picked • Exploratory Analytics, specific UX goals/measures • Other data streams
Product Scorecard Components The Experience Stack Concept Dev Beta Production What do we design? Did we design it right? UX research Marketing research Customer data Cost of production/develop ment Quality measures Sales Transactions Product usage data Customer surveys Call center data Experimentals Intercept surveysCTAs
Product Scorecard Components Financial Indicators System Produced indicators Experience indicators Return On Investment Transactions Net Profit/LossCash Flow Capital Reserves / Debt Site Traffic Bounce Rate Click Analytics CTA Engagement CSat NPS eXperience Score (Task Success, Ease of Use, Empowerment) Call Center DataCustomer Data
Product Scorecard For the Matrix Product 1 Product 2 Product 3 Product 4 Product 5 Product Name Product description and success statements Product Name Experience Data
Product 1 Product 2 Product 3 Product 4 Product 5 Product Name Product description and success statements Product Name Experience Data % UX Score Data representation Product Usage by Product KPI • Over time • By segment Task Success Ease of Use Empower- ment UX Story Major UX Findings Product Name Feature | Feature | Feature | Feature Use Use Use Use Use Use Use Use Use Use Use
Product 1 Product 2 Product 3 Product 4 Product 5 Product Name Product description and success statements Product Name Experience Data % UX Score UX Story Major UX Findings Data representation Product Usage by Product KPI • Over time • By segmentTask Success Ease of Use Empower- ment
UserZoom Tableau Web Data Connector 3 Types of Dashboards Individual Research Engagements Product Scorecard Executive Dashboards Single user research engagement. Sharing the findings with a small set of internal stakeholders. Light reporting, more about take the data and run into dev. Next level – Have a consistent UX measure that you report on to help stakeholders know what to prioritize with the findings. A consistent way to show UX research data to the product team that is always accessible to all members of the product team and reports on items that are important to that product team. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data in one view. A consistent way to show UX research data to leadership that is always accessible and reports on items that are important to established business KPIs. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data and business KPI data in one view.
Executive Dashboard • Interviews to get the key dashboards on there • Build the dashboards and iterate • Some components they had to have • Critical Analytics, Business KPIs, aggregate and all product eXperience Scores • Some components they picked • Exploratory Analytics, specific UX goals/measures aggregate • Other data streams
Product Scorecard Components Financial Indicators System Produced indicators Experience indicators Return On Investment Transactions Net Profit/LossCash Flow Capital Reserves / Debt Site Traffic Bounce Rate Click Analytics CTA Engagement CSat NPS eXperience Score (Task Success, Ease of Use, Empowerment) Call Center DataCustomer Data
Product 1 Product 2 Product 3 Product 4 Product 5 Product Name Product description and success statements Product Name Experience Data % UX Score UX Story Major UX Activity Data representation Task Success Ease of Use Empower- ment Key Initiatives Initiative 1 Initiative 2 Initiative 3 Initiative 4 # # # # $xM
The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
What does it take to make this happen in your organization?
UserZoom Tableau Web Data Connector How do you actually make this happen in your organization? User researcher doing user research using UserZoom. (Note: All methods are supported.) Database support Data base people that can do the work or provide consult. They have infrastructure knowledge on your Tableau instance 1 2Required Required
UserZoom Tableau Web Data Connector How do you actually make this happen in your organization? Visual Designer Someone to add cool visuals to your dashboards and fit with your brand. Helps deliver the story better. 3 4Next Level Next Level Data Architect Support These folks can pull multiple data sources together into one view. They can help you tell a full story of all the data in your experience stack. Data source Data source Data source
Tableau Data Tables
Invest in the Infrastructure to Synthesize the Experience Stack For the Matrix 1. The stuff that goes into products to track use 6. Transformation to business value foundation (ie: avg session duration calculation) 7. Transformation to business value meaning making 8. Transformation to business value predictive/causation 3. Core infrastructure – data ownership, residency, structures 10. Access – Visualization to insights (take action on insights) 2. Access – Raw data 9. Access – Visualization to fundamentals 4. Connecting data to the business context (ie: URL management, product awareness) Collection Meaning Making Data management Business Use 5. Implement the meaningful algorithms and standard definitions (ie: what constitutes an active user)
ProdU Work Buckets - Starting Project Reports 12% AdHoc 15% Data Infrastructure 36% Security 3% GA Management 3% ProU Site Management 13% Design 5% Other 13% PRODUCT UTILIZATION TIME TODAY
ProdU Work Buckets - Goal Project Reports 40% AdHoc 10% Data Infrastructure 30% Security 1% GA Management 1% ProU Site Management 10% Design 5% Other 3% PRODUCT UTILIZATION TIME GOAL
ProdU Work Buckets Project Reports AdHoc Data Infrastructure Security GA Management ProU Site Management Design Other TODAY 13% 15% 36% 3% 3% 13% 5% 13% 2016 GLOAL 40% 10% 30% 1% 1% 10% 5% 3% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% % Allocation ProdU Work % Allocation
Questions?

More Related Content

Supercharge Your Corporate Dashboards With UX Analytics

  • 1. Supercharge your Corporate Dashboards Andrea Peer, Ph.D. | Director of Onboarding | January 18, 2017
  • 2. Quick Housekeeping • Chat box is available if you have questions • There will be time for Q&A at the end • We will be recording the webinar for future viewing Additional information about these feature can be found in the UserZoom Support Center
  • 3. Focus Today Current State of the Union Individual Research Engagements 3 Types of Reporting Andrea Peer apeer@userzoom.com +1 (719) 661-8440 The UserZoom Vision Wrap Up Executive Dashboards Product Scorecards
  • 4. Let’s start with a poll What is your role?
  • 5. Let’s start with a poll Do you use UserZoom today?
  • 6. Let’s start with a poll Do you have a single place where UX data is presented to stakeholders AND those stakeholders can access is later?
  • 7. What is the current state of the union for user research?
  • 8. Current State of the Union What should we design/build? UX research answers one of two questions. Did we design it right?
  • 9. Current State of the Union Portfolio Concept Prototype Alpha Beta Prod • Strategic focus • Big picture • Business risk • Innovation • R&D • Longitudinal • Quasi strategic but have to get to tactical • Design iterations focused • More tactical focus to feed dev • Feel the need for speed • Small group of loyal customers focused • Major risks identified • The gatekeepers • Larger group of loyal customers or target users/regions • Monitor risks • Continuous improvement • Looking for gaps • Intercept strategy • Marketing integration UX research falls into one or more of these areas on the PDLC.
  • 10. Current State of the Union UX research usually looks like one of these models in organizations. Shared Service for Org • One central UX group that serves the organization • Research requests usually come ad hoc from prod teams. • Sometimes UX team will determine their own work based on strategic needs. UX Research Prod Team Prod Team UX Research Prod Team Deeply Embedded with Dev • Many UX people/groups that may collaborate but they all serve different parts of the organization and can work independently • Research requests come on a cadence that matches the development cadence. Usually more tactical.
  • 11. Current State of the Union UX research usually looks like one of these models in organizations. Shared Service with Embedded • One central UX group that serves the leadership and support individual UX researchers embedded with product teams. • Research requests are more strategic in nature focused on big organization wide questions. • Central UX may act as consult to product UX. Central UX Leadership Team Product UX Prod Team Designers
  • 12. Current State of the Union Irrespective of where in the PDLC you do research and what organization model you follow for UX research, we are all trying to do the same thing….. Grow UX research practice within the organization. Influence Impact Authority Budget Headcount
  • 13. Current State of the Union Guiding principles to grow UX research practice within the organization. • Have a vision / value prop statement • Establish a consistent UX measurement (ie: Universal Measures) • Establish a consistent way to deliver UX data • Penetrate into process (PDLC) • Be the behavioral data powerhouse (cadence & rigor) • Integrate with the experience data stack • Make your awesome work known!
  • 14. What do I mean with I say “Dashboard?”
  • 15. 3 Types of Dashboards Individual Research Engagements Product Scorecard Executive Dashboards Single user research engagement. Sharing the findings with a small set of internal stakeholders. Light reporting, more about take the data and run into dev. Next level – Have a consistent UX measure that you report on to help stakeholders know what to prioritize with the findings. A consistent way to show UX research data to the product team that is always accessible to all members of the product team and reports on items that are important to that product team. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data in one view. A consistent way to show UX research data to leadership that is always accessible and reports on items that are important to established business KPIs. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data and business KPI data in one view.
  • 16. UserZoom and Dashboards The UserZoom Vision… All of your UX dashboards in one spot for any stakeholder to access, explore, and make data-driven decisions.
  • 17. UserZoom and Dashboards The UserZoom Vision… All of your UX dashboards in one spot for any stakeholder to access, explore, and make data-driven decisions. Step 1: In 2017, UserZoom made a significant architecture Investment Step 2: Dec 2017, provide MVP solution to clients to gain the value from the architecture investment. à Tableau Web Data Connector Step 3: 2018, make this value known in the UserZoom Manager. Step 4: Date TBD, Voyage into the BI space with APIs.
  • 18. 3 Types of Dashboards Individual Research Engagements Product Scorecard Executive Dashboards Single user research engagement. Sharing the findings with a small set of internal stakeholders. Light reporting, more about take the data and run into dev. Next level – Have a consistent UX measure that you report on to help stakeholders know what to prioritize with the findings. Tie to analytics. A consistent way to show UX research data to the product team that is always accessible to all members of the product team and reports on items that are important to that product team. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data in one view. A consistent way to show UX research data to leadership that is always accessible and reports on items that are important to established business KPIs. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data and business KPI data in one view.
  • 19. UserZoom vs Dashboard 2017 • Support one place to find data across research engagements • Modify chart formats or colors to align with brand • Combine individual research results with other data (ie: web analytics) • Show pivots • Show qualitative video results quickly • Fast project summaries and reporting • Results reported from one project only
  • 20. UserZoom vs Dashboard 2017 Research Engagements need to show pivots Example 1 • Segment factor 1 • Segment factor 2 Example 2 • Task 1 with design A • Task 1 with design B Example 3 • Study 1 – Pre • Study 2 – Post Dashboard Experience 2017
  • 21. UserZoom Tableau Web Data Connector Research Engagements need to show pivots Example 2 • Task 1 with design A • Task 1 with design B Example 3 • Study 1 – Pre • Study 2 – Post
  • 22. UserZoom Tableau Web Data Connector Research Engagements need to show pivots Example 1 • Segment factor 1 • Segment factor 2
  • 23. UserZoom Tableau Web Data Connector References https://community.tableau.com/community/developers/web-data-connectors https://www.tableau.com/about/blog/2015/8/connect-just-about-any-web-data-new-web-data-connector-42246
  • 24. 3 Types of Dashboards Individual Research Engagements Product Scorecard Executive Dashboards Single user research engagement. Sharing the findings with a small set of internal stakeholders. Light reporting, more about take the data and run into dev. Next level – Have a consistent UX measure that you report on to help stakeholders know what to prioritize with the findings. A consistent way to show UX research data to the product team that is always accessible to all members of the product team and reports on items that are important to that product team. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data in one view. A consistent way to show UX research data to leadership that is always accessible and reports on items that are important to established business KPIs. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data and business KPI data in one view.
  • 26. The Story – The Matrix The Organizational Structure Me 5 Researchers 10 Designers 3 Web Analytics Product Team 1 Product Team 2 Senior Leadership Team User Experience & Product Analytics Team VP Marketing VP Product VP Engineering Product Team 17 Engineering Team 1 Engineering Team 17 Product Teams PO Team 1 PO Team 2 PO Team 17 Data Science Team Data Science Manager Data Science 1 Data Science 10
  • 27. The Experience Stack Concept Dev Beta Production What do we design? Did we design it right? UX research Marketing research Customer data Cost of production/develop ment Quality measures Sales Transactions Product usage data Customer surveys Call center data Experimentals Intercept surveysCTAs
  • 28. We want to make data-driven product decisions and We have a ton of data from everywhere!
  • 29. The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
  • 30. The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
  • 31. 1 Place Where all Data Can Live and Be Accessed by All Stakeholders For the Matrix Product 1 Product 2 Product 3 Product 4 Product 5 Product Name Product description and success statements Product Name Experience Data
  • 32. The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
  • 33. Categorize Data by Exploratory and Critical For the Matrix Critical Transactions Exploratory Analytics Critical transactions are customer behaviors that directly contribute to the vision, mission and return on investment for the company. Exploratory analytics are queries that can aid in design and business decisions but have not yet been recognized on the critical transaction paths. Realized Opportunity Total Opportunity % of Opportunity Realized Findings here are very good to highlight areas for design improvement
  • 34. The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
  • 35. Establish the Intake and Reporting Process For the Matrix • Form to submit requests • Backlog is visible to all • Product Owners work with UX team to build their desired product scorecard • UX interviews senior level leadership to build executive dashboards • All UX research shown on the site in a consistent format • Scorecard • Report • Tagged Videos • Tagged Transcripts
  • 36. The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
  • 37. 3 Types of Dashboards Individual Research Engagements Product Scorecard Executive Dashboards Single user research engagement. Sharing the findings with a small set of internal stakeholders. Light reporting, more about take the data and run into dev. Next level – Have a consistent UX measure that you report on to help stakeholders know what to prioritize with the findings. A consistent way to show UX research data to the product team that is always accessible to all members of the product team and reports on items that are important to that product team. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data in one view. A consistent way to show UX research data to leadership that is always accessible and reports on items that are important to established business KPIs. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data and business KPI data in one view.
  • 38. Product Scorecard Components • Product Owners work with UX team to build their desired product scorecard • Some components they had to have • Critical Analytics, Business KPIs, eXperience Score • Some components they picked • Exploratory Analytics, specific UX goals/measures • Other data streams
  • 39. Product Scorecard Components The Experience Stack Concept Dev Beta Production What do we design? Did we design it right? UX research Marketing research Customer data Cost of production/develop ment Quality measures Sales Transactions Product usage data Customer surveys Call center data Experimentals Intercept surveysCTAs
  • 40. Product Scorecard Components Financial Indicators System Produced indicators Experience indicators Return On Investment Transactions Net Profit/LossCash Flow Capital Reserves / Debt Site Traffic Bounce Rate Click Analytics CTA Engagement CSat NPS eXperience Score (Task Success, Ease of Use, Empowerment) Call Center DataCustomer Data
  • 41. Product Scorecard For the Matrix Product 1 Product 2 Product 3 Product 4 Product 5 Product Name Product description and success statements Product Name Experience Data
  • 42. Product 1 Product 2 Product 3 Product 4 Product 5 Product Name Product description and success statements Product Name Experience Data % UX Score Data representation Product Usage by Product KPI • Over time • By segment Task Success Ease of Use Empower- ment UX Story Major UX Findings Product Name Feature | Feature | Feature | Feature Use Use Use Use Use Use Use Use Use Use Use
  • 43. Product 1 Product 2 Product 3 Product 4 Product 5 Product Name Product description and success statements Product Name Experience Data % UX Score UX Story Major UX Findings Data representation Product Usage by Product KPI • Over time • By segmentTask Success Ease of Use Empower- ment
  • 44. UserZoom Tableau Web Data Connector 3 Types of Dashboards Individual Research Engagements Product Scorecard Executive Dashboards Single user research engagement. Sharing the findings with a small set of internal stakeholders. Light reporting, more about take the data and run into dev. Next level – Have a consistent UX measure that you report on to help stakeholders know what to prioritize with the findings. A consistent way to show UX research data to the product team that is always accessible to all members of the product team and reports on items that are important to that product team. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data in one view. A consistent way to show UX research data to leadership that is always accessible and reports on items that are important to established business KPIs. Next level – Have a consistent UX measure that you track over time and couple UX research data with analytics data and business KPI data in one view.
  • 45. Executive Dashboard • Interviews to get the key dashboards on there • Build the dashboards and iterate • Some components they had to have • Critical Analytics, Business KPIs, aggregate and all product eXperience Scores • Some components they picked • Exploratory Analytics, specific UX goals/measures aggregate • Other data streams
  • 46. Product Scorecard Components Financial Indicators System Produced indicators Experience indicators Return On Investment Transactions Net Profit/LossCash Flow Capital Reserves / Debt Site Traffic Bounce Rate Click Analytics CTA Engagement CSat NPS eXperience Score (Task Success, Ease of Use, Empowerment) Call Center DataCustomer Data
  • 47. Product 1 Product 2 Product 3 Product 4 Product 5 Product Name Product description and success statements Product Name Experience Data % UX Score UX Story Major UX Activity Data representation Task Success Ease of Use Empower- ment Key Initiatives Initiative 1 Initiative 2 Initiative 3 Initiative 4 # # # # $xM
  • 48. The Approach 1. 1 place where all data can live and be accessed by all stakeholders 2. Categorize data by “Exploratory” and “Critical” 3. Identify the data needs intake and reporting process 4. Invest in the infrastructure to support synthesizing the experience stack For the Matrix
  • 49. What does it take to make this happen in your organization?
  • 50. UserZoom Tableau Web Data Connector How do you actually make this happen in your organization? User researcher doing user research using UserZoom. (Note: All methods are supported.) Database support Data base people that can do the work or provide consult. They have infrastructure knowledge on your Tableau instance 1 2Required Required
  • 51. UserZoom Tableau Web Data Connector How do you actually make this happen in your organization? Visual Designer Someone to add cool visuals to your dashboards and fit with your brand. Helps deliver the story better. 3 4Next Level Next Level Data Architect Support These folks can pull multiple data sources together into one view. They can help you tell a full story of all the data in your experience stack. Data source Data source Data source
  • 53. Invest in the Infrastructure to Synthesize the Experience Stack For the Matrix 1. The stuff that goes into products to track use 6. Transformation to business value foundation (ie: avg session duration calculation) 7. Transformation to business value meaning making 8. Transformation to business value predictive/causation 3. Core infrastructure – data ownership, residency, structures 10. Access – Visualization to insights (take action on insights) 2. Access – Raw data 9. Access – Visualization to fundamentals 4. Connecting data to the business context (ie: URL management, product awareness) Collection Meaning Making Data management Business Use 5. Implement the meaningful algorithms and standard definitions (ie: what constitutes an active user)
  • 54. ProdU Work Buckets - Starting Project Reports 12% AdHoc 15% Data Infrastructure 36% Security 3% GA Management 3% ProU Site Management 13% Design 5% Other 13% PRODUCT UTILIZATION TIME TODAY
  • 55. ProdU Work Buckets - Goal Project Reports 40% AdHoc 10% Data Infrastructure 30% Security 1% GA Management 1% ProU Site Management 10% Design 5% Other 3% PRODUCT UTILIZATION TIME GOAL
  • 56. ProdU Work Buckets Project Reports AdHoc Data Infrastructure Security GA Management ProU Site Management Design Other TODAY 13% 15% 36% 3% 3% 13% 5% 13% 2016 GLOAL 40% 10% 30% 1% 1% 10% 5% 3% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% % Allocation ProdU Work % Allocation