SlideShare a Scribd company logo
#SMX #21C @ayat
Breaking Through
Practical Steps to follow to identify what and where to test
#SMX #21C @ayat
Breaking Through
Practical Steps to follow to identify what and where to test
#SMX #21C @ayat
• 3,000+ successful tests
• 400+ CRO projects
• 11 different countries
• 65 years of combined experience
THE NUMBERS
#SMX #21C @ayat
#SMX #21C @ayat
780 Tests/Year
#SMX #21C @ayat
Why?
#SMX #21C @ayat
• Font/color of texts
• "lowest price guarantee" vs
"110% price match
guarantee"
• Add to cart button - different
designs/sizes
• a badge that shows "XXX
products sold
The old way
#SMX #21C @ayat
Conversion
Optimization
System
#SMX #21C @ayat
Qualitative data focuses on the “Why”
#SMX #21C @ayat
Qualitative data
• Heuristic Evaluation
• Customer voice
• Polls
• Surveys
• 1 on 1 interviews
• Customer service feedback
• Usability testing
• Focus group
• User testing
• Video Recording
#SMX #21C @ayat
§ Helps you determine the broken areas on the website
§ Helps you get a full view of what the visitor experiences, especially if
you’ve done your homework and considered most the different scenarios
a visitor may experience
§ Gives you a sense of possible testing areas (which need to vetted)
§ Allows you to then categorize the experience for “to testing ideas” and
overall fixes that need to happen (broken areas on a website do not to
be “tested”)
§ Should be conducted with more than one person
STOP THE BLEEDING REPORT
#SMX #21C @ayat
Case study
25% increase in on-site search usage
Search users more likely to convert
#SMX #21C @ayat
Polls and Surveys
#SMX #21C @ayat
Polls and Surveys
§Motivators
§Barriers
§Hooks
§Satisfaction
– Overall NPS
– Page Level
§Demographical information
§Feedback
§Scoping competitive space
#SMX #21C @ayat
“What is preventing you from making a purchase?”
1. I cannot find the information I need
2. I don’t know if the item fits my phone
3. I want a different color
4. I cannot find the shipping costs
5. I am just comparing prices
Result: understanding of what bothers
the visitors + testing ideas
#SMX #21C @ayat
“What is preventing you from making a purchase?”
Result Action
I cannot find the information I need Investigate further
I don’t know if the item fits my phone Fix right away –Test placement
I want a different color Fix right away -Test design/placement
I cannot find the shipping costs Test
I am just comparing prices Capture email
#SMX #21C @ayat
Which
design
increased
conversions?
1
#SMX #21C @ayat
Which design
increased
conversions?
2
#SMX #21C @ayat
Which
design
increased
conversions?
3
#SMX #21C @ayat
Which design
increased
conversions?
1
2 3
#SMX #21C @ayat
The winner
23.4 %
more	
conversions
2
#SMX #21C @ayat
Why?
Social proof via reviews
placement
Max of 4 benefits of the
product displayed
More options for logical
persona
Specific compatibility info
clearly labeled
2
#SMX #21C @ayat
The Essentials of Multivariate and AB Testing
ü The Basics of AB Testing
ü The Basics of Multivariate Testing
ü What Elements Should You Test In An A/B
Test?
ü AB Testing Best Practices
ü 14 Beginner Mistakes That Will Kill Your A/B
Testing (And What You Can Do About
Them)
invespcro.com/ab
#SMX #21C @ayat
Remember!
• Qualitative data collection is not as simple as it looks.
If not well-planned and executed, can lead to more questions
than answers.
• Design the data collection plan to uncover visitor problems and
issues.
• Each method of data collection uncovers different types of
data. Used together, they lead to greater insight into site
performance and visitor perspectives.
#SMX #21C @ayat
Quantitative data focuses on the “What”
#SMX #21C @ayat
Randomly selecting pages
• Pages with high bounce rate
• Pages with high exit rate
• Pages at the bottom of the funnel
• Pages with high index value
• Let’s test all pages/all the time
#SMX #21C @ayat
Element level testing
#SMX #21C @ayat
Page level testing
#SMX #21C @ayat
Checkout Checkout
Visitor flow testing
#SMX #21C @ayat
Different way of looking at metrics
• Category to product page ratio
• PPER (product page
effectiveness rate)
• Cart abandonment
• Checkout abandonment
• Lost value on a Class of Pages
• (Bounces / Exits)*AOV * 10%
CATEGORY PAGE UV
(PDP UV – PDP ENTRANCES)
*UV - unique views
TOTAL “ADD TO CART” CLICKS
PDP UV
#SMX #21C @ayat
#SMX #21C @ayat
All visitors
Direct
New Returning
Organic
New Returning
CPC
New Returning
Email
New Returning
#SMX #21C @ayat
How do you segment?
• By traffic source
• By visitor behavior
• By outcome
#SMX #21C @ayat
Remember!
• Emotions create preferences which lead to buying decisions
• Visitors perceive the same type of personality characteristics in
websites as they do in people. So, what is your website
personality? Does it lead to rich and powerful mental
representation?
• Your website “narrative” should communicate “why” visitors
should care.
• Analytics supports the why and sometimes can even uncover
why visitors behave a certain way on the website as well.
#SMX #21C @ayat
How can I help you?
@ayat
Ayat @invesp.com
www.invespcro.com
@ayat

More Related Content

Breaking Through: Practical Steps to Follow to Identify What and Where to Test By Ayat Shukairy

  • 1. #SMX #21C @ayat Breaking Through Practical Steps to follow to identify what and where to test
  • 2. #SMX #21C @ayat Breaking Through Practical Steps to follow to identify what and where to test
  • 3. #SMX #21C @ayat • 3,000+ successful tests • 400+ CRO projects • 11 different countries • 65 years of combined experience THE NUMBERS
  • 5. #SMX #21C @ayat 780 Tests/Year
  • 7. #SMX #21C @ayat • Font/color of texts • "lowest price guarantee" vs "110% price match guarantee" • Add to cart button - different designs/sizes • a badge that shows "XXX products sold The old way
  • 9. #SMX #21C @ayat Qualitative data focuses on the “Why”
  • 10. #SMX #21C @ayat Qualitative data • Heuristic Evaluation • Customer voice • Polls • Surveys • 1 on 1 interviews • Customer service feedback • Usability testing • Focus group • User testing • Video Recording
  • 11. #SMX #21C @ayat § Helps you determine the broken areas on the website § Helps you get a full view of what the visitor experiences, especially if you’ve done your homework and considered most the different scenarios a visitor may experience § Gives you a sense of possible testing areas (which need to vetted) § Allows you to then categorize the experience for “to testing ideas” and overall fixes that need to happen (broken areas on a website do not to be “tested”) § Should be conducted with more than one person STOP THE BLEEDING REPORT
  • 12. #SMX #21C @ayat Case study 25% increase in on-site search usage Search users more likely to convert
  • 13. #SMX #21C @ayat Polls and Surveys
  • 14. #SMX #21C @ayat Polls and Surveys §Motivators §Barriers §Hooks §Satisfaction – Overall NPS – Page Level §Demographical information §Feedback §Scoping competitive space
  • 15. #SMX #21C @ayat “What is preventing you from making a purchase?” 1. I cannot find the information I need 2. I don’t know if the item fits my phone 3. I want a different color 4. I cannot find the shipping costs 5. I am just comparing prices Result: understanding of what bothers the visitors + testing ideas
  • 16. #SMX #21C @ayat “What is preventing you from making a purchase?” Result Action I cannot find the information I need Investigate further I don’t know if the item fits my phone Fix right away –Test placement I want a different color Fix right away -Test design/placement I cannot find the shipping costs Test I am just comparing prices Capture email
  • 18. #SMX #21C @ayat Which design increased conversions? 2
  • 20. #SMX #21C @ayat Which design increased conversions? 1 2 3
  • 21. #SMX #21C @ayat The winner 23.4 % more conversions 2
  • 22. #SMX #21C @ayat Why? Social proof via reviews placement Max of 4 benefits of the product displayed More options for logical persona Specific compatibility info clearly labeled 2
  • 23. #SMX #21C @ayat The Essentials of Multivariate and AB Testing ü The Basics of AB Testing ü The Basics of Multivariate Testing ü What Elements Should You Test In An A/B Test? ü AB Testing Best Practices ü 14 Beginner Mistakes That Will Kill Your A/B Testing (And What You Can Do About Them) invespcro.com/ab
  • 24. #SMX #21C @ayat Remember! • Qualitative data collection is not as simple as it looks. If not well-planned and executed, can lead to more questions than answers. • Design the data collection plan to uncover visitor problems and issues. • Each method of data collection uncovers different types of data. Used together, they lead to greater insight into site performance and visitor perspectives.
  • 25. #SMX #21C @ayat Quantitative data focuses on the “What”
  • 26. #SMX #21C @ayat Randomly selecting pages • Pages with high bounce rate • Pages with high exit rate • Pages at the bottom of the funnel • Pages with high index value • Let’s test all pages/all the time
  • 27. #SMX #21C @ayat Element level testing
  • 28. #SMX #21C @ayat Page level testing
  • 29. #SMX #21C @ayat Checkout Checkout Visitor flow testing
  • 30. #SMX #21C @ayat Different way of looking at metrics • Category to product page ratio • PPER (product page effectiveness rate) • Cart abandonment • Checkout abandonment • Lost value on a Class of Pages • (Bounces / Exits)*AOV * 10% CATEGORY PAGE UV (PDP UV – PDP ENTRANCES) *UV - unique views TOTAL “ADD TO CART” CLICKS PDP UV
  • 32. #SMX #21C @ayat All visitors Direct New Returning Organic New Returning CPC New Returning Email New Returning
  • 33. #SMX #21C @ayat How do you segment? • By traffic source • By visitor behavior • By outcome
  • 34. #SMX #21C @ayat Remember! • Emotions create preferences which lead to buying decisions • Visitors perceive the same type of personality characteristics in websites as they do in people. So, what is your website personality? Does it lead to rich and powerful mental representation? • Your website “narrative” should communicate “why” visitors should care. • Analytics supports the why and sometimes can even uncover why visitors behave a certain way on the website as well.
  • 35. #SMX #21C @ayat How can I help you? @ayat Ayat @invesp.com www.invespcro.com @ayat