AtlasCamp 2013: Show Me Number! Automated Browser Performance Testing
•
0 likes•1,018 views
This document summarizes a presentation on automated browser performance testing. It discusses how browser performance is influenced by many factors like networks, parsing time, JavaScript execution and reflows. It also describes the JIRA browser performance test suite that was built using page objects, a testing framework, plugins for integration, a database for results and charts for visualization. The test suite automates interactions and measures performance baselines over time to catch regressions. Key techniques discussed include batching, predictive fetching and caching to optimize performance.
Report
Share
AtlasCamp 2013: Show Me Number! Automated Browser Performance Testing
- 2. Show Me the Numbers Automated Browser Performance Testing Chris Mountford Senior Software Developer, JIRA Atlassian @chromosundrift 2
- 10. Chris Mountford Inline Edit Search Detail View 4
- 11. Chris Mountford Inline Edit Search Detail View 4
- 13. Browser Performance Testing “of specific interactions for a single user Measuring the perceived performance ” 6
- 14. Browser Performance Testing “of specific interactions for a single user Measuring the perceived performance perceived ” 7
- 15. Outline • Intro • Optimisation Techniques • JavaScript Renaissance Engineering • Perception • Performance Factors • Gotchas • Take Aways • JIRA Perf Tests 8
- 16. Outline • Intro • Optimisation Techniques • JavaScript Renaissance Engineering • Perception • Performance Factors • Gotchas • Take Aways • JIRA Perf Tests 8
- 19. Browser Wars 10
- 22. V8 Firebug 10
- 25. 11
- 26. 12
- 27. 13
- 28. “Why do we split everything into so many files?” 13
- 29. “Why do we split everything into so many files?” “Performance.” 13
- 30. 14
- 31. “Why do we merge all the files into large batches?” 14
- 32. “Why do we merge all the files into large batches?” “Performance.” 14
- 33. 15
- 34. “Why...” 15
- 37. “ Show me the numbers! ” – Not Jacky Chan Hey it’s Jacky 16
- 38. “ If all we have are opinions, let’s go with mine. ” If we have data, let’s look at data. Netscape CEO – Jim Barksdale 17
- 39. Factors That Influence Browser Performance sad panda
- 40. Mobile Web • Everything is the same but worse • network / radio power mode steps, sleep, etc. • cpu / battery impact • memory • standards compliance • tooling 19
- 41. Network Time 20
- 42. Parse Time 21
- 43. JavaScript Execution • JS Engines • native bind(), map() etc • DOM Manipulation • events • JQuery • Sizzle selectors • micro-optimisation ☠ • Memory leaks 22
- 44. Blocking 23
- 46. Top Tools Chrome DevTools Firebug Lite dynaTrace 24
- 47. Reflow
- 48. 26
- 50. Will It Reflow? inspired by @stoyanstefanov 27
- 51. Will It Reflow? inspired by @stoyanstefanov 27
- 52. Yes! Will It Reflow? inspired by @stoyanstefanov 27
- 53. But wait, there’s more! • CSS Transforms • Hardware acceleration? • Data weight • Concurrent request count • Inline Base64 Encoded resources 28
- 54. But wait, there’s more! • CSS Transforms • Hardware acceleration? • Data weight • Concurrent request count • Inline Base64 Encoded resources 28
- 55. Designing the JIRA Browser Performance Test Suite Sweet!
- 56. JIRA Browser Perf Tests 30
- 57. JIRA Browser Perf Tests Page Objects 30
- 58. JIRA Browser Perf Tests Page Objects Soke Framework 30
- 59. JIRA Browser Perf Tests Page Objects Soke Framework Bamboo 30
- 60. JIRA Browser Perf Tests Page Objects Soke Framework Plugin Bamboo 30
- 61. JIRA Browser Perf Tests Page Objects Soke Framework Plugin DB Bamboo 30
- 62. JIRA Browser Perf Tests Hacking Page Objects Soke Framework Plugin DB Bamboo 30
- 63. JIRA Browser Perf Tests Koto Charts Hacking Page Objects Soke Framework Plugin DB Bamboo 30
- 64. JIRA Browser Perf Tests Koto Charts Psycho Events Hacking Page Objects Soke Framework Plugin DB Bamboo 30
- 65. JIRA Browser Perf Tests Koto Charts Psycho Events OnDemand Analytics Hacking Page Objects Soke Framework Plugin DB Bamboo 30
- 66. JIRA Browser Perf Tests Baselines Koto Charts Psycho Events OnDemand Analytics Hacking Page Objects Soke Framework Plugin DB Bamboo 30
- 67. JIRA Browser Perf Tests Baselines Koto Charts Vigilance Psycho Events OnDemand Analytics Hacking Page Objects Soke Framework Plugin DB Bamboo 30
- 68. Soke Interactions • Define Timed Sections • Number of Repetitions • Personas 31
- 69. Soke Interactions • Define Timed Sections • Number of Repetitions • Personas Big ups to Atlassian PerfEng Team 31
- 70. Soke Interactions in Scala 32
- 71. 33
- 72. 33
- 73. 34
- 74. Optimisation Techniques • Batching • Atlassian Plugins, No Worries • Predictive Fetching • Guided by Analytics • Caching • Watch for memory leaks 35
- 75. “ I don’t know... it just feels slow. ” – a JIRA Product Manager Show me the numbers! 36
- 76. Perception is Reality No, it only seems that way
- 77. 38
- 78. Psycho Latency • Ready for Action • When does the User believe the app is waiting for her? • Manual “psycho” event placement • and maintenance! • Yardstick tuning against a real system • Optimisation Example: Execution Reordering 39
- 80. click( )
- 82. Selenium / Webdriver • Chromedriver implemented by Chromium team 42
- 83. Selenium / Webdriver • Chromedriver implemented by Chromium team • mouse click on geometric centre of target 42
- 84. Selenium / Webdriver • Chromedriver implemented by Chromium team • mouse click on geometric centre of target • W3C Webdriver wire protocol draft 42
- 85. Selenium / Webdriver • Chromedriver implemented by Chromium team • mouse click on geometric centre of target • W3C Webdriver wire protocol draft • OK Browsers, follow the standard! 42
- 86. Selenium / Webdriver • Chromedriver implemented by Chromium team • mouse click on geometric centre of target • W3C Webdriver wire protocol draft • OK Browsers, follow the standard! 42
- 88. Selenium / Webdriver • Windows and IE process control 43
- 89. Selenium / Webdriver • Windows and IE process control • Browser auto-upgrades 43
- 90. Selenium / Webdriver • Windows and IE process control • Browser auto-upgrades • Changing what you measure 43
- 91. Selenium / Webdriver • Windows and IE process control • Browser auto-upgrades • Changing what you measure • Prefetch cache benefit in JIRA 6.0 43
- 92. Take Aways • Lots of factors • Measure perceived reality • Be holistic, quantitative, beware micro-optimisation trap • Monitor and Maintain your performance suite • Use Chrome DevTools etc. 44
- 93. Thank you!