SlideShare a Scribd company logo

Quality engineering approaches (published)

•
•
Y
Yotam Sharan

Quality Engineering Approaches by Yotam Sharan A lecture given by Yotam on October 8, 2013 at eBay Israel Development Center, as part of eBay's QA Conference.

Related slideshows

Software engineering 12 process quality metrics
Software engineering 12 process quality metricsSoftware engineering 12 process quality metrics
Software engineering 12 process quality metrics
 
Engineering quality assurance manual
Engineering quality assurance manualEngineering quality assurance manual
Engineering quality assurance manual
 
Software engineering 11 software quality assurance plans
Software engineering 11 software quality assurance plansSoftware engineering 11 software quality assurance plans
Software engineering 11 software quality assurance plans
 
1 of 16
QUALITY ENGINEERING APPROACHES Yotam Sharan, Quality Engineering Manager Oct 2013 Ver. 1.1_P
Quality engineering approaches (published)
MYTHS & FACTS QA documentation Run sanity tests Full-regression time COTS test tools Test Automation Mocks and tools No Developers Hours-run of No X-org F.W. QA does
4 Every 30 sec. a woman’s handbag is sold on Every 7 sec. a pair of shoes is sold on Every 1 sec. transaction volume is $2000 on Every 10 sec. an Israeli buys an item on Every 5 sec. an Israeli seller uploads a new listing to Quality Engineering Approaches THE SMALL NUMBERS
55 2B pageviews every day >1TB data turned every second >100PB processed every day 100B DB calls every day >50TB new data every day Quality Engineering Approaches THE BIG NUMBERS
• Buy/Sell anything, anywhere, anyhow • Big Data • 10K app servers, 50MLOC • Multiple releases every day • Services & Systems Dependencies • Partial Test env. Vs. Live Data • Manual Testing 110M ACTIVE USERS 25M SELLERS $175B COMMERCE VOLUME FROM BUSINESS TO TECHNICAL CHALLENGES
Quality Engineering Approaches 7 OPEN-SOURCE TOOLS JUnit / TestNG Selenium/WD SoapUI Hudson & Jenkins Jbehave Jmeter Sonar
• Exploratory testing • Regression testing • Compatibility testing • Automation testing • Unit testing • Code review • Paired testing • Test-driven development • Behavior-driven development • Bug bashes • Capacity planning • Load and Performance testing • Accessibility testing • Mocks and stubs • Analytic testing tools (FindBugs, HTTPWatch, CAL, FireBug, Sonar) TEST INFECTION • Integration testing • E2E testing • Smoke testing • Sprint Planning/sizing • Test case review • Dogfood testing • Sprint demo • Security testing • Usability testing • Definition of Done • CI monitors • Production monitors • Live Site Testing • CrowdTesting
Automation testing Automate everything, regression first. Plan the right automation approach. Business Value Providing thin vertical slices of functionality with business value, every iteration. Continuous Integration Constant integration testing to assure build integrity and quality, every drop. Design for Testability Ensure that it is easier to develop and apply tests for the designed product. Early Start The whole-team approach engages test engineers at early design phases. AGILE DEV & TESTING PRINCIPLES
Everyone needs to let go of titles and self-image and engage in a constant feedback, open, transparent, punishment-free work environment. Developer test and Testers develop Analysts, programmers and PM write test scenarios All engineers help define requirements Quality is everyone’s responsibility QUALITY DRIVEN APPROACH
Sequential Execution Test execution: 7000 minutes Quality Engineering Approaches 11 WEBDRIVER GRID Parallel Execution Test execution: 60 minutes 150 VMs/Browsers on WD Grid
Quality Engineering Approaches 12 WEBDRIVER GRID Selenium Grid Hub Log Analyzer: test logs, stack trace, server logs, config urat’n WD Listeners
Quality Engineering Approaches 13 INTEGRATED QE PROCESSES • Automation Framework for component reuse • Generic test cases for data-driven testing • Fully-integrated env. for anyone to run tests easily
“In God we trust; all others must bring data” – W. Edwards Deming Engineering clockspeed Adopting/Adjusting best practices Instant feedback: CI, Pair-x, RT dashboards, Quality indicators (bugs, code QI) Constant feedback: Stories, Planning, Test results, Reviews, Retrospective, Surveys FEEDBACK & FINE-TUNE
CONTINUOUS INTEGRATION & CODE QUALITY
Oct 2013 We value your thoughts… Win a Prize! http://x.co/QAconf

More Related Content

Quality engineering approaches (published)

  • 1. QUALITY ENGINEERING APPROACHES Yotam Sharan, Quality Engineering Manager Oct 2013 Ver. 1.1_P
  • 3. MYTHS & FACTS QA documentation Run sanity tests Full-regression time COTS test tools Test Automation Mocks and tools No Developers Hours-run of No X-org F.W. QA does
  • 4. 4 Every 30 sec. a woman’s handbag is sold on Every 7 sec. a pair of shoes is sold on Every 1 sec. transaction volume is $2000 on Every 10 sec. an Israeli buys an item on Every 5 sec. an Israeli seller uploads a new listing to Quality Engineering Approaches THE SMALL NUMBERS
  • 5. 55 2B pageviews every day >1TB data turned every second >100PB processed every day 100B DB calls every day >50TB new data every day Quality Engineering Approaches THE BIG NUMBERS
  • 6. • Buy/Sell anything, anywhere, anyhow • Big Data • 10K app servers, 50MLOC • Multiple releases every day • Services & Systems Dependencies • Partial Test env. Vs. Live Data • Manual Testing 110M ACTIVE USERS 25M SELLERS $175B COMMERCE VOLUME FROM BUSINESS TO TECHNICAL CHALLENGES
  • 7. Quality Engineering Approaches 7 OPEN-SOURCE TOOLS JUnit / TestNG Selenium/WD SoapUI Hudson & Jenkins Jbehave Jmeter Sonar
  • 8. • Exploratory testing • Regression testing • Compatibility testing • Automation testing • Unit testing • Code review • Paired testing • Test-driven development • Behavior-driven development • Bug bashes • Capacity planning • Load and Performance testing • Accessibility testing • Mocks and stubs • Analytic testing tools (FindBugs, HTTPWatch, CAL, FireBug, Sonar) TEST INFECTION • Integration testing • E2E testing • Smoke testing • Sprint Planning/sizing • Test case review • Dogfood testing • Sprint demo • Security testing • Usability testing • Definition of Done • CI monitors • Production monitors • Live Site Testing • CrowdTesting
  • 9. Automation testing Automate everything, regression first. Plan the right automation approach. Business Value Providing thin vertical slices of functionality with business value, every iteration. Continuous Integration Constant integration testing to assure build integrity and quality, every drop. Design for Testability Ensure that it is easier to develop and apply tests for the designed product. Early Start The whole-team approach engages test engineers at early design phases. AGILE DEV & TESTING PRINCIPLES
  • 10. Everyone needs to let go of titles and self-image and engage in a constant feedback, open, transparent, punishment-free work environment. Developer test and Testers develop Analysts, programmers and PM write test scenarios All engineers help define requirements Quality is everyone’s responsibility QUALITY DRIVEN APPROACH
  • 11. Sequential Execution Test execution: 7000 minutes Quality Engineering Approaches 11 WEBDRIVER GRID Parallel Execution Test execution: 60 minutes 150 VMs/Browsers on WD Grid
  • 12. Quality Engineering Approaches 12 WEBDRIVER GRID Selenium Grid Hub Log Analyzer: test logs, stack trace, server logs, config urat’n WD Listeners
  • 13. Quality Engineering Approaches 13 INTEGRATED QE PROCESSES • Automation Framework for component reuse • Generic test cases for data-driven testing • Fully-integrated env. for anyone to run tests easily
  • 14. “In God we trust; all others must bring data” – W. Edwards Deming Engineering clockspeed Adopting/Adjusting best practices Instant feedback: CI, Pair-x, RT dashboards, Quality indicators (bugs, code QI) Constant feedback: Stories, Planning, Test results, Reviews, Retrospective, Surveys FEEDBACK & FINE-TUNE
  • 15. CONTINUOUS INTEGRATION & CODE QUALITY
  • 16. Oct 2013 We value your thoughts… Win a Prize! http://x.co/QAconf