Nova event-s florida-dboutin
•Download as PPTX, PDF•
1 like•345 views
This talk will dig deep into the process of continuous integration (CI) and the key processes that make up the overall CI process. We will discuss the relationships and process flows between change management, configuration management, and release/build management and how the CI process, when coupled with a solid performance engineering discipline across the product lifecycle, can result in a better user experience for your web & mobile application. We will speak about the entire lifecycle, the "conveyor belt" of the application lifecycle, with concentration on the "Big 3" processes that support the overall CI strategy. We will include a real-world example of how SOASTA uses Jenkins and other open source solutions for its "conveyor belt" and how this process enables SOASTA to complete over 100 product releases in 2014 and still maintain its customer SLA's for its Saas product offerings.
Report
Share
Nova event-s florida-dboutin
- 1. May 28, 2015 South Florida Agile Association – Fort Lauderdale, FL “It’s not called Continuous Integration for Nothing!” Dan Boutin – Senior Product Evangelist dboutin@soasta.com Mobile (404) 304-9529 @DanBoutinSOASTA
- 2. © 2014 SOASTA. All rights reserved. 2 Agenda: • SOASTA Introduction • The need for “Continuous” • The “Big 3” • CI with mobile and performance testing
- 3. © 2014 SOASTA. All rights reserved. 3 100 Billion User Experiences Tested 10 Million Tests Performed 5 Million Failures Avoided Actual CloudTest view
- 4. © 2014 SOASTA. All rights reserved. 4 Agenda: • SOASTA Introduction • The need for “Continuous” • The “Big 3” • CI with mobile and performance testing
- 5. © 2014 SOASTA. All rights reserved. 5
- 6. © 2014 SOASTA. All rights reserved. 6 o Small Batch Sizes
- 7. © 2014 SOASTA. All rights reserved. 7 o Small Batch Sizes o Comprehensive Version Control
- 8. © 2014 SOASTA. All rights reserved. 8 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies
- 9. © 2014 SOASTA. All rights reserved. 9 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation
- 10. © 2014 SOASTA. All rights reserved. 10 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback
- 11. © 2014 SOASTA. All rights reserved. 11 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback o Emphasis on working builds
- 12. © 2014 SOASTA. All rights reserved. 12 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback o Emphasis on working builds o Consistent environments
- 13. © 2014 SOASTA. All rights reserved. 13 o Small Batch Sizes o Comprehensive Version Control o Simple Branch Strategies o Automation o Continuous Feedback o Emphasis on working builds o Consistent environments o Developer – Tester Collaboration
- 14. © 2014 SOASTA. All rights reserved. 14 Agenda: • SOASTA Introduction • The need for “Continuous” • The “Big 3” • CI with mobile and performance testing
- 15. © 2014 SOASTA. All rights reserved. 15
- 16. © 2014 SOASTA. All rights reserved. 16 o It’s not what’s within the silos o It’s about the relationships
- 17. A typical silo’d organization lacks end-to-end integration Performance Engineering & Continuous Integration-Full Lifecycle Operational Configuration Continuous Integration touchpoints ADLC QA/Test Pre- Production Production QA/Test Environment Operational Production Environment Application Development Environment
- 18. Development Test Configuration management Release management Change management Performance management CI Focus Areas – The assembly line Hand-offs Key Processes
- 19. © 2014 SOASTA. All rights reserved. 19 o Bill of Materials (BOM!) o The “Big 3” => Change, Configuration & Release • That is your software assembly line conveyor belt
- 20. © 2014 SOASTA. All rights reserved. 20 • We can only test as fast as the conveyor belt allows. • If the conveyor belt is moving slow, testing is delayed. • The Big 3 drives testing cycle. o Why is CI critical? • Speed • New features, functions, etc. need to get to the customer, ASAP!
- 21. © 2014 SOASTA. All rights reserved. 21 o CI is a key component of Performance Engineering o Performance Engineering spans the silos BIG 3 Testing Application Development Life Cycle IT Operations Capacity Management Performance Management Performance Engineering
- 22. © 2014 SOASTA. All rights reserved. 22 Does it work as designed? Does it perform under extreme circumstances? Approve materials Real Drivers What happens in actual conditions? Assembly Line
- 23. © 2014 SOASTA. All rights reserved. 23 Functional validation Performance, reliability, scalability tests Real Users Build What happens in actual conditions? Unit-tested code
- 24. © 2014 SOASTA. All rights reserved. 24 Agenda: • SOASTA Introduction • The need for “Continuous” • The “Big 3” • CI with mobile and performance testing
- 25. © 2014 SOASTA. All rights reserved. 25
- 26. © 2014 SOASTA. All rights reserved. 26 Jenkins runs any unit tests as usual
- 27. © 2014 SOASTA. All rights reserved. 27 Run iterative load tests via Jenkins Spin up suitable, consistent test environments in the cloud or the lab Jenkins runs any unit tests as usual
- 28. © 2014 SOASTA. All rights reserved. 28 Run iterative load tests via Jenkins Mobile? Run functional automation on real devices for functional & performance validation Spin up suitable, consistent test environments in the cloud or the lab Jenkins runs any unit tests as usual
- 29. © 2014 SOASTA. All rights reserved. 29 Run iterative load tests via Jenkins Daily (or more frequent) updates keep teams quality focused Embedded failure details & bottleneck data isolate issues Mobile? Run functional automation on real devices for functional & performance validation Spin up suitable, consistent test environments in the cloud or the lab Jenkins runs any unit tests as usual
- 30. © 2014 SOASTA. All rights reserved. 30 Run iterative load tests via Jenkins Daily (or more frequent) updates keep teams quality focused Embedded failure details & bottleneck data isolate issues Manage to a performance baseline Mobile? Run functional automation on real devices for functional & performance validation Spin up suitable, consistent test environments in the cloud or the lab Jenkins runs any unit tests as usual
- 31. © 2014 SOASTA. All rights reserved. 31 Use real user data to iterate dev & test plans Run iterative load tests via Jenkins Daily (or more frequent) updates keep teams quality focused Embedded failure details & bottleneck data isolate issues Manage to a performance baseline Mobile? Run functional automation on real devices for functional & performance validation Spin up suitable, consistent test environments in the cloud or the lab Jenkins runs any unit tests as usual
- 33. SOASTA – Branch Build from CloudTest
- 34. SOASTA – Test Results
- 35. SOASTA – Drill Down (Test Failures)
- 36. SOASTA – UI Testing Bundle Results
- 37. SOASTA – UI Testing Failure Drill down into SOASTA CloudTest from Jenkins
- 38. SOASTA – Successful regression test results
- 39. SOASTA – Release Promotion
- 40. © 2014 SOASTA. All rights reserved. 40 o SOASTA’s Process Metrics • 3 Main Products • (2) Builds per day of each product (Dev branch + Customer branch) • 8,000 tests executed per build • For Mobile (TouchTest), Over 300 fully-automated tests each for iOS & Android
- 41. © 2014 SOASTA. All rights reserved. 41 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
- 42. © 2014 SOASTA. All rights reserved. 42 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
- 43. © 2014 SOASTA. All rights reserved. 43 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
- 44. © 2014 SOASTA. All rights reserved. 44 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
- 45. © 2014 SOASTA. All rights reserved. 45 o Understand your requirements o Identify the cloud advantages for you o Automate the obvious and most critical o Connect to a continuous process o Align teams with actionable information
- 46. © 2014 SOASTA. All rights reserved. 46CONFIDENTIAL – Not for Distribution Revenue Brand Competitive advantage
- 47. © 2014 SOASTA. All rights reserved. 47 We will close with some Mobile Trivia @DanBoutinSOASTA
- 48. © 2014 SOASTA. All rights reserved. 48 Questions? Dan Boutin – Senior Product Evangelist dboutin@soasta.com Mobile (404) 304-9529 @DanBoutinSOASTA o SOASTA TouchTest • FREE DownLoad: http://goo.gl/6CErMN o Visit our Blog: http://goo.gl/TA2HKP o Questions? @DanBoutinSOASTA
- 49. May 28, 2015 South Florida Agile Association – Fort Lauderdale, FL “It’s not called Continuous Integration for Nothing!” Dan Boutin – Senior Product Evangelist dboutin@soasta.com Mobile (404) 304-9529 @DanBoutinSOASTA