SlideShare a Scribd company logo

Continuous Delivery without Significant Test Automation

•
•
Maaret Pyhäjärvi
Maaret Pyhäjärvi

All organisations aspiring agile don’t have a significant level of automated checks available. Deciding to start adding automation on top of legacy code is a long road to travel and while on route, releases will rely on more manual testing. Meanwhile, automating builds to a single click deploy already enables delivering.  This talk tells a story of going into mindset of continuous delivery of continuous releases to production without the level of test automation often seen as a key practice. You will learn from the changes we went through moving from monthly (scrum) to continuous (kanban) releases as we started delivering value in production in small incremental changes. The talk will help you see one route to a happy agile team with satisfied customers, where the recipe is built more around collaboration and smart exploratory testing than test automation to deliver a steady stream value.

1 of 15
Download to read offline
Continuous Delivery without Significant Test Automation Maaret Pyhäjärvi Email: <maaret@iki.fi> | Twitter: maaretp Maaret Pyhäjärvi Nimeä | Attribution (Finland) http://creativecommons.org/licenses/by/1.0/fi/ http://creativecommons.org/licenses/by/1.0/fi/deed.en
Outline •  Why and how delivering continuously improved our quality even without automation •  How we use Git branches and Jira Kanban to control the delivery contents and schedules for split value without effort estimates •  How continuous delivery changed the way we do exploratory testing with developer-tester collaboration •  How we’ve organized for gradual improvement towards higher degrees of automation that we need in the long run
http://blog.crisp.se/2013/02/05/yassalsundman/continuous-delivery-vs-continuous-deployment every change to the system is releasable can release any version at the push of a button
CHANGE – WHY AND HOW?
Change in 2014 From Scrum+TFS •  Jira Scrum, 30-day sprints •  Negative cycle of feedback from failing with estimates & broken build & hotfixing prod. •  Everyone working on a common deadline (monthly release) + delay if deadline missed To Kanban+Git(TFS) •  Visualising flow and limiting Work in Progress with Jira Kanban, measure lead time •  Every item worked on in it’s own schedule focusing on lead time •  New valuable features to use faster; monitoring production WHY? Availability of Git in TFS without plugins Henri Karhatsu & #NoEstimates Problems with Testing in Scrum
Scheduled release Feature 1 Feature 2 Feature 3 Feature 4 Testing Feature 1 Feature 2 Feature 3 Feature 4 R1 R2 R3 Done includes Tested Schedule over Quality SCRUM + SCHEDULED DELIVERY with continuous integration KANBAN + CONTINUOUS DELIVERY
MECHANISMS OF DELIVERY: GIT & KANBAN
Branches: Master (current in Production), Integration (current in Development), + Feature Branches Build & Deploy from a Branch
DEVELOPER –TESTER COLLABORATION
Things Can Look Different from Different Perspectives
Changes •  Active discussion about schedules and merging, and needs of testing in the branches •  More pairing for testing the features •  More group work on defining the features •  Introducing Flowdock due to increased need to collaborate; integrating logs, emails
ADDINGTEST AUTOMATION
Hardships Faced •  Test automation maintenance & buildup for untested legacy •  Size of work items – branches not short-lived enough •  Developer skills, attitudes and habits – dismantling silo mentality –  Requirements & Design Work –  Testing Work –  Pairing and Mobbing, effective learning •  Production control abilities –  Monitoring –  Releasing for a subset of users •  The right tooling & services
Test Automation •  Performance Tests –  Exploratory Test Automation •  Unit Tests –  Due to structures, more tests on “does mocking work” than the logic we’ve implemented –  One piece of relevant feedback from tests failing –  Tests being dropped with technology change •  Selenium Tests –  Adding GUI tests make more sense (for now) –  Relevant feedback from tests not passing •  Database Monitoring Tests –  Support for exploratory testing & testing in production
Continuous Delivery without Significant Test Automation

More Related Content

Continuous Delivery without Significant Test Automation

  • 1. Continuous Delivery without Significant Test Automation Maaret Pyhäjärvi Email: <maaret@iki.fi> | Twitter: maaretp Maaret Pyhäjärvi Nimeä | Attribution (Finland) http://creativecommons.org/licenses/by/1.0/fi/ http://creativecommons.org/licenses/by/1.0/fi/deed.en
  • 2. Outline •  Why and how delivering continuously improved our quality even without automation •  How we use Git branches and Jira Kanban to control the delivery contents and schedules for split value without effort estimates •  How continuous delivery changed the way we do exploratory testing with developer-tester collaboration •  How we’ve organized for gradual improvement towards higher degrees of automation that we need in the long run
  • 3. http://blog.crisp.se/2013/02/05/yassalsundman/continuous-delivery-vs-continuous-deployment every change to the system is releasable can release any version at the push of a button
  • 4. CHANGE – WHY AND HOW?
  • 5. Change in 2014 From Scrum+TFS •  Jira Scrum, 30-day sprints •  Negative cycle of feedback from failing with estimates & broken build & hotfixing prod. •  Everyone working on a common deadline (monthly release) + delay if deadline missed To Kanban+Git(TFS) •  Visualising flow and limiting Work in Progress with Jira Kanban, measure lead time •  Every item worked on in it’s own schedule focusing on lead time •  New valuable features to use faster; monitoring production WHY? Availability of Git in TFS without plugins Henri Karhatsu & #NoEstimates Problems with Testing in Scrum
  • 6. Scheduled release Feature 1 Feature 2 Feature 3 Feature 4 Testing Feature 1 Feature 2 Feature 3 Feature 4 R1 R2 R3 Done includes Tested Schedule over Quality SCRUM + SCHEDULED DELIVERY with continuous integration KANBAN + CONTINUOUS DELIVERY
  • 8. Branches: Master (current in Production), Integration (current in Development), + Feature Branches Build & Deploy from a Branch
  • 10. Things Can Look Different from Different Perspectives
  • 11. Changes •  Active discussion about schedules and merging, and needs of testing in the branches •  More pairing for testing the features •  More group work on defining the features •  Introducing Flowdock due to increased need to collaborate; integrating logs, emails
  • 13. Hardships Faced •  Test automation maintenance & buildup for untested legacy •  Size of work items – branches not short-lived enough •  Developer skills, attitudes and habits – dismantling silo mentality –  Requirements & Design Work –  Testing Work –  Pairing and Mobbing, effective learning •  Production control abilities –  Monitoring –  Releasing for a subset of users •  The right tooling & services
  • 14. Test Automation •  Performance Tests –  Exploratory Test Automation •  Unit Tests –  Due to structures, more tests on “does mocking work” than the logic we’ve implemented –  One piece of relevant feedback from tests failing –  Tests being dropped with technology change •  Selenium Tests –  Adding GUI tests make more sense (for now) –  Relevant feedback from tests not passing •  Database Monitoring Tests –  Support for exploratory testing & testing in production