SlideShare a Scribd company logo

Roadmap to Enterprise Quality

•Download as PPTX, PDF•
•
Jeff Bramwell
Jeff Bramwell

In this presentation you will learn how Farm Credit Services of America/Frontier Farm Credit transformed their quality practices and tooling to bring visibility and consistency to Enterprise Quality, including: testing as a team approach, creating an automated test architecture, measuring progress with dashboards and standardizing on a set of testing tools.

Related slideshows

Modern CI/CD Pipeline Using Azure DevOps
Modern CI/CD Pipeline Using Azure DevOpsModern CI/CD Pipeline Using Azure DevOps
Modern CI/CD Pipeline Using Azure DevOps
 
Microsoft Tech Series 2019 - Azure DevOps
Microsoft Tech Series 2019 - Azure DevOpsMicrosoft Tech Series 2019 - Azure DevOps
Microsoft Tech Series 2019 - Azure DevOps
 
Using Azure DevOps to continuously build, test, and deploy containerized appl...
Using Azure DevOps to continuously build, test, and deploy containerized appl...Using Azure DevOps to continuously build, test, and deploy containerized appl...
Using Azure DevOps to continuously build, test, and deploy containerized appl...
 
1 of 27
Road Map to Enterprise Quality Jeff Bramwell VP – Enterprise Architecture Farm Credit Services of America Jeff.Bramwell@fcsamerica.com @jbramwell
Agenda • Enterprise Quality Vision • Strategy • Tooling
Enterprise Quality Vision • Why?
Why Enterprise Quality? 125+ Applications & Services 950+ deployments in 2016 11 AppDev teams 1,000+ deployments in 2017
Why Enterprise Quality?
Strategy • Shift Left • People • Process • Tools
Shifting Left
Success Takes People! • Quality Assurance role – not just for testing any more, also… • System health analysis • Visibility & understanding • Peer group • Collaboration • Development • Ability to understand and write code
Focus on Enterprise Quality • Enterprise architect with applied focus on enterprise quality • …and health • …and visibility
QA Role QAEQAD
Lean Quality Delivery (LQD) • Vision • To be the chosen partner • Goals • Functional quality • Structural quality • Process quality • Strategies • Consistent across teams • Tactics • Can vary by team and project • Teams revisit LQD/tactics at the end/beginning of projects
Lean Quality Delivery (LQD) – Cont’d Lean Quality Delivery (LQD) Vision Goal Strategy Tactics Score (1-5) ToBetheChosenPartner FunctionalQuality 1. Delivers the desired value  Story writing and review  UI automation testing  Performance testing (Does it meet SLA needs?)  Service-level testing  Unit testing  Discussion with development teams 2. Deliberate testing plan at the story and project level  Testing starts with the creation of the story and review  Testing is a first-class citizen  Tests should answer questions o What makes the story done? o How is the user going to use it? 3. High reliability, minimal defects  Pairing/Code reviews  Health monitoring (Are we notified when something goes awry?) 4. Delightful user experience  User interviews (in our case, this is typically developers)  Process mapping  Prototyping  Follow-up discussions/surveys StructuralQuality 1. Design for testability  Incorporate design patterns (as appropriate)  TDD/Code reviews  Continuous Integration  Continuous Delivery 2. Security best practices  Security training for developers  Static and dynamic code scanning (for public-/partner-facing apps) 3. Follows project guidance  Application reviews  Reuse of enterprise components  Reference Tech Radar (and follow the process for new technologies)  Create Project Start Architecture document ProcessQuality 1. Responsive support  Proactive support (e.g. Do we send “ack” within 15 minutes?)  Someone takes ownership of issue and “walks it through” 2. Consistent Process  The team can describe the process and how it impacts quality  The team implements the agreed upon tactics  Reviewing and grading of the team this document 3. Predictable Delivery  Measure Cycle Time  Groom Backlog  Manage Budget  Manage Timeline  Automated deployments 4. Visibility & Transparency of  Count defects and kickbacks  Count production-related issues  System Health (for systems under our team’s control) 5. Routinely follows process for feedback and learning  Retrospectives  Demos  5 Why's for production defects
Lean Quality Delivery (LQD) – Cont’d
Process – CI/CD
CD vs. CD • Automation is key • But which is it – Continuous Delivery or Continuous Deployment?See the difference?
CD vs. CD – Cont’d • What’s the difference? • Trust! • It will take time
Tooling
Version Control (VSTS) • Use version control as the single source of truth for Automation Scripts. Dev • VSTS • Visual Studio • GIT • Common lib • Project Repo QA • VSTS • Visual Studio • GIT (SourceTree) • QA • QA Repo Software Development/Quality Tools
Communication Dev • Lead Dev Meeting • AppDev Team Blog • SharePoint • Email Distribution Lists • DOUG QA • QAD Meeting • AppDev Team Blog • QA Website • QA SharePoint • Email distribution lists Communication Points
Learning Dev •On-site Training •Dev Pairing •Pluralsight •Website/Blogs •Tech Conferences •DOUG •.NET/Other User Groups QA •On-site Training •QAD/Dev Pairing •Pluralsight •Website/Blogs •Tech Conferences •Book Club •DOUG Learning Opportunities
Applied Testing (Testing Pyramid) UI Telerik Integration FitNesse Postman jMeter Fortify on Demand (Security) Unit NUnit Jasmine/Karma FakeItEasy NCrunch, Wallaby.js Visibility via Jenkins
Jenkins • Why Jenkins? • Flexibility! • Single Platform • Highly configurable • Automates the build and test (we use it for scheduling/running tests) • Get source code from repository (VSTS) • Dashboard - everyone can see what’s happening • Generate report and notify stakeholders of build status • Plug-in extensibility: Unit test coverage, last success and failure, trends
Jenkin’s Example Dashboard
QA Image Target Install • Telerik Runtime/ Chrome Plugin • Postman(Node/Newman) • jMeter • NUnit Common Software • GIT • Jenkins • Beyond Compare • Notepad++ Group Policies • Auto-login with test account, Disable screen lock, UAC • Browser settings, Screen resolution Base Win 10 Image • Includes (Java, Nuance PDF)
Summary • The problem: No unified approach to quality • Solution: • Adopted shift-left approach to software development and quality • Implemented the testing pyramid • Testing architecture using Jenkins • Increased collaboration and communication between QA and Devs • Impact: • Friction leads to lower adoption rates • We increased test automation from 400 tests suites per day 2,300 • Standardized on a common set of tools • Increased collaboration across teams • Increased confidence in delivering software
What’s next ? • Continue to focus on enterprise health & visibility • Further grow our shift-left practices • Automate our test data scenarios • Grow the quality community • Deliver great, high-quality software and experiences
Questions or Thoughts?

More Related Content

Roadmap to Enterprise Quality

  • 1. Road Map to Enterprise Quality Jeff Bramwell VP – Enterprise Architecture Farm Credit Services of America Jeff.Bramwell@fcsamerica.com @jbramwell
  • 2. Agenda • Enterprise Quality Vision • Strategy • Tooling
  • 4. Why Enterprise Quality? 125+ Applications & Services 950+ deployments in 2016 11 AppDev teams 1,000+ deployments in 2017
  • 6. Strategy • Shift Left • People • Process • Tools
  • 8. Success Takes People! • Quality Assurance role – not just for testing any more, also… • System health analysis • Visibility & understanding • Peer group • Collaboration • Development • Ability to understand and write code
  • 9. Focus on Enterprise Quality • Enterprise architect with applied focus on enterprise quality • …and health • …and visibility
  • 11. Lean Quality Delivery (LQD) • Vision • To be the chosen partner • Goals • Functional quality • Structural quality • Process quality • Strategies • Consistent across teams • Tactics • Can vary by team and project • Teams revisit LQD/tactics at the end/beginning of projects
  • 12. Lean Quality Delivery (LQD) – Cont’d Lean Quality Delivery (LQD) Vision Goal Strategy Tactics Score (1-5) ToBetheChosenPartner FunctionalQuality 1. Delivers the desired value  Story writing and review  UI automation testing  Performance testing (Does it meet SLA needs?)  Service-level testing  Unit testing  Discussion with development teams 2. Deliberate testing plan at the story and project level  Testing starts with the creation of the story and review  Testing is a first-class citizen  Tests should answer questions o What makes the story done? o How is the user going to use it? 3. High reliability, minimal defects  Pairing/Code reviews  Health monitoring (Are we notified when something goes awry?) 4. Delightful user experience  User interviews (in our case, this is typically developers)  Process mapping  Prototyping  Follow-up discussions/surveys StructuralQuality 1. Design for testability  Incorporate design patterns (as appropriate)  TDD/Code reviews  Continuous Integration  Continuous Delivery 2. Security best practices  Security training for developers  Static and dynamic code scanning (for public-/partner-facing apps) 3. Follows project guidance  Application reviews  Reuse of enterprise components  Reference Tech Radar (and follow the process for new technologies)  Create Project Start Architecture document ProcessQuality 1. Responsive support  Proactive support (e.g. Do we send “ack” within 15 minutes?)  Someone takes ownership of issue and “walks it through” 2. Consistent Process  The team can describe the process and how it impacts quality  The team implements the agreed upon tactics  Reviewing and grading of the team this document 3. Predictable Delivery  Measure Cycle Time  Groom Backlog  Manage Budget  Manage Timeline  Automated deployments 4. Visibility & Transparency of  Count defects and kickbacks  Count production-related issues  System Health (for systems under our team’s control) 5. Routinely follows process for feedback and learning  Retrospectives  Demos  5 Why's for production defects
  • 13. Lean Quality Delivery (LQD) – Cont’d
  • 15. CD vs. CD • Automation is key • But which is it – Continuous Delivery or Continuous Deployment?See the difference?
  • 16. CD vs. CD – Cont’d • What’s the difference? • Trust! • It will take time
  • 18. Version Control (VSTS) • Use version control as the single source of truth for Automation Scripts. Dev • VSTS • Visual Studio • GIT • Common lib • Project Repo QA • VSTS • Visual Studio • GIT (SourceTree) • QA • QA Repo Software Development/Quality Tools
  • 19. Communication Dev • Lead Dev Meeting • AppDev Team Blog • SharePoint • Email Distribution Lists • DOUG QA • QAD Meeting • AppDev Team Blog • QA Website • QA SharePoint • Email distribution lists Communication Points
  • 20. Learning Dev •On-site Training •Dev Pairing •Pluralsight •Website/Blogs •Tech Conferences •DOUG •.NET/Other User Groups QA •On-site Training •QAD/Dev Pairing •Pluralsight •Website/Blogs •Tech Conferences •Book Club •DOUG Learning Opportunities
  • 21. Applied Testing (Testing Pyramid) UI Telerik Integration FitNesse Postman jMeter Fortify on Demand (Security) Unit NUnit Jasmine/Karma FakeItEasy NCrunch, Wallaby.js Visibility via Jenkins
  • 22. Jenkins • Why Jenkins? • Flexibility! • Single Platform • Highly configurable • Automates the build and test (we use it for scheduling/running tests) • Get source code from repository (VSTS) • Dashboard - everyone can see what’s happening • Generate report and notify stakeholders of build status • Plug-in extensibility: Unit test coverage, last success and failure, trends
  • 24. QA Image Target Install • Telerik Runtime/ Chrome Plugin • Postman(Node/Newman) • jMeter • NUnit Common Software • GIT • Jenkins • Beyond Compare • Notepad++ Group Policies • Auto-login with test account, Disable screen lock, UAC • Browser settings, Screen resolution Base Win 10 Image • Includes (Java, Nuance PDF)
  • 25. Summary • The problem: No unified approach to quality • Solution: • Adopted shift-left approach to software development and quality • Implemented the testing pyramid • Testing architecture using Jenkins • Increased collaboration and communication between QA and Devs • Impact: • Friction leads to lower adoption rates • We increased test automation from 400 tests suites per day 2,300 • Standardized on a common set of tools • Increased collaboration across teams • Increased confidence in delivering software
  • 26. What’s next ? • Continue to focus on enterprise health & visibility • Further grow our shift-left practices • Automate our test data scenarios • Grow the quality community • Deliver great, high-quality software and experiences