SlideShare a Scribd company logo

Devops architecture

•
•
O
Ojasvi Jagtap

Devops architecture involves three main categories of infrastructure: IT infrastructure (version control, issue tracking, etc.), build infrastructure (build servers with access to source code), and test infrastructure (deployment, acceptance, and functional testing). Continuous integration involves automating the integration of code changes, while continuous delivery ensures code is always releasable but actual deployment is manual. Continuous deployment automates deployment so that any code passing tests is immediately deployed to production. The document discusses infrastructure hosting options, automation approaches, common CI/CD workflows, and provides examples of low and medium-cost devops tooling setups using open source and proprietary software.

Related slideshows

Slide DevSecOps Microservices
Slide DevSecOps Microservices Slide DevSecOps Microservices
Slide DevSecOps Microservices
 
DevOps Tutorial For Beginners | DevOps Tutorial | DevOps Tools | DevOps Train...
DevOps Tutorial For Beginners | DevOps Tutorial | DevOps Tools | DevOps Train...DevOps Tutorial For Beginners | DevOps Tutorial | DevOps Tools | DevOps Train...
DevOps Tutorial For Beginners | DevOps Tutorial | DevOps Tools | DevOps Train...
 
Devops Devops Devops, at Froscon
Devops Devops Devops, at FrosconDevops Devops Devops, at Froscon
Devops Devops Devops, at Froscon
 
1 of 35
Devops Architecture By Ojasvi Jagtap
Agenda • Devops Introduction • Devops Infrastructure • Automation • Devops Workflow with CI-CD-CD • Devops Architecture • Cost based analysis
Devops Introduction
Devops • What is DevOps? Devops = Automation + Continuous Improvement + Co-ordination • Devops key work areas are, - Infrastructure creation and management. - Automation - Workflow creation and maintenance - Co-ordination between teams.
Continuous Improvement with Devops Communicate/Co- ordinate with teams Discover bottlenecks/ improvement areas / Optimization areas / refinement scope Fix and automate discovered areas Test automation Integrate with Devops workflow
Infrastructure
Devops Infrastructure Organizations Infrastructure can be categorize into below three high level categories. IT Infrastructure • VCS (Git/Stash) • JIRA • Code review Tool (Gerrit) • Repository Manager(Artifactory) • Authentication Management server(LDAP) • Database • …….. Build Infrastructure • Platform build servers • Servers to run code coverage/ test frameworks which needs access to source code. • …… Test Infrastructure • Deployment test • Acceptance test • Sanity test • Functional test • …..
GIT Jira Code Review Tool Production staging area Artifactory I T I N F R A An IT Infrastructure should have  High Security ( Has sensitive data – > Source code + org data)  100% Uptime  High availability  High Performance  More robust An IT Infrastructure is recommended to be hosted (Considering high data sensitivity)  On Premise ( Procure/lease H/W) virtualized machines.  On Private Cloud On premise Infra Setup
Build Servers Test servers having source code B U I L D I N F R A Build Infrastructure should have  High Security (It host source code)  High Performance  Easily Reproducible  Highly available  High uptime  More robust  Less complex Build Infrastructure can be  Created run time dynamically (Devops says “Treat your infrastructure as a code”)  Or Assigned Dedicated VMs (Traditional Approach - Fixed infa) On Premise + Private Cloud containerized Infra Setup Build Infrastructure recommended to host  On Premise ( Procure/lease H/W) virtualized machines.  On Private Cloud  And use containerization Configuration management tools
Test Servers (Sanity/Functional, Acceptance etc ) Test deployment platform T E S T I N F R A Test Infrastructure has  Less security issue (does not host source code)  High Performance  Easily Reproducible  Highly available  High uptime  More robust  Less complex Test Infrastructure can be  Created run time dynamically (On Demand infra, infrastructure as a code)  Assigned Dedicated VMs (Fixed infa) On Cloud containerized Infra Setup Test Infrastructure recommended to host  On Public Cloud  Use containerization  Use cloud platforms Configuration management tools
Infrastructure Hosting
On Premise Infrastructure (Virtualization) Hypervisors like VMware, HyperV, KVM, Xen etc. Market Share • Magic Quadrant • Figure 1. Magic Quadrant for x86 Server Virtualization Infrastructure • Source: Gartner (August 2016)
Cloud Services (IaaS) Cloud Providers like Amazon Web Services (AWS), Windows Azure, Google Compute Engine, Rackspace Open Cloud, and IBM SmartCloud Enterprise etc Market Share • Magic Quadrant • Figure 1. Magic Quadrant for Cloud Infrastructure as a Service, Worldwide • Source: Gartner (August 2016) Costing Sample
Automation
Automation Success rate of Devops is highly depend on the automation percentage. More you automate, more you go close to Devops…. 1) Automated Infrastructure management. 2) Automated Builds. 3) Automated Tests. (Unit test, Coverage test, Functional test, Sanity tests etc) 4) Automated Deployment.
Workflow
Devops Workflows Continuous Integration Continuous Delivery Continuous Deployment
Continuous Integration (CI)
What is Continuous Integration “Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible “ -Martin Fowler
Why Continuous Integration? • Make the whole process from check-in to release more efficient • Automate more, shorten development cycle, release faster! • Quickly identify check-ins that break builds or cause unit test failures • Find more defects early in the software cycle • Finding and fixing defects early and quickly is inexpensive in the long run • Debugging and investigating defects later on takes time and resources that could otherwise be used on new development
Why Jenkins ?  Its free open source system.  Jenkins is a highly configurable.  Supported by over 400 plugins - SCM, Testing, Notifications, Reporting, Artifact Saving, Triggers, External Integration, Jenkins administration.  There is a large support community and thorough documentation.  There are additional development communities which are providing more support and flexibility to Jenkins.  It’s easy to write plugins  You can fix, if you find anything wrong with Jenkins.  Jenkins work very well with build tools like Ant, Maven, and Gradle etc.  Using different means you can communicate with Jenkins. Web based GUI Command line interface Rest API  You can also get help from third-party vendors to run your Jenkins. They offers extra features to manage your Jenkins. They also provides cloud-based solutions to run a Jenkins instance Cloudbees - http://www.cloudbees.com/ ShiningPanda - https://www.shiningpanda.com/ CI Tools • Jenkins • Teamcity • Travis • Bamboo • Circle CI
Continuous Delivery (CD)
What is Continuous Delivery • Continuous Delivery (CD) is the natural extension of Continuous Integration: an approach in which teams ensure that every change to the system is releasable. In CD teams keep producing valuable software in short cycles and ensure that the software can be reliably released at any time. It is used in software development to automate and improve the process of software delivery. • Continuous Delivery doesn't mean every change is deployed to production all the time, It means that every change is proven to be deployable and of releasable quality at any time.
Core Tools of CI/CD • Use of Source / Version Control • CI Tool like Jenkins • Configuration management tools like Ansible, Vagrant etc • Build and Test Infrastructure • Automated Build • Automated Tests • Automated Deployment
Common CI/CD Workflow
Continuous Deployment (CD)
What is Continuous Deployment • Continuous deployment is the next extension of continuous delivery, every change that passes the automated tests is deployed to production automatically. Continuous deployment should be the goal of most companies that are not constrained by regulatory or other requirements. • Continuous Deployment is a software development practice in which every code change goes through the entire pipeline and is put into production, automatically, resulting in many production deployments every day. • With Continuous Delivery your software is always release-ready, yet the timing of when to push it into production is a business decision, and so the final deployment is a manual step. With Continuous Deployment, any updated working version of the application is automatically pushed to production. Continuous Deployment mandates Continuous Delivery, but the opposite is not required.
Continuous Delivery Vs Deployment  You need to decide scope of your Devops workflow automation.  Continuous deployment is more suitable for the web based application release cycles where it requires to deploy new version as soon as it is available.  For products which does not require frequent deployment, and release time is more than a month, in that case you can automate deploy but can trigger deployment manually.
Architecture Design
Continuous Delivery Pipeline Architecture
Continuous Deployment Pipeline Architecture
Cost Based Tool Analysis
Sample 1 Low Cost Setup using Open source / free Tools Infrastructure • OpenStack / Trial versions of AWS/Azur / google cloud • Dockers, Kubernetes • Vagrant, Ansible • Version Control – GITHub • Code Review – Gerrit • Bug Tracker – MantisBT • Repository management tool – Artifactory trial version, docker hub • Web server – Nginix • Monitoring tools – Elasticsearch, Logstash, and Kibana (ELK stack) , Nagios Build • Build Tools – Make, Ant, Maven • CI Tool – Jenkins • Configuration management tools – Ansible, Vagrant Test • Static Analysis(code quality) - SonarQube • Unit test framework – Junit • Java Code Coverage – JaCoCo • Acceptance test framework(BDD) - Cucumber • Test framework for web applications – Selenium • PaaS for Deployment Testing – Cloud foundry
Sample 2 Medium cost setup combining Open source / free Tools + Proprietary Tools Infrastructure • lease/procure hardware •VMWare,HyperV,KVM,XEN •AWS, Azur, Google – licensed version •Dockers, Kubernetes •Vagrant, Ansible •Version Control – Setup Git server •Code Review – Gerrit •Bug Tracker – Jira •Repository management tool – Artifactory licensed version •Web server – Nginix •Monitoring tools – Elasticsearch, Logstash, and Kibana (ELK stack) , Nagios, splunk Build •Build Tools – Make, Ant, Maven •CI Tool – Jenkins/ CloudBee Jenkins •Configuration management tools – Ansible, Puppet, Chef •Provisioning tool- Vagrant Test •Static Analysis(code quality) - SonarQube •Unit test framework – Junit •Java Code Coverage – JaCoCo, clover •Acceptance test framework(BDD) - Cucumber •Test framework for web applications – Selenium •PaaS for Deployment Testing – Cloud foundry
Thank You

More Related Content

Devops architecture

  • 2. Agenda • Devops Introduction • Devops Infrastructure • Automation • Devops Workflow with CI-CD-CD • Devops Architecture • Cost based analysis
  • 4. Devops • What is DevOps? Devops = Automation + Continuous Improvement + Co-ordination • Devops key work areas are, - Infrastructure creation and management. - Automation - Workflow creation and maintenance - Co-ordination between teams.
  • 5. Continuous Improvement with Devops Communicate/Co- ordinate with teams Discover bottlenecks/ improvement areas / Optimization areas / refinement scope Fix and automate discovered areas Test automation Integrate with Devops workflow
  • 7. Devops Infrastructure Organizations Infrastructure can be categorize into below three high level categories. IT Infrastructure • VCS (Git/Stash) • JIRA • Code review Tool (Gerrit) • Repository Manager(Artifactory) • Authentication Management server(LDAP) • Database • …….. Build Infrastructure • Platform build servers • Servers to run code coverage/ test frameworks which needs access to source code. • …… Test Infrastructure • Deployment test • Acceptance test • Sanity test • Functional test • …..
  • 8. GIT Jira Code Review Tool Production staging area Artifactory I T I N F R A An IT Infrastructure should have  High Security ( Has sensitive data – > Source code + org data)  100% Uptime  High availability  High Performance  More robust An IT Infrastructure is recommended to be hosted (Considering high data sensitivity)  On Premise ( Procure/lease H/W) virtualized machines.  On Private Cloud On premise Infra Setup
  • 9. Build Servers Test servers having source code B U I L D I N F R A Build Infrastructure should have  High Security (It host source code)  High Performance  Easily Reproducible  Highly available  High uptime  More robust  Less complex Build Infrastructure can be  Created run time dynamically (Devops says “Treat your infrastructure as a code”)  Or Assigned Dedicated VMs (Traditional Approach - Fixed infa) On Premise + Private Cloud containerized Infra Setup Build Infrastructure recommended to host  On Premise ( Procure/lease H/W) virtualized machines.  On Private Cloud  And use containerization Configuration management tools
  • 10. Test Servers (Sanity/Functional, Acceptance etc ) Test deployment platform T E S T I N F R A Test Infrastructure has  Less security issue (does not host source code)  High Performance  Easily Reproducible  Highly available  High uptime  More robust  Less complex Test Infrastructure can be  Created run time dynamically (On Demand infra, infrastructure as a code)  Assigned Dedicated VMs (Fixed infa) On Cloud containerized Infra Setup Test Infrastructure recommended to host  On Public Cloud  Use containerization  Use cloud platforms Configuration management tools
  • 12. On Premise Infrastructure (Virtualization) Hypervisors like VMware, HyperV, KVM, Xen etc. Market Share • Magic Quadrant • Figure 1. Magic Quadrant for x86 Server Virtualization Infrastructure • Source: Gartner (August 2016)
  • 13. Cloud Services (IaaS) Cloud Providers like Amazon Web Services (AWS), Windows Azure, Google Compute Engine, Rackspace Open Cloud, and IBM SmartCloud Enterprise etc Market Share • Magic Quadrant • Figure 1. Magic Quadrant for Cloud Infrastructure as a Service, Worldwide • Source: Gartner (August 2016) Costing Sample
  • 15. Automation Success rate of Devops is highly depend on the automation percentage. More you automate, more you go close to Devops…. 1) Automated Infrastructure management. 2) Automated Builds. 3) Automated Tests. (Unit test, Coverage test, Functional test, Sanity tests etc) 4) Automated Deployment.
  • 17. Devops Workflows Continuous Integration Continuous Delivery Continuous Deployment
  • 19. What is Continuous Integration “Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible “ -Martin Fowler
  • 20. Why Continuous Integration? • Make the whole process from check-in to release more efficient • Automate more, shorten development cycle, release faster! • Quickly identify check-ins that break builds or cause unit test failures • Find more defects early in the software cycle • Finding and fixing defects early and quickly is inexpensive in the long run • Debugging and investigating defects later on takes time and resources that could otherwise be used on new development
  • 21. Why Jenkins ?  Its free open source system.  Jenkins is a highly configurable.  Supported by over 400 plugins - SCM, Testing, Notifications, Reporting, Artifact Saving, Triggers, External Integration, Jenkins administration.  There is a large support community and thorough documentation.  There are additional development communities which are providing more support and flexibility to Jenkins.  It’s easy to write plugins  You can fix, if you find anything wrong with Jenkins.  Jenkins work very well with build tools like Ant, Maven, and Gradle etc.  Using different means you can communicate with Jenkins. Web based GUI Command line interface Rest API  You can also get help from third-party vendors to run your Jenkins. They offers extra features to manage your Jenkins. They also provides cloud-based solutions to run a Jenkins instance Cloudbees - http://www.cloudbees.com/ ShiningPanda - https://www.shiningpanda.com/ CI Tools • Jenkins • Teamcity • Travis • Bamboo • Circle CI
  • 23. What is Continuous Delivery • Continuous Delivery (CD) is the natural extension of Continuous Integration: an approach in which teams ensure that every change to the system is releasable. In CD teams keep producing valuable software in short cycles and ensure that the software can be reliably released at any time. It is used in software development to automate and improve the process of software delivery. • Continuous Delivery doesn't mean every change is deployed to production all the time, It means that every change is proven to be deployable and of releasable quality at any time.
  • 24. Core Tools of CI/CD • Use of Source / Version Control • CI Tool like Jenkins • Configuration management tools like Ansible, Vagrant etc • Build and Test Infrastructure • Automated Build • Automated Tests • Automated Deployment
  • 27. What is Continuous Deployment • Continuous deployment is the next extension of continuous delivery, every change that passes the automated tests is deployed to production automatically. Continuous deployment should be the goal of most companies that are not constrained by regulatory or other requirements. • Continuous Deployment is a software development practice in which every code change goes through the entire pipeline and is put into production, automatically, resulting in many production deployments every day. • With Continuous Delivery your software is always release-ready, yet the timing of when to push it into production is a business decision, and so the final deployment is a manual step. With Continuous Deployment, any updated working version of the application is automatically pushed to production. Continuous Deployment mandates Continuous Delivery, but the opposite is not required.
  • 28. Continuous Delivery Vs Deployment  You need to decide scope of your Devops workflow automation.  Continuous deployment is more suitable for the web based application release cycles where it requires to deploy new version as soon as it is available.  For products which does not require frequent deployment, and release time is more than a month, in that case you can automate deploy but can trigger deployment manually.
  • 32. Cost Based Tool Analysis
  • 33. Sample 1 Low Cost Setup using Open source / free Tools Infrastructure • OpenStack / Trial versions of AWS/Azur / google cloud • Dockers, Kubernetes • Vagrant, Ansible • Version Control – GITHub • Code Review – Gerrit • Bug Tracker – MantisBT • Repository management tool – Artifactory trial version, docker hub • Web server – Nginix • Monitoring tools – Elasticsearch, Logstash, and Kibana (ELK stack) , Nagios Build • Build Tools – Make, Ant, Maven • CI Tool – Jenkins • Configuration management tools – Ansible, Vagrant Test • Static Analysis(code quality) - SonarQube • Unit test framework – Junit • Java Code Coverage – JaCoCo • Acceptance test framework(BDD) - Cucumber • Test framework for web applications – Selenium • PaaS for Deployment Testing – Cloud foundry
  • 34. Sample 2 Medium cost setup combining Open source / free Tools + Proprietary Tools Infrastructure • lease/procure hardware •VMWare,HyperV,KVM,XEN •AWS, Azur, Google – licensed version •Dockers, Kubernetes •Vagrant, Ansible •Version Control – Setup Git server •Code Review – Gerrit •Bug Tracker – Jira •Repository management tool – Artifactory licensed version •Web server – Nginix •Monitoring tools – Elasticsearch, Logstash, and Kibana (ELK stack) , Nagios, splunk Build •Build Tools – Make, Ant, Maven •CI Tool – Jenkins/ CloudBee Jenkins •Configuration management tools – Ansible, Puppet, Chef •Provisioning tool- Vagrant Test •Static Analysis(code quality) - SonarQube •Unit test framework – Junit •Java Code Coverage – JaCoCo, clover •Acceptance test framework(BDD) - Cucumber •Test framework for web applications – Selenium •PaaS for Deployment Testing – Cloud foundry