Since it's inception over four years ago, OpenStack has become the most popular open source software for building many types of clouds in part due to the flexibility it provides. As more adoption increases, interest has increased in building OpenStack clouds on a highly available control plane infrastructure. In this talk we will provide an introduction to today's OpenStack community and software, then dive deeper into how to build more highly available, scalable OpenStack architectures. - See more at: http://www.percona.com/news-and-events/percona-university-smart-data-raleigh/openstack-toward-more-resilient-cloud#sthash.wicdUMdH.dpuf
What's New in Grizzly & Deploying OpenStack with Puppet
The document outlines the agenda for the May Triangle OpenStack Meetup, including:
- Welcome and introductions starting at 4:30pm
- Two technical talks from 4:45-5:30pm on new features in the Grizzly release of OpenStack and automating OpenStack with Puppet
- An open question and answer forum from 5:30-5:45pm
- Pizza will be served around 5:45pm
Bios of the three meetup organizers are also provided.
OpenStack Cloud Tutorial | What is OpenStack | OpenStack Tutorial | OpenStack...
This Edureka OpenStack Cloud tutorial explains how to setup your own OpenStack Cloud infrastructure and also tells you the different OpenStack Cloud deployment models. It also focuses on how to setup different types of cloud infrastructure according to your needs like storage focused, network focused etc. Below are the topics covered in this tutorial:
1. Cloud Computing Virtualization Concepts
2. Cloud Services
3. Types of Cloud
4. What is Openstack?
5. Openstack Architecture
6. Openstack Community
7. Openstack Cloud Deployment Models
8. Openstack Cloud Design Guide
The document discusses building a public or private cloud using OpenStack. It begins with an introduction to cloud computing and OpenStack. OpenStack is an open source cloud platform that is scalable and used by many large companies. The document then covers use cases for OpenStack like building test environments, provides examples of OpenStack implementations at CERN and Rackspace, and discusses how the group has setup OpenStack and built a sample Rails application using OpenStack Swift for photo storage. It acknowledges the challenges of implementing OpenStack and hopes to continue using it to support building large scale web applications.
2 Day Bootcamp for OpenStack--Cloud Training by Mirantis (Preview)
Mirantis, the Global Engineering Services leader for OpenStack™ presents 2-day Bootcamp for OpenStack
www.mirantis.com/training
This two-day intensive course provides hands-on technical training for OpenStack aimed at system administrators and IT professionals looking to get started on an OpenStack Cloud deployment. Each of the two days will consist of lecture, demos and group exercises. Topics include:
• OpenStack Overview & Architecture: Project goals and use cases, basic operating and deployment principles
• Cloud Usage Patterns: OpenStack codebase overview; creating networks, tenants, roles, troubleshooting; Nexenta Volume Driver
• In Production: Deploying OpenStack for real-world use, and practice of OpenStack operation on multiple nodes
• Swift Object Storage: use cases, architecture, capabilities, configuration, security and deployment
• Advanced Topics: Software Defined Networking, deployment and issues workshop, VMWare/OpenStack comparison
PRE-REQUISITES: Comfortable with Linux CLI, understanding of virtualization & hypervisors, Some experience with Linux networking
All course materials will be provided by Mirantis, including access to shared compute resources for labs. A light breakfast and lunch will be available to all course participants.
Mirantis instructors are active code committers to the OpenStack project, with proven experience building OpenStack clouds in the real world. In parallel to delivering expert training, they also consult for some of the notable global companies using OpenStack – including Cisco, NASA, Dell and Internap.
OpenStack Explained: Learn OpenStack architecture and the secret of a success...
OpenStack can help your business in cutting costs and have a faster time to market. A lot of people are looking at OpenStack as an alternative to VMware and most of the vendors are trying to let you think that visualization is cloud. While Cloud implies a virtualized environment, virtualization is not a cloud.
This ebook will go through the concept of Cloud and help you understand the architecture of OpenStack and its benefits. It also explores DevOps and reveal the "secret ingredient" to have a successful cloud project.
This ebook was created to raise funds for the Nepalese population after the Earthquake in 2015.
(1) The document describes OpenStack networking architecture that scales to over 1,000 servers without using Neutron. It uses a layer 3 spine and leaf topology with no VLANs.
(2) Key components include Nova network which is distributed, NAT services on the edge, and a VIF driver that sets up routing and DHCP for each VM.
(3) Challenges included moving OpenStack targets and an immature Neutron. The architecture provides a solid underlay for SDN and future integration with Neutron.
Apache CloudStack is a mature, easy to deploy IaaS platform. That doesn't mean that it can be done without thought or preparation. Learn how CloudStack can be most efficiently deployed, and the problems to avoid in the process.
About David Nalley
David is a recovering sysadmin with a decade of experience. He’s a committer on the Apache CloudStack (incubating) project, a contributor to the Fedora Project and the Vice President of Infrastructure at the Apache Software Foundation.
I gave this presentation on 5/17 to the New Mexico VMUG in Santa Fe. The presentation provides an overview of OpenStack, what it is (and isn't), and some things you might learn to get started with OpenStack.
Presentation of OpenStack survey to Internet Research Lab at National Taiwan University, Taiwan. OpenStack framework and architecture overview. (ppt slide for download.) Materials collected from various resources, not originally produced by the author.
Briefly explained Nova, Swift, Glance, Keystone, and Quantum.
OpenNebulaConf2015 1.07 Cloud for Scientific Computing @ STFC - Alexander Dibbo
The Science and Technology Facilities Council is a UK Research Council which funds research and provides large facilities to the UK Scientific Community. This includes running a Tier 1 site for the LHC computing project, the JASMIN Super Data Cluster and a number of other HPC and HTC facilities. The Scientific Computing Department at the Rutherford Appleton Laboratory has been developing a cloud for use across both sites of the Department and in the wider scientific community. This is an OpenNebula backed by Ceph block storage. I will give a brief background of the project, describe our set up, some use cases and the work we have done around OpenNebula (including a simplified web front-end and a number of hooks to provide us with traceability). I will also discuss how we are creating an elastic boundary between our HTC batch farm and cloud.
Author Biography
I am a Systems Administrator in the Scientific Computing Department of the UK’s Science and Technology Facilities Council. I work as part of the cloud team and I also work on a number of Grid services including our HTC batch farm for the LHC computing project.
Prior to my position here I worked in IT at a SMB focusing on Storage and Virtualisation, in particular Hyper-V and VMWare.
Successfully Deliver and Operate OpenStack in Production with VMware VIO
This document discusses VMware's approach to integrating their virtualization technologies with OpenStack. It emphasizes that VMware contributes to OpenStack projects while also competing to make VMware the best platform for running OpenStack. Specifically, it promotes using vSphere for compute (Nova), NSX for networking (Neutron), and vSAN for block storage (Cinder), arguing these provide the best features for reliability, ease-of-use, and management. It also describes how VMware technologies can help simplify and improve OpenStack operations through tools like vCenter and vCOps.
VMware Integrated OpenStack (VIO) provides a tightly-integrated product that combines OpenStack APIs with VMware technologies for a less complex build and operation compared to a loosely-integrated framework. VIO includes common OpenStack projects like Nova, Neutron, Cinder, and Glance that are optimized to run on VMware vSphere and integrate with VMware management and automation tools for a unified experience. VIO addresses the challenges of operating OpenStack at scale through this tight integration with VMware technologies and single support contact.
Bridging The Gap: Explaining OpenStack To VMware Administrators
Updated from Kenneth Hui and Scott Lowe's joint talk at the Fall 2013 OpenStack Summit in Hong Kong. This is from a talk given by Cody Bunch and Kenneth Hui at the New England VTUG 2014 Winter Warmer.
OpenStack is an open-source cloud computing platform that manages large pools of compute, storage, and networking resources throughout a datacenter. It includes several independent services like Nova (compute), Neutron (networking), Swift (object storage), and Glance (image service). Hands-on experience with OpenStack can be gained through all-in-one installations or multi-node configurations on physical or virtual machines using various OpenStack distributions from companies like Red Hat, Ubuntu, and Mirantis. Neutron provides virtual networking and integration with technologies like Open vSwitch, namespaces, and plugins to enable multi-tenant isolation.
The document provides information about an OpenStack Meetup happening on October 7th, 2014 in San Antonio, Texas. It includes details about the event location and organizers, as well as an agenda for introductory OpenStack training modules covering an overview of OpenStack, its architecture, and installing OpenStack.
Openstack is an open source cloud computing platform that consists of several independent components that work together to provide infrastructure as a service capabilities. It allows users to provision compute, storage, and networking resources on demand in a self-service manner similar to public cloud providers like AWS. Some key components include Nova for compute, Glance for images, Swift for object storage, Cinder for block storage, Neutron for networking, and Keystone for identity services. Openstack can be used to build public, private, or hybrid clouds and supports a variety of use cases and workloads.
This document provides an overview of Apache CloudStack, an open source cloud computing platform. It describes CloudStack's key characteristics including on-demand self-service, broad network access, resource pooling, rapid elasticity, and API access. It outlines CloudStack's support for different cloud service models including SaaS, PaaS, and IaaS and discusses its hypervisor support, zone, pod, and cluster architecture. The document also summarizes CloudStack's management server, high availability features, networking, security groups, and usage accounting capabilities.
The document provides an agenda and overview of a session on hacking Apache CloudStack. The agenda includes introductions, a session on introducing CloudStack, and a hands-on session with DevCloud. The overview discusses what CloudStack is, how it works as an orchestration platform for IAAS clouds, its architecture and core components, and how users can consume and manage resources through it.
Private clouds are cloud infrastructure that resides within a company's own datacenter and is managed internally. They allow a company to have cloud-like capabilities while maintaining control and security over their own data. Private clouds work by using virtualization and a controller to provision and track physical resources like servers, storage, and networks as more capacity is needed. The document discusses the private cloud platforms Eucalyptus, OpenStack, and CloudStack, comparing their architectures, development histories, and strengths for different use cases.
This document provides an overview of how to create your own cloud using Apache CloudStack. It discusses the key characteristics of clouds, different cloud service and deployment models supported by CloudStack, and the core components that make up a CloudStack deployment including zones, pods, clusters, primary and secondary storage, virtual routers, hypervisors, and the management server. The document also touches on CloudStack's networking, security, high availability, resource allocation, and usage accounting features.
This presentation provides an overview of Apache CloudStack, an open source cloud computing platform. It discusses CloudStack's history and licensing, its ability to provide infrastructure as a service across multiple hypervisors, and how it enables multi-tenancy, high availability, scalability, and resource allocation. Key CloudStack components and concepts are also summarized, such as networking models, security groups, primary and secondary storage, usage tracking, and its management architecture.
This document discusses containerization and the Docker ecosystem. It begins by describing the challenges of managing different software stacks across multiple environments. It then introduces Docker as a solution that packages applications into standardized units called containers that are portable and can run anywhere. The rest of the document covers key aspects of the Docker ecosystem like orchestration tools like Kubernetes and Docker Swarm, networking solutions like Flannel and Weave, storage solutions, and security considerations. It aims to provide an overview of the container landscape and components.
- Alfresco solutions can be provisioned and deployed on Amazon Web Services (AWS) infrastructure as a service (IaaS) cloud platform. This provides advantages like infinite resources, pay as you go pricing, rapid scalability, and provisioning speed.
- AWS services like EC2, S3, and EBS allow deployment of virtual servers and storage. DevOps tools like Puppet can help automate configuration and provisioning.
- Deploying Alfresco on AWS provides opportunities for auto scaling, historical usage statistics from monitoring, and a potential Alfresco admin console for management of the AWS environment. However, challenges include network and storage configuration and automated provisioning of new nodes
OpenStack is an open source cloud computing platform that consists of several components including Keystone for identity, Glance for images, Nova for compute, Cinder for block storage, and Quantum for networking. The document provides an overview of each component, describing their main functions and how they interact through messaging queues like RabbitMQ. It also describes the original "Nova networking" approach and how Quantum improved on this by decoupling logical and physical networking and providing plugins to support technologies like Open vSwitch.
This document summarizes a presentation about the open source CloudStack cloud computing platform. CloudStack provides tools for provisioning and managing virtual infrastructure as a service, including APIs for self-service provisioning, distributed management of compute, storage and networking resources, and high availability features. The presentation outlines CloudStack's history and goals of multi-tenancy, broad hardware support, orchestration of resources behind firewalls, and scalability. It describes key CloudStack components and features such as the management server, domains and users, hypervisor and storage support, resource allocation policies, and networking functionality.
Presented at the CloudStack Silicon Valley User Group in September 2015 at Nuage Networks. Discussed impact of containers, emerging software defined networking platforms, NFV, IPv6 and performance.
Better, faster, cheaper infrastructure with apache cloud stack and riak cs redux
Software is eating infrastructure. Migrating reliability and
scalability responsibilities up the stack from specialized hardware to software, cloud orchestration platforms such as Apache CloudStack (ACS) and object stores such as Riak CS increase the utilization and density of compute and storage resources by dynamically shifting workloads based on demand. Together, these platform can saturate compute and storage of 1000s of commodity hosts with strong operational visibility and end-user self-service.
This presentation explores cloud design strategies to achieve high availability and reliability using commodity components. It then applies these strategies using Apache CloudStack and Riak CS.
Using apache camel for microservices and integration then deploying and managing on Docker and Kubernetes. When we need to make changes to our app, we can use Fabric8 continuous delivery built on top of Kubernetes and OpenShift.
This document discusses integration in the age of DevOps. It describes how microservices help solve the problem of decoupling services and teams to move quickly at scale. Apache Camel is presented as a solution for integration that allows for reliable and distributed integration through mechanisms like messaging. Kubernetes and Docker are discussed as platforms that help develop and run microservices locally and at scale by providing automation, configuration, isolation and service discovery capabilities.
This document provides an overview of migrating applications and workloads to AWS. It discusses key considerations for different migration approaches including "forklift", "embrace", and "optimize". It also covers important AWS services and best practices for architecture design, high availability, disaster recovery, security, storage, databases, auto-scaling, and cost optimization. Real-world customer examples of migration lessons and benefits are also presented.
There is a growing trend today of enterprises leveraging both Amazon Web Services (AWS) and on-premise OpenStack-based private clouds. However, the default networking option in OpenStack remains broken and the plethora of confusing plug-ins makes networking in OpenStack mysterious and difficult to manage.
Enter MidoNet, the open source network virtualization solution from Midokura favored by DevOps cultures in web scale enterprises and service providers around the world. This session will present case studies from several end user deployments, showing how they use MidoNet to build, run and manage large-scale virtual networks in OpenStack clouds. The session will also discuss how transitioning from a public to private cloud enables organizations to accomplish much more with the same resources, without over-simplifying the inherent complexity of running an OpenStack cloud.
Open Clouds: The New Primitives in Enterprise IT & Mobile Networks
This talk was presented in the "101" track at All Things Open 2019.
In this talk, we’ll explore open source projects like OpenStack and Kubernetes that form the basis for many cloud use cases today. This talk will serve as a general introduction to the new “primitives” of IT infrastructure, such as server, VM’s, containers, functions, API’s, etc. By examining the primitives that popular modern tools provide, it's substantially easier to make sense of the overall technology landscape and discern how to make the best technology choices for your use cases. We’ll also take a brief look at open source tools used to interact with open infrastructure, and examine some more concrete use cases describing how enterprise IT and mobile networks are using open source to build their next generation infrastructure.
Open Source in the Era of 5G - All Things Open 2018
1) 5G networks will enable much faster speeds, lower latency, and more connections compared to previous generations of wireless technology.
2) For 5G to be fully realized, network infrastructure will need to become more distributed through small cell sites and edge computing deployments. Open source software and network function virtualization will be critical to manage this complex infrastructure.
3) Initiatives like ONAP are developing open source platforms to automate the deployment and management of virtualized network functions across distributed infrastructure as defined by standards bodies like the O-RAN Alliance.
OpenStack has come a long way since 2010. What started as a collaboration on compute and storage between NASA and Rackspace has changed dramatically and grown into a large, successful open source project that meets the needs of thousands of organizations. But OpenStack hasn’t evolved in a vacuum over the past seven years: the technology landscape around it has been changing as well. Join VMware’s chief OpenStack architect and longtime community member Mark Voelker for a look at the new technology landscape around OpenStack, how we got here, and where we might go next. We’ll discuss how what started as an IaaS platform ending up being a winning platform for Network Functions Virtualization and telco applications, how OpenStack came to be selected as a common underpinning for container orchestration systems like Kubernetes, how OpenStack governance influenced other open source communities, and how OpenStack changed the way companies looked at Open Source. We’ll consider the role IaaS might play in a future that includes options like functions-as-a-service, containers, and the internet of things. We’ll consider OpenStack as a common foundation for a variety of new technologies, and discuss OpenStack’s lasting impact in the cloud ecosystem. We’ll also discuss how OpenStack is changing and adapting to shifts in the technology landscape, both as an open source community and in terms of product offerings. Learn about new interoperability programs targeted at use cases that didn’t exist seven years ago, and new initiatives from the OpenStack technical community and Foundation.
Interoperable Clouds and How to Build (or Buy) Them
This document summarizes a presentation about interoperability between OpenStack and other open source infrastructure projects like Kubernetes. It discusses how the OpenStack Interoperability Working Group defines guidelines for compatibility between different OpenStack distributions. While no distribution will be identical, following these guidelines ensures core capabilities are supported across approved implementations. It encourages selecting technologies based on common interfaces like APIs, Terraform, or Kubernetes rather than any single vendor implementation to avoid lock-in and allow flexibility.
The document summarizes the OpenStack Interoperability Working Group's efforts to promote interoperability across OpenStack distributions and products. It discusses how the group develops guidelines specifying required capabilities and tests. Products must pass these tests to be considered interoperable and qualify for the OpenStack logo program. The guidelines aim to ensure a consistent user experience while allowing flexibility in implementations. The document also outlines the group's governance process and opportunities for participants to provide feedback to help improve interoperability standards over time.
(SCALE 12x) OpenStack vs. VMware - A System Administrator PerspectiveStackStorm
By Dmitri Zimine, CTO of StackStorm (www.stackstorm.com)
SCALE 12x Conference
February 22, 2014
Los Angeles, CA
VMware has achieved broad usage, with some studies indicating that 80% or more of enterprises now use some VMware products. OpenStack, on the other hand, has quickly become the most important OpenSource community since Linux itself.
What’s it like to use OpenStack for virtualization and private cloud? And how does that compare to VMware’s solutions?
CERN is expanding its computing infrastructure to support growing data and computing needs. It is adopting open source tools like Puppet for configuration management and OpenStack for cloud computing. CERN plans to deploy OpenStack into production in 2013 to manage over 15,000 hypervisors and 100,000 VMs across its data centers by 2015, supporting both traditional and cloud-based workflows. This will enable CERN to more efficiently manage resources and better support dynamic workloads and temporary spikes in demand.
The Future of SDN in CloudStack by Chiradeep Vittalbuildacloud
The core of CloudStack networking has always been software-defined. As the networking industry evolves to a software-defined future, CloudStack will have to evolve with it.
The presentation will examine the present state of SDN in CloudStack, look at some industry directions and attempt to predict the evolution of CloudStack with those trends.
Bio
Chiradeep Vittal is a Distinguished Engineer in the Converged Infrastructure Group at Citrix where he has technology leadership responsibilities around Citrix Cloud Platform, Citrix Lifecycle Manager and Citrix Workspace Pod. He is also a Project Management Committee member of the Apache CloudStack Project. At cloud.com (acquired by Citrix), he was a founding engineer, often tasked with the thorny details of virtualized networking and storage. Prior to cloud.com, he worked at several Silicon Valley startups in various architectural roles.
Chiradeep has a B.Tech in Computer Science from IIT, Bombay and a M.Sc from the University of Alberta. He has spoken / presented at several conferences, including CloudStack Collab, LISA, OSCON, ONS, SDN Summit and LinuxCon. His twitter handle is @chiradeep and occasionally blogs at http://cloudierthanthou.wordpress.com
What's New in Grizzly & Deploying OpenStack with PuppetMark Voelker
The document outlines the agenda for the May Triangle OpenStack Meetup, including:
- Welcome and introductions starting at 4:30pm
- Two technical talks from 4:45-5:30pm on new features in the Grizzly release of OpenStack and automating OpenStack with Puppet
- An open question and answer forum from 5:30-5:45pm
- Pizza will be served around 5:45pm
Bios of the three meetup organizers are also provided.
OpenStack Cloud Tutorial | What is OpenStack | OpenStack Tutorial | OpenStack...Edureka!
This Edureka OpenStack Cloud tutorial explains how to setup your own OpenStack Cloud infrastructure and also tells you the different OpenStack Cloud deployment models. It also focuses on how to setup different types of cloud infrastructure according to your needs like storage focused, network focused etc. Below are the topics covered in this tutorial:
1. Cloud Computing Virtualization Concepts
2. Cloud Services
3. Types of Cloud
4. What is Openstack?
5. Openstack Architecture
6. Openstack Community
7. Openstack Cloud Deployment Models
8. Openstack Cloud Design Guide
The document discusses building a public or private cloud using OpenStack. It begins with an introduction to cloud computing and OpenStack. OpenStack is an open source cloud platform that is scalable and used by many large companies. The document then covers use cases for OpenStack like building test environments, provides examples of OpenStack implementations at CERN and Rackspace, and discusses how the group has setup OpenStack and built a sample Rails application using OpenStack Swift for photo storage. It acknowledges the challenges of implementing OpenStack and hopes to continue using it to support building large scale web applications.
2 Day Bootcamp for OpenStack--Cloud Training by Mirantis (Preview)Mirantis
Mirantis, the Global Engineering Services leader for OpenStack™ presents 2-day Bootcamp for OpenStack
www.mirantis.com/training
This two-day intensive course provides hands-on technical training for OpenStack aimed at system administrators and IT professionals looking to get started on an OpenStack Cloud deployment. Each of the two days will consist of lecture, demos and group exercises. Topics include:
• OpenStack Overview & Architecture: Project goals and use cases, basic operating and deployment principles
• Cloud Usage Patterns: OpenStack codebase overview; creating networks, tenants, roles, troubleshooting; Nexenta Volume Driver
• In Production: Deploying OpenStack for real-world use, and practice of OpenStack operation on multiple nodes
• Swift Object Storage: use cases, architecture, capabilities, configuration, security and deployment
• Advanced Topics: Software Defined Networking, deployment and issues workshop, VMWare/OpenStack comparison
PRE-REQUISITES: Comfortable with Linux CLI, understanding of virtualization & hypervisors, Some experience with Linux networking
All course materials will be provided by Mirantis, including access to shared compute resources for labs. A light breakfast and lunch will be available to all course participants.
Mirantis instructors are active code committers to the OpenStack project, with proven experience building OpenStack clouds in the real world. In parallel to delivering expert training, they also consult for some of the notable global companies using OpenStack – including Cisco, NASA, Dell and Internap.
OpenStack Explained: Learn OpenStack architecture and the secret of a success...Giuseppe Paterno'
OpenStack can help your business in cutting costs and have a faster time to market. A lot of people are looking at OpenStack as an alternative to VMware and most of the vendors are trying to let you think that visualization is cloud. While Cloud implies a virtualized environment, virtualization is not a cloud.
This ebook will go through the concept of Cloud and help you understand the architecture of OpenStack and its benefits. It also explores DevOps and reveal the "secret ingredient" to have a successful cloud project.
This ebook was created to raise funds for the Nepalese population after the Earthquake in 2015.
(1) The document describes OpenStack networking architecture that scales to over 1,000 servers without using Neutron. It uses a layer 3 spine and leaf topology with no VLANs.
(2) Key components include Nova network which is distributed, NAT services on the edge, and a VIF driver that sets up routing and DHCP for each VM.
(3) Challenges included moving OpenStack targets and an immature Neutron. The architecture provides a solid underlay for SDN and future integration with Neutron.
Introduction to Apache CloudStack by David Nalleybuildacloud
Apache CloudStack is a mature, easy to deploy IaaS platform. That doesn't mean that it can be done without thought or preparation. Learn how CloudStack can be most efficiently deployed, and the problems to avoid in the process.
About David Nalley
David is a recovering sysadmin with a decade of experience. He’s a committer on the Apache CloudStack (incubating) project, a contributor to the Fedora Project and the Vice President of Infrastructure at the Apache Software Foundation.
I gave this presentation on 5/17 to the New Mexico VMUG in Santa Fe. The presentation provides an overview of OpenStack, what it is (and isn't), and some things you might learn to get started with OpenStack.
Presentation of OpenStack survey to Internet Research Lab at National Taiwan University, Taiwan. OpenStack framework and architecture overview. (ppt slide for download.) Materials collected from various resources, not originally produced by the author.
Briefly explained Nova, Swift, Glance, Keystone, and Quantum.
OpenNebulaConf2015 1.07 Cloud for Scientific Computing @ STFC - Alexander DibboOpenNebula Project
The Science and Technology Facilities Council is a UK Research Council which funds research and provides large facilities to the UK Scientific Community. This includes running a Tier 1 site for the LHC computing project, the JASMIN Super Data Cluster and a number of other HPC and HTC facilities. The Scientific Computing Department at the Rutherford Appleton Laboratory has been developing a cloud for use across both sites of the Department and in the wider scientific community. This is an OpenNebula backed by Ceph block storage. I will give a brief background of the project, describe our set up, some use cases and the work we have done around OpenNebula (including a simplified web front-end and a number of hooks to provide us with traceability). I will also discuss how we are creating an elastic boundary between our HTC batch farm and cloud.
Author Biography
I am a Systems Administrator in the Scientific Computing Department of the UK’s Science and Technology Facilities Council. I work as part of the cloud team and I also work on a number of Grid services including our HTC batch farm for the LHC computing project.
Prior to my position here I worked in IT at a SMB focusing on Storage and Virtualisation, in particular Hyper-V and VMWare.
Successfully Deliver and Operate OpenStack in Production with VMware VIOArraya Solutions
This document discusses VMware's approach to integrating their virtualization technologies with OpenStack. It emphasizes that VMware contributes to OpenStack projects while also competing to make VMware the best platform for running OpenStack. Specifically, it promotes using vSphere for compute (Nova), NSX for networking (Neutron), and vSAN for block storage (Cinder), arguing these provide the best features for reliability, ease-of-use, and management. It also describes how VMware technologies can help simplify and improve OpenStack operations through tools like vCenter and vCOps.
VMware Integrated OpenStack (VIO) provides a tightly-integrated product that combines OpenStack APIs with VMware technologies for a less complex build and operation compared to a loosely-integrated framework. VIO includes common OpenStack projects like Nova, Neutron, Cinder, and Glance that are optimized to run on VMware vSphere and integrate with VMware management and automation tools for a unified experience. VIO addresses the challenges of operating OpenStack at scale through this tight integration with VMware technologies and single support contact.
Bridging The Gap: Explaining OpenStack To VMware AdministratorsKenneth Hui
Updated from Kenneth Hui and Scott Lowe's joint talk at the Fall 2013 OpenStack Summit in Hong Kong. This is from a talk given by Cody Bunch and Kenneth Hui at the New England VTUG 2014 Winter Warmer.
OpenStack is an open-source cloud computing platform that manages large pools of compute, storage, and networking resources throughout a datacenter. It includes several independent services like Nova (compute), Neutron (networking), Swift (object storage), and Glance (image service). Hands-on experience with OpenStack can be gained through all-in-one installations or multi-node configurations on physical or virtual machines using various OpenStack distributions from companies like Red Hat, Ubuntu, and Mirantis. Neutron provides virtual networking and integration with technologies like Open vSwitch, namespaces, and plugins to enable multi-tenant isolation.
The document provides information about an OpenStack Meetup happening on October 7th, 2014 in San Antonio, Texas. It includes details about the event location and organizers, as well as an agenda for introductory OpenStack training modules covering an overview of OpenStack, its architecture, and installing OpenStack.
Openstack is an open source cloud computing platform that consists of several independent components that work together to provide infrastructure as a service capabilities. It allows users to provision compute, storage, and networking resources on demand in a self-service manner similar to public cloud providers like AWS. Some key components include Nova for compute, Glance for images, Swift for object storage, Cinder for block storage, Neutron for networking, and Keystone for identity services. Openstack can be used to build public, private, or hybrid clouds and supports a variety of use cases and workloads.
This document provides an overview of Apache CloudStack, an open source cloud computing platform. It describes CloudStack's key characteristics including on-demand self-service, broad network access, resource pooling, rapid elasticity, and API access. It outlines CloudStack's support for different cloud service models including SaaS, PaaS, and IaaS and discusses its hypervisor support, zone, pod, and cluster architecture. The document also summarizes CloudStack's management server, high availability features, networking, security groups, and usage accounting capabilities.
The document provides an agenda and overview of a session on hacking Apache CloudStack. The agenda includes introductions, a session on introducing CloudStack, and a hands-on session with DevCloud. The overview discusses what CloudStack is, how it works as an orchestration platform for IAAS clouds, its architecture and core components, and how users can consume and manage resources through it.
Private clouds are cloud infrastructure that resides within a company's own datacenter and is managed internally. They allow a company to have cloud-like capabilities while maintaining control and security over their own data. Private clouds work by using virtualization and a controller to provision and track physical resources like servers, storage, and networks as more capacity is needed. The document discusses the private cloud platforms Eucalyptus, OpenStack, and CloudStack, comparing their architectures, development histories, and strengths for different use cases.
This document provides an overview of how to create your own cloud using Apache CloudStack. It discusses the key characteristics of clouds, different cloud service and deployment models supported by CloudStack, and the core components that make up a CloudStack deployment including zones, pods, clusters, primary and secondary storage, virtual routers, hypervisors, and the management server. The document also touches on CloudStack's networking, security, high availability, resource allocation, and usage accounting features.
This presentation provides an overview of Apache CloudStack, an open source cloud computing platform. It discusses CloudStack's history and licensing, its ability to provide infrastructure as a service across multiple hypervisors, and how it enables multi-tenancy, high availability, scalability, and resource allocation. Key CloudStack components and concepts are also summarized, such as networking models, security groups, primary and secondary storage, usage tracking, and its management architecture.
This document discusses containerization and the Docker ecosystem. It begins by describing the challenges of managing different software stacks across multiple environments. It then introduces Docker as a solution that packages applications into standardized units called containers that are portable and can run anywhere. The rest of the document covers key aspects of the Docker ecosystem like orchestration tools like Kubernetes and Docker Swarm, networking solutions like Flannel and Weave, storage solutions, and security considerations. It aims to provide an overview of the container landscape and components.
- Alfresco solutions can be provisioned and deployed on Amazon Web Services (AWS) infrastructure as a service (IaaS) cloud platform. This provides advantages like infinite resources, pay as you go pricing, rapid scalability, and provisioning speed.
- AWS services like EC2, S3, and EBS allow deployment of virtual servers and storage. DevOps tools like Puppet can help automate configuration and provisioning.
- Deploying Alfresco on AWS provides opportunities for auto scaling, historical usage statistics from monitoring, and a potential Alfresco admin console for management of the AWS environment. However, challenges include network and storage configuration and automated provisioning of new nodes
OpenStack is an open source cloud computing platform that consists of several components including Keystone for identity, Glance for images, Nova for compute, Cinder for block storage, and Quantum for networking. The document provides an overview of each component, describing their main functions and how they interact through messaging queues like RabbitMQ. It also describes the original "Nova networking" approach and how Quantum improved on this by decoupling logical and physical networking and providing plugins to support technologies like Open vSwitch.
This document summarizes a presentation about the open source CloudStack cloud computing platform. CloudStack provides tools for provisioning and managing virtual infrastructure as a service, including APIs for self-service provisioning, distributed management of compute, storage and networking resources, and high availability features. The presentation outlines CloudStack's history and goals of multi-tenancy, broad hardware support, orchestration of resources behind firewalls, and scalability. It describes key CloudStack components and features such as the management server, domains and users, hypervisor and storage support, resource allocation policies, and networking functionality.
Presented at the CloudStack Silicon Valley User Group in September 2015 at Nuage Networks. Discussed impact of containers, emerging software defined networking platforms, NFV, IPv6 and performance.
Better, faster, cheaper infrastructure with apache cloud stack and riak cs reduxJohn Burwell
Software is eating infrastructure. Migrating reliability and
scalability responsibilities up the stack from specialized hardware to software, cloud orchestration platforms such as Apache CloudStack (ACS) and object stores such as Riak CS increase the utilization and density of compute and storage resources by dynamically shifting workloads based on demand. Together, these platform can saturate compute and storage of 1000s of commodity hosts with strong operational visibility and end-user self-service.
This presentation explores cloud design strategies to achieve high availability and reliability using commodity components. It then applies these strategies using Apache CloudStack and Riak CS.
Using apache camel for microservices and integration then deploying and managing on Docker and Kubernetes. When we need to make changes to our app, we can use Fabric8 continuous delivery built on top of Kubernetes and OpenShift.
This document discusses integration in the age of DevOps. It describes how microservices help solve the problem of decoupling services and teams to move quickly at scale. Apache Camel is presented as a solution for integration that allows for reliable and distributed integration through mechanisms like messaging. Kubernetes and Docker are discussed as platforms that help develop and run microservices locally and at scale by providing automation, configuration, isolation and service discovery capabilities.
This document provides an overview of migrating applications and workloads to AWS. It discusses key considerations for different migration approaches including "forklift", "embrace", and "optimize". It also covers important AWS services and best practices for architecture design, high availability, disaster recovery, security, storage, databases, auto-scaling, and cost optimization. Real-world customer examples of migration lessons and benefits are also presented.
There is a growing trend today of enterprises leveraging both Amazon Web Services (AWS) and on-premise OpenStack-based private clouds. However, the default networking option in OpenStack remains broken and the plethora of confusing plug-ins makes networking in OpenStack mysterious and difficult to manage.
Enter MidoNet, the open source network virtualization solution from Midokura favored by DevOps cultures in web scale enterprises and service providers around the world. This session will present case studies from several end user deployments, showing how they use MidoNet to build, run and manage large-scale virtual networks in OpenStack clouds. The session will also discuss how transitioning from a public to private cloud enables organizations to accomplish much more with the same resources, without over-simplifying the inherent complexity of running an OpenStack cloud.
Similar to OpenStack: Toward a More Resilient Cloud (20)
Open Clouds: The New Primitives in Enterprise IT & Mobile NetworksMark Voelker
This talk was presented in the "101" track at All Things Open 2019.
In this talk, we’ll explore open source projects like OpenStack and Kubernetes that form the basis for many cloud use cases today. This talk will serve as a general introduction to the new “primitives” of IT infrastructure, such as server, VM’s, containers, functions, API’s, etc. By examining the primitives that popular modern tools provide, it's substantially easier to make sense of the overall technology landscape and discern how to make the best technology choices for your use cases. We’ll also take a brief look at open source tools used to interact with open infrastructure, and examine some more concrete use cases describing how enterprise IT and mobile networks are using open source to build their next generation infrastructure.
Open Source in the Era of 5G - All Things Open 2018Mark Voelker
1) 5G networks will enable much faster speeds, lower latency, and more connections compared to previous generations of wireless technology.
2) For 5G to be fully realized, network infrastructure will need to become more distributed through small cell sites and edge computing deployments. Open source software and network function virtualization will be critical to manage this complex infrastructure.
3) Initiatives like ONAP are developing open source platforms to automate the deployment and management of virtualized network functions across distributed infrastructure as defined by standards bodies like the O-RAN Alliance.
OpenStack & the Evolving Cloud EcosystemMark Voelker
OpenStack has come a long way since 2010. What started as a collaboration on compute and storage between NASA and Rackspace has changed dramatically and grown into a large, successful open source project that meets the needs of thousands of organizations. But OpenStack hasn’t evolved in a vacuum over the past seven years: the technology landscape around it has been changing as well. Join VMware’s chief OpenStack architect and longtime community member Mark Voelker for a look at the new technology landscape around OpenStack, how we got here, and where we might go next. We’ll discuss how what started as an IaaS platform ending up being a winning platform for Network Functions Virtualization and telco applications, how OpenStack came to be selected as a common underpinning for container orchestration systems like Kubernetes, how OpenStack governance influenced other open source communities, and how OpenStack changed the way companies looked at Open Source. We’ll consider the role IaaS might play in a future that includes options like functions-as-a-service, containers, and the internet of things. We’ll consider OpenStack as a common foundation for a variety of new technologies, and discuss OpenStack’s lasting impact in the cloud ecosystem. We’ll also discuss how OpenStack is changing and adapting to shifts in the technology landscape, both as an open source community and in terms of product offerings. Learn about new interoperability programs targeted at use cases that didn’t exist seven years ago, and new initiatives from the OpenStack technical community and Foundation.
Interoperable Clouds and How to Build (or Buy) ThemMark Voelker
This document summarizes a presentation about interoperability between OpenStack and other open source infrastructure projects like Kubernetes. It discusses how the OpenStack Interoperability Working Group defines guidelines for compatibility between different OpenStack distributions. While no distribution will be identical, following these guidelines ensures core capabilities are supported across approved implementations. It encourages selecting technologies based on common interfaces like APIs, Terraform, or Kubernetes rather than any single vendor implementation to avoid lock-in and allow flexibility.
The document summarizes the OpenStack Interoperability Working Group's efforts to promote interoperability across OpenStack distributions and products. It discusses how the group develops guidelines specifying required capabilities and tests. Products must pass these tests to be considered interoperable and qualify for the OpenStack logo program. The guidelines aim to ensure a consistent user experience while allowing flexibility in implementations. The document also outlines the group's governance process and opportunities for participants to provide feedback to help improve interoperability standards over time.
Considerations for Operating An OpenStack CloudMark Voelker
My talk from All Things Open 2014
Over the past four years, OpenStack has become a widely adopted cloud operating system. Cloud computing has made many tasks like creating new servers and networks easy for end users by creating abstractions above the infrastructure. However, cloud operators need to maintain not only the cloud operating system itself, but all of the underpinning systems beneath it. The challenges of managing a set of distributed systems isn’t small, but with proper tooling is well within reach. This talk will discuss considerations for cloud operators such as logging, storage, monitoring, high availability, configuration management with a focus on OpenStack clouds with a focus on open source solutions for common issues encountered when operating an OpenStack cloud. We’ll consider data gathered from the community and discuss “day 1″ and “day 2″ concerns as well as established patterns and technology choices among OpenStack deployers today.
How RPA Help in the Transportation and Logistics Industry.pptxSynapseIndia
Revolutionize your transportation processes with our cutting-edge RPA software. Automate repetitive tasks, reduce costs, and enhance efficiency in the logistics sector with our advanced solutions.
Implementations of Fused Deposition Modeling in real worldEmerging Tech
The presentation showcases the diverse real-world applications of Fused Deposition Modeling (FDM) across multiple industries:
1. **Manufacturing**: FDM is utilized in manufacturing for rapid prototyping, creating custom tools and fixtures, and producing functional end-use parts. Companies leverage its cost-effectiveness and flexibility to streamline production processes.
2. **Medical**: In the medical field, FDM is used to create patient-specific anatomical models, surgical guides, and prosthetics. Its ability to produce precise and biocompatible parts supports advancements in personalized healthcare solutions.
3. **Education**: FDM plays a crucial role in education by enabling students to learn about design and engineering through hands-on 3D printing projects. It promotes innovation and practical skill development in STEM disciplines.
4. **Science**: Researchers use FDM to prototype equipment for scientific experiments, build custom laboratory tools, and create models for visualization and testing purposes. It facilitates rapid iteration and customization in scientific endeavors.
5. **Automotive**: Automotive manufacturers employ FDM for prototyping vehicle components, tooling for assembly lines, and customized parts. It speeds up the design validation process and enhances efficiency in automotive engineering.
6. **Consumer Electronics**: FDM is utilized in consumer electronics for designing and prototyping product enclosures, casings, and internal components. It enables rapid iteration and customization to meet evolving consumer demands.
7. **Robotics**: Robotics engineers leverage FDM to prototype robot parts, create lightweight and durable components, and customize robot designs for specific applications. It supports innovation and optimization in robotic systems.
8. **Aerospace**: In aerospace, FDM is used to manufacture lightweight parts, complex geometries, and prototypes of aircraft components. It contributes to cost reduction, faster production cycles, and weight savings in aerospace engineering.
9. **Architecture**: Architects utilize FDM for creating detailed architectural models, prototypes of building components, and intricate designs. It aids in visualizing concepts, testing structural integrity, and communicating design ideas effectively.
Each industry example demonstrates how FDM enhances innovation, accelerates product development, and addresses specific challenges through advanced manufacturing capabilities.
An invited talk given by Mark Billinghurst on Research Directions for Cross Reality Interfaces. This was given on July 2nd 2024 as part of the 2024 Summer School on Cross Reality in Hagenberg, Austria (July 1st - 7th)
The Rise of Supernetwork Data Intensive ComputingLarry Smarr
Invited Remote Lecture to SC21
The International Conference for High Performance Computing, Networking, Storage, and Analysis
St. Louis, Missouri
November 18, 2021
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Chris Swan
Have you noticed the OpenSSF Scorecard badges on the official Dart and Flutter repos? It's Google's way of showing that they care about security. Practices such as pinning dependencies, branch protection, required reviews, continuous integration tests etc. are measured to provide a score and accompanying badge.
You can do the same for your projects, and this presentation will show you how, with an emphasis on the unique challenges that come up when working with Dart and Flutter.
The session will provide a walkthrough of the steps involved in securing a first repository, and then what it takes to repeat that process across an organization with multiple repos. It will also look at the ongoing maintenance involved once scorecards have been implemented, and how aspects of that maintenance can be better automated to minimize toil.
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...Bert Blevins
Today’s digitally connected world presents a wide range of security challenges for enterprises. Insider security threats are particularly noteworthy because they have the potential to cause significant harm. Unlike external threats, insider risks originate from within the company, making them more subtle and challenging to identify. This blog aims to provide a comprehensive understanding of insider security threats, including their types, examples, effects, and mitigation techniques.
7 Most Powerful Solar Storms in the History of Earth.pdfEnterprise Wired
Solar Storms (Geo Magnetic Storms) are the motion of accelerated charged particles in the solar environment with high velocities due to the coronal mass ejection (CME).
How Social Media Hackers Help You to See Your Wife's Message.pdfHackersList
In the modern digital era, social media platforms have become integral to our daily lives. These platforms, including Facebook, Instagram, WhatsApp, and Snapchat, offer countless ways to connect, share, and communicate.
Details of description part II: Describing images in practice - Tech Forum 2024BookNet Canada
This presentation explores the practical application of image description techniques. Familiar guidelines will be demonstrated in practice, and descriptions will be developed “live”! If you have learned a lot about the theory of image description techniques but want to feel more confident putting them into practice, this is the presentation for you. There will be useful, actionable information for everyone, whether you are working with authors, colleagues, alone, or leveraging AI as a collaborator.
Link to presentation recording and transcript: https://bnctechforum.ca/sessions/details-of-description-part-ii-describing-images-in-practice/
Presented by BookNet Canada on June 25, 2024, with support from the Department of Canadian Heritage.
Support en anglais diffusé lors de l'événement 100% IA organisé dans les locaux parisiens d'Iguane Solutions, le mardi 2 juillet 2024 :
- Présentation de notre plateforme IA plug and play : ses fonctionnalités avancées, telles que son interface utilisateur intuitive, son copilot puissant et des outils de monitoring performants.
- REX client : Cyril Janssens, CTO d’ easybourse, partage son expérience d’utilisation de notre plateforme IA plug & play.
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfNeo4j
Presented at Gartner Data & Analytics, London Maty 2024. BT Group has used the Neo4j Graph Database to enable impressive digital transformation programs over the last 6 years. By re-imagining their operational support systems to adopt self-serve and data lead principles they have substantially reduced the number of applications and complexity of their operations. The result has been a substantial reduction in risk and costs while improving time to value, innovation, and process automation. Join this session to hear their story, the lessons they learned along the way and how their future innovation plans include the exploration of uses of EKG + Generative AI.
Coordinate Systems in FME 101 - Webinar SlidesSafe Software
If you’ve ever had to analyze a map or GPS data, chances are you’ve encountered and even worked with coordinate systems. As historical data continually updates through GPS, understanding coordinate systems is increasingly crucial. However, not everyone knows why they exist or how to effectively use them for data-driven insights.
During this webinar, you’ll learn exactly what coordinate systems are and how you can use FME to maintain and transform your data’s coordinate systems in an easy-to-digest way, accurately representing the geographical space that it exists within. During this webinar, you will have the chance to:
- Enhance Your Understanding: Gain a clear overview of what coordinate systems are and their value
- Learn Practical Applications: Why we need datams and projections, plus units between coordinate systems
- Maximize with FME: Understand how FME handles coordinate systems, including a brief summary of the 3 main reprojectors
- Custom Coordinate Systems: Learn how to work with FME and coordinate systems beyond what is natively supported
- Look Ahead: Gain insights into where FME is headed with coordinate systems in the future
Don’t miss the opportunity to improve the value you receive from your coordinate system data, ultimately allowing you to streamline your data analysis and maximize your time. See you there!
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxSynapseIndia
Your comprehensive guide to RPA in healthcare for 2024. Explore the benefits, use cases, and emerging trends of robotic process automation. Understand the challenges and prepare for the future of healthcare automation
2. • Who is this guy?
• A little background on OpenStack
• Building more resilient clouds
– Withstanding failures
– Quickly recovering from failures
• Questions?
3. @marktvoelker
• OpenStack Architect @ VMware, OpenStack ATC, Ex-StackForge Puppet
core dev, Triangle OpenStack Meetup founder, OS Foundation Member #54
• Fact: can be bribed with doughnuts
• Currently works in VMware’s Software Defined Datacenter R&D group
• In copious (hah!) spare time: OpenStack solutions, Big Data, Massively
Scalable Data Centers, Devops, making sawdust with extreme prejudice
4. • Tech lead, manager, software developer, architect
• Started in OpenStack in 2011 at the Diablo Design Summit
8. • Who is this guy?
• A little background on OpenStack
• Building more resilient clouds
– Withstanding failures
– Quickly recovering from failures
• Questions?
9. “OpenStack is a global collaboration of developers and cloud computing
technologists producing the ubiquitous open source cloud computing
platform for public and private clouds. The project aims to deliver
solutions for all types of clouds by being simple to implement, massively
scalable, and feature rich. The technology consists of a series of
interrelated projects delivering various components for a cloud
infrastructure solution.”
-- openstack.org
Basically, it’s software to run cloud
services—including compute, network,
storage, and security—and the
community behind that software.
13. • IRC Channels and Mailing Lists
• User/Meetup Groups
• Social Networking
– Twitter
– LinkedIn
– Facebook
– Ohloh
• Code in cgit, mirrored on GitHub, Bugs/Milestones in Launchpad
• For now…may move to StoryBoard in future
• Over 20 million lines of code by over 1,419 contributors
• Two Annual Design Summit/Conferences (coinciding roughly
w/releases)
• Want to contribute? Start here.
14. • Don’t be intimidated.
• HolycrapthingsmovereallyreallyfastinOpenStack
• Jump in feet first: be agile and flexible.
• This is going to feel a little different for some of you.
17. Library Projects
Supporting Projects
Documentation
Oslo (common code libraries)
Client libraries
Incubated Projects
(may become core
components in the future)
Designate (DNS service)
Zaqar (queuing service)
Gating Projects
CI & Infrastructure
DevStack (deployment script)
Tempest (integration test)
Barbican (key management)
Manila (shared FS as a
service)
19. • Who is this guy?
• A little background on OpenStack
• Building more resilient clouds
– Withstanding failures
– Quickly recovering from failures
• Questions?
20. What’s a “resilient” cloud?
re·sil·ient
/rəˈzilyənt/
(adjective) Able to withstand or recover quickly from difficult
conditions.
21. • Today we’ll primarily focus on the cloud itself
• Workloads running *in* clouds are another story…but we’ve only
got one hour!
22. 8am: “Uh-oh. Something tells me it’s going to be an interesting day in
the datacenter….”
23. • Hardware Failures
• OpenStack software bugs (yep, those exist)
• Underpinning software failures (database, message queue, etc)
• Operating system failures
• Network/storage/power failures
• Planned maintenance windows
• Hackers and malcontents
• Upgrades
• Automation failures
• “Whoops, did I do that?”
24. Some causes of outages in the past year
….did you plan for these?
CONFIDENTIAL
24
25. Sometimes things break (in *any* system).
25
Withstand what you can. Quickly recover from the rest.
Because you don’t look this cute when your cloud is down.
29. What Does “HA” Mean in an OpenStack Cloud?
CONFIDENTIAL
29
• Compute
• Multiple clusters
• Consider segmenting with Availability Zones, Host Aggregates, etc
• Consider ability to live migrate instances for hypervisor node
maintenance
• Ensure some capacity buffer for maintenance operations
• Storage
• Avoid single points of failure
• Multiple technologies can be used…but each has it’s own limitations
• Don’t think just Cinder here…your Glance backend and compute
storage matter too!
• Network
• Network disruptions will inevitably occur, so plan for them
• Design for control plane disruption (and pick technology accordingly)
• Control Plane
• May depend on the other things above
• Essential to keeping the cloud operational
• Data Plane
• Stuff that workloads running in the cloud depends on
30. High Availability Is Part of the Story….
….we need to think a bit about architecture.
(I’ll use a reference architecture from VMware Integrated OpenStack
as an example)
CONFIDENTIAL
30
32. Notice something?
There’s a lot of stuff in there that isn’t OpenStack, but upon which
OpenStack depends.
CONFIDENTIAL
32
33. VIO Architecture – RabbitMQ
• RabbitMQ is a messaging broker - an
intermediary for messaging. It gives
applications a common platform to send and
receive messages, and the messages a safe
place to live until received.
• RabbitMQ is the default AMQP server used by
OpenStack services (Qpid is also an option,
some support for 0mq). In production clouds,
this should be a highly available infrastructure
component.
• The OpenStack subcomponents (nova-
scheduler to nova-compute, for example)
communicate among themselves using this
hosted message queue service. They also
utilize the hosted Memcached services for
caching authentication tokens etc. As always,
they persist data to the Database.
• Component-to-Component communications
(Nova-> Neutron) is done via REST.
• For more details about the HA implementation
of RabbitMQ, please click here.
34. VIO Architecture – Database
• The database cluster is at the heart of the
infrastructure. Typically MySQL or MariaDB
are used, but other options such as
PostgreSQL are also supported.
• The VIO MariaDB implementation makes use
of a 3-node Galera cluster, which in itself is
Active-Active-Active. However, since some
OpenStack services enforce table locking,
reads and writes are directed to a single
node via the Load Balancers.
• Note that this database is for management
plane data. OpenStack services that store
data as part of their purpose may use
additional DB’s. For example: Ceilometer
may store meter data to MySQL, Mongo,
PostgreSQL, HBase, or DB2.
35. VIO Architecture – Load Balancers
• Most OpenStack Services run on the
Controllers, which are mirrored on each
controller VM and load-balanced. They
are accessible via the internal virtual IP.
• Some of the services, such as the
Dashboard, compute-api, glance-api,
keystone, cinder, neutron and
novncproxy are exposed to the end
users via the load balancer’s public virtual
IP.
• Likewise, the hosted Message Queue
(RabbitMQ) and Memcached services are
also load-balanced between 2 VMs.
• For the Database Service, the load
balancer is configured to use a primary
DB VM. In case of failure it will switch to
one of the two backup DB VMs.
• Load Balancers user here are HAProxy
with Keepalived for high availability
36. Etc, etc, etc
CONFIDENTIAL
36
• There’s a network connecting all that stuff
• It’s running (as virtual machines) on servers which have operating systems
• Things may get wonky if NTP fails and clocks are out of sync
• If DNS can’t resolve, Bad Things ™ will probably happen
37. • Consider whether you want active/active or active/passive
• Setup and tooling differs a bit, but I generally like active/active
• Note that docs.openstack.org has an HA Guide
• Currently undergoing lots of updates…patches welcome!
• Prioritize HA for the control plane
• That also means thinking about your database, network, and RPC bus
• Note: HA == more hardware
• Some components need at least 3 nodes
• Mitigate by virtualizing control plane
38. • Stuff OpenStack needs to run: message brokers
• Check out RabbitMQ clustering and mirrored queues
• Check out Galera for MySQL/MariaDB
• I often see Percona XtraDB in the wild
• Frontend with an HAProxy/Keepalived pair
• Memcached for caching
39. • Don’t do rabbit clustering
over a WAN
• Be aware of the SELECT…
FOR UPDATE issue
40. • Long story short: Neutron and some parts of Nova invoke an SQL
pattern known as “SELECT…FOR UPDATE” which Galera doesn’t
support due to issues with cross-node locking.
• Can cause deadlock-like symptoms due to locks not being
replicated.
• Neutron/nova code being refactored, but will likely not be done
soon.
• Meanwhile: use HAProxy to send writes to a single Galera node
and you should be fine
• With the obvious scalability bottleneck
• More info here, here, & here.
• Thank Jay Pipes of Mirantis & Peter
Boros of Percona for the find!
41. • Pick a highly available storage to back Glance
• Pick a highly available storage backend for Cinder too
– SAN, distributed, software defined, plethora of options here
• Use Keepalived/HAProxy to front-end multiple API servers
• Or another load balancer technology of your choice
• Can be deployed as dedicated nodes for scale, or cohabitate
• Data plane network: DVR & Provider Network Extensions
• Distributed Virtual Routers are a new experimental feature in Juno (not yet
ready for production)
• Please go test it and report/fix bugs!
• Provider networks essentially punt the availability issue to your physical
network
• Allows you to use standard tools like virtual port channels and VRRP
• Also highly performant
42. • Network: consider your backend technology
– Neutron offers a variety of plugins for various open source and vendor-supplied
network technology
– Physical networks need usual redundancy protections
– Overlays are popular for segmentation/isolation; some scale better than others
– Shameless plug: check out VMware NSX which has been used in some very
large OpenStack deployments!
43. • Actually, most of these techniques and technologies are things that
seasoned developers and sysadmins have used before.
• It doesn’t take a genius to learn lessons from the past and apply
them, tweak them, and tune them (but it’s a fair amount of work).
44. Simple Rules of Thumb
Planning for availability can go to extreme levels, so start simple:
– Can I take any one [server | switch | storage unit] out of service in my control
plane and still be operational?
– For all of the above, what’s the impact?
• Performance hit?
• Capacity loss?
• World is broken?!?
– For all the above, how easy is it to reintroduce a repaired/replaced $thing?
• Is there a recovery period that will further impact performance?
• Is it a complex procedure?
• Does the procedure cause more $things to be temporarily unavailable?
– For all of the above, how can I monitor & alert for failure?
Once you have that down, dig deeper to your heart’s content.
CONFIDENTIAL
44
46. Rule 1: Assume You Will Need to Change Stuff
• Change can be a lot of things:
– Hardware or software upgrades/patches/replacements
– Configuration tweaks
– Adding or subtracting capacity
– All systems change over time; OpenStack clouds are no exception.
CONFIDENTIAL
46
“Change in OpenStack?
Yeah, I’m good with
that…”
47. Rule 2: Assume You Can’t Manually Log In To All
The Nodes To Make Those Changes
• OpenStack is a series of cooperating distributed systems
– That means you could (potentially) have a lot of nodes
– Software & config must often be placed on many machines
– Manual changes == slow changes != quick recovery
CONFIDENTIAL
47
“I guess multitasking
only speeds things up
to a certain extent…”
48. Rule 3: Assume You Will Need To Test Stuff
• It’s a good idea to have a small test cloud where you can examine the
impact of changes
• When possible, roll out changes to a portion of your cloud and evaluate
before rolling out the rest
– Note: this means you need tests and monitoring…otherwise you don’t know
what “ok” looks like
CONFIDENTIAL
48
“It’s 3am and I’m still
debugging in
production…maybe I
should have taken the
time to set up a test
environment and
automate some testing
after all…”
50. • Software developers and operators are increasingly the same
people.
– Agile development
– Automate (almost) everything
– Treat config & changes as you would code
– Continuous integration, testing, deployment
– Incremental change & iteration
– Unified tooling & versioning
– Critical approach to working at scale
– Really useful for building resilient clouds
Image courtesy of Rajiv Pant (http://en.wikipedia.org/wiki/File:Devops.png)
51. How Configuration Tools Management Help
• Can orchestrate deployment….and re-deployment.
• Most can idempotently check configuration (no-op if everything is ok)
• Can touch many nodes in parallel
• Can type much faster and more accurately than you
• Are a great way to collaborate amongst teams of operators
• Most have strong communities within the OpenStack universe
– Using a commercial OpenStack? Most vendors are using one of these tools
to deploy and manage your cloud, whether you know it or not.
– Rolling your own? Check out StackForge for tons of Ansible/Puppet/Chef
modules you can use today
• Allow you to manage other things besides OpenStack itself
CONFIDENTIAL
51
54. • I’ve worked on a lot of OpenStack clouds and almost everyone has
their own preferred monitoring toolset.
• One possible exception: almost everybody seems to love Graphite.
• The golden rule is: use the tools that work for you!
• Very often this will be whatever you’re using in the rest of your infrastructure.
• Break it down into at least two buckets:
• Up/down and alerting (ex: Nagios or it’s derivatives…yes, there are
OpenStack plugins out there on NagiosExchange)
• Trending data collection/plotting (ex: collectd/statsd feeding graphite)
• Don’t forget logging
• LogInsight, Logstash, etc.
• Also: use your peers!
• Operators often willing to share, so ask on the openstack-operators list.
55. • Ok, this could take another hour, so I’ll just hit a few highlights…
• Make use of OpenStack’s segregation features
– Availability zones, host aggregates, regions, server groups for compute
– Regions and zones for Swift
• Plan to make infrastructure maintenance less impacting
– Put API servers behind load balancers
– Virtualize tenant-facing parts of the control plane for greater scale and mobilty
– Use host evacuation and live migration to reduce impact
– OpenStack is extremely pluggable…choose your backends wisely
• You should know how to operate, monitor, and troubleshoot them
• Understand how their drivers interact with OpenStack
• You should be comfortable with their failure and recovery modes
Messaging enables software applications to connect and scale. Applications can connect to each other, as components of a larger application. Messaging is asynchronous, decoupling applications by separating sending and receiving data.
Advanced Message Queuing Protocol (AMQP) is an open standard and it enables conforming client applications to communicate with conforming messaging middleware servers
Image courtesy of Rajiv Pant (http://en.wikipedia.org/wiki/File:Devops.png)