SlideShare a Scribd company logo

Building FOSS clouds

•Download as PPT, PDF•
•
Sebastien Goasguen
Sebastien Goasguen

The document discusses building open source clouds using Apache CloudStack, an open source cloud computing platform. It provides an overview of CloudStack's architecture and features, including its flexible IaaS capabilities. The presentation aims to introduce developers to CloudStack and various tools for building and automating open source clouds.

Related slideshows

CloudStack Architecture
CloudStack ArchitectureCloudStack Architecture
CloudStack Architecture
 
Cloudstack vs Openstack
Cloudstack vs OpenstackCloudstack vs Openstack
Cloudstack vs Openstack
 
Who carries your container? Zun or Magnum?
Who carries your container? Zun or Magnum?Who carries your container? Zun or Magnum?
Who carries your container? Zun or Magnum?
 
1 of 41
Building FOSS Clouds Sebastien Goasguen, Kris Buytaert LinuxCon EU, Nov 6th Barcelona, Spain Slides at: slideshare.net/sebastiengoasguen
Speakers… • Joe stayed home to deal with the release, • Kris Buytaert stepped in:
Tutorial Outline • 10:10 – 10:45 FOSS Clouds and CloudStack intro • 10:45 – 11:00 DevCloud demo • 11:00 – 11:50 FOSS Cloud Automation, DevOps configuration and monitoring tools
Define:tutorial • A tutorial is a method of transferring knowledge and may be used as a part of a learning process. More interactive and specific than a book or a lecture…<snip> • Therefore:
DevCloud • A CloudStack sandbox – Pass the USB keys around and get the Vbox appliance devcloud.ova file • Import the appliance in VirtualBox – Play with CloudStack – http://wiki.cloudstack.org/display/COMM/DevCloud – https://cwiki.apache.org/confluence/display/CLOUDSTA CK/CloudStack+devcloud+environment+setup
Objectives • Learn the basic architecture of a cloud • Learn a few new tools • Explore CloudStack via its sandbox • Discover some concepts of automation • Get home with some ideas to try out • Mileage may vary
NIST Definition of Cloud • "cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.“ – NIST
NIST Service/Deployment Models • Service Models: • IaaS (e.g EC2, rackspace) • PaaS (e.g GAE, Azure) • SaaS (e.g Gapps) • Deployment model: • Private cloud • Public cloud • Hybrid cloud (Private than can use Public)
NIST Key Characteristics of Cloud • Elasticity • Expands and Shrinks as needed • E.g Elastic Load Balancing, Elastic Map Reduce • Measured Service • Utility Pricing. You pay for what you use • E.g $0.08 for one hour of small instance • On-demand • You get it when you ask for it • E.g One api call and you get your machine, ~no queue
“Guidelines” • Virtualization and automation are key enablers of the cloud characteristics • Build your infrastructure to enable on- demand elastic provisioning of –servers, storage, network – • Setup a PaaS that suits you and your applications • Then move to continuous delivery of applications …
OSS SaaS…and more
OSS PaaS…
OSS IaaS…
IaaS challenges • Not out of the box by itself. • Need a farm of hypervisors – Xen, KVM, Vmware… • Need storage – For image catalogue – For volume/snapshot management • Need flexible network that can be configured on-demand – VLANS, no VLANS, existing net infra, SDN…
Let’s make Red Hat happy
Announcing Apache CloudStack 4.0
Coming at 12:00 EST • Apache CloudStack 4.0 • Check out the testing procedure: – https://cwiki.apache.org/CLOUDSTACK/cloudstack -40-test-procedure.html • Or watch my amazing screencast: – http://vimeo.com/52150218
A bit of History • Original company VMOPs (2008) – Founded by Sheng Liang former lead dev on JVM • Open source (GPLv3) as CloudStack • Acquired by Citrix (July 2011) • Relicensed under ASL v2 April 3, 2012 • Accepted as Apache Incubating Project April 16, 2012 • First Apache (ACS 4.0) release expected now
Apache Software Foundation
Apache Process • 100% community driven • New ideas, decisions only taken on mailing lists. Votes taken by community • Project led by Project Management Committee (PMC): – http://www.apache.org/dev/pmc.html – http://incubator.apache.org/guides/committer.html • Non committers get invited as committers: – http://community.apache.org/newcommitter.html
Apache Processes
Contributions Examples: Sungard: Announced that 6 developers were joining the Apache project Schuberg Philis: Big contribution in building/packaging and Nicira support Go Daddy: Early proto of Maven building Caringo: Support for own object store Basho: Devcloud packaging
A Very Flexible IaaS Platform Compute Hypervisor Storage Block & Object Primary Storage Secondary Storage Network Network & Network Services http://www.slideshare.net/cloudstack/cloudstack-architecture
Architecture / Language • Java application • Tomcat6, Axis2, Maven build + ant – Ant going away in 4.1 • Collaboration Conference, Nov 30th -Dec 2nd – http://collab12.cloudstack.org/
Build and Run in 4.1 • git clone https://git-wip- us.apache.org/repos/asf/incubat or-cloudstack.git • mvn clean • mvn install • mvn –P developer –pl developer –Ddeploydb • mvn –pl :cloud-client-ui jetty:run
Cloud Interactions OVM Cluster Primary Storage Ap i Xen vcenter vcenter Monitoring Primary CS API Clu I ste AP vSphere Cluster Storage r M n ter End End gm vCe t User UI User UI Primary XS Cluster Storage Admin Admin Clustered Clustered CloudStack XAPI UI UI CloudStack Domain Domain CS Admin & CloudStack CloudStack CloudStack CloudStack End-user API Primary Admin Admin UI UI Management Management JSON KVM Cluster Storage Server Server NetConf Juniper SRX Cloud user Nitro API {API client (Fog/etc)} VNC JSON ec2 API JSON Netscaler Cloud user Console Console Console {ec2 API client } ProxyConsole VM Proxy VM Proxy VM Proxy VM NFS MySQL MySQL Server Sec. Storage Sec. Storage NFS NFS {Proxied} SSH Sec. VM Storage Sec. Storage VM Ajax Ajax HTTPS VM VM Console Console Router VM Router VM HTTP (Template Download) Router VM Router VM HTTP (Template Copy) Router VM Router VM Cloud user HTTP (Swift) http://www.slideshare.net/cloudstack/cloudstack-architecture
Terminology Zone: Availability zone, aka Regions. Could be worldwide. Different data centers Pods: Racks or aisles in a data center Clusters: Group of machines with a common type of Hypervisor Host: A Single server Primary Storage: Shared storage across a cluster Secondary Storage: Shared storage in a single Zone
Storage • Primary Storage: – Anything that can be mounted on the node of a cluster. – Cluster LVM…iSCSI… – Holds disk images of running VMs – Support for CEPH with KVM hypervisors • Secondary Storage: – Available across the zone – Holds snapshots and templates (image repo) – Can use Openstack swift or any object store (Gluster FS…) – New support for Caringo • Can use NFS for both to start • Storage Abstraction refactoring underway
Networking • Extremely flexible to: – Provide isolation with VLANs – Provide isolation at L3 with shared L2 (scalability) – Support hardware devices that exposes API – Deployed on existing networking infrastructure – Support new networking paradigm (SDN) • Support for Nicira Virtual P • Extensive use of Open VSwitch
A customizable GUI AJAX + API
A very extensive API
API • Not really REST • A set of methods available over http(s) • Unauthenticated on integration port • Authenticated on 8080 using Access and Secret Key • Python/Ruby clients available • Internal Marvin client • Cloudmonkey CLI • Other clouds client support the API
Enabling EC2 and S3 • Via the GUI • Via API call on integration API port 8096 http://localhost:8096/client/api? command=updateConfiguration&name=enable.s3.api&value=true http://localhost:8096/client/api? command=updateConfiguration&name=enable.ec2.api&value=true
DevCloud • Several use cases: – Try CloudStack in an isolated sandbox. Runs within the appliance – Develop CloudStack on own machine, build locally and deploy new version in DevCloud (Build and test) – Develop and Run locally, use DevCloud as Xen hosts
DevCloud: self-contained CloudStack runs in the appliance
Testing “4.0” code in DevCloud • Deploy new CloudStack code in self-contained DevCloud: • mvn –P deps • ant rdeploy • ant rdeploydb – Wipes database of mgt server, you will need to reconfigure the “data center” • ant rdebug
DevCloud: as Host Run CloudStack on local machine Use DevCloud to setup hosts
DevCloud to test 4.1 branch • mvn -P developer clean • mvn -P developer install • mvn -P developer -pl developer –Ddeploydb • mvn -P developer -pl tools/devcloud – Ddeploydb • mvn -pl :cloud-client-ui jetty:run Configure infrastructure: • mvn -P developer,deploysvr -pl tools/devcloud -Ddeploysvr
Testing Framework –for the PyUG  [environment] • Marvin is a Python dns=10.147.28.6 mshost=10.147.39.69 based framework to run mysql.host=10.147.39.69 tests against a [cloudstack] private.gateway=10.147.40.1 CloudStack install private.pod.startip=10.147.41.121 private.pod.endip=10.147.41.160 • Could be used as a private.netmask=255.255.254.0 public.gateway=10.147.40.1 simulator of a public.vlan.startip=10.147.41.162 datacenter public.vlan.endip=10.147.41.200 public.netmask=255.255.254.0 • Used to configure an hypervisor=XenServer host=10.147.40.10 infrastructure on a mgt host.password=password #storage pools server primary.pool=nfs://10.147.28.7:/expo rt/home/automation/sadhu/primary secondary.pool=nfs://10.147.28.6:/ex port/home/automation/sadhu/secondary
Demo time • http://vimeo.com/52150218
Info • Apache incubator project • http://www.cloudstack.org • http://incubator.apache.org/cloudstack/ • #cloudstack on irc.freenode.net • @cloudstack on Twitter • http://www.slideshare.net/cloudstack • http://cloudstack.org/discuss/mailing-lists.html Welcoming contributions and feedback, Join the fun !

More Related Content

Building FOSS clouds

  • 1. Building FOSS Clouds Sebastien Goasguen, Kris Buytaert LinuxCon EU, Nov 6th Barcelona, Spain Slides at: slideshare.net/sebastiengoasguen
  • 2. Speakers… • Joe stayed home to deal with the release, • Kris Buytaert stepped in:
  • 3. Tutorial Outline • 10:10 – 10:45 FOSS Clouds and CloudStack intro • 10:45 – 11:00 DevCloud demo • 11:00 – 11:50 FOSS Cloud Automation, DevOps configuration and monitoring tools
  • 4. Define:tutorial • A tutorial is a method of transferring knowledge and may be used as a part of a learning process. More interactive and specific than a book or a lecture…<snip> • Therefore:
  • 5. DevCloud • A CloudStack sandbox – Pass the USB keys around and get the Vbox appliance devcloud.ova file • Import the appliance in VirtualBox – Play with CloudStack – http://wiki.cloudstack.org/display/COMM/DevCloud – https://cwiki.apache.org/confluence/display/CLOUDSTA CK/CloudStack+devcloud+environment+setup
  • 6. Objectives • Learn the basic architecture of a cloud • Learn a few new tools • Explore CloudStack via its sandbox • Discover some concepts of automation • Get home with some ideas to try out • Mileage may vary
  • 7. NIST Definition of Cloud • "cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.“ – NIST
  • 8. NIST Service/Deployment Models • Service Models: • IaaS (e.g EC2, rackspace) • PaaS (e.g GAE, Azure) • SaaS (e.g Gapps) • Deployment model: • Private cloud • Public cloud • Hybrid cloud (Private than can use Public)
  • 9. NIST Key Characteristics of Cloud • Elasticity • Expands and Shrinks as needed • E.g Elastic Load Balancing, Elastic Map Reduce • Measured Service • Utility Pricing. You pay for what you use • E.g $0.08 for one hour of small instance • On-demand • You get it when you ask for it • E.g One api call and you get your machine, ~no queue
  • 10. “Guidelines” • Virtualization and automation are key enablers of the cloud characteristics • Build your infrastructure to enable on- demand elastic provisioning of –servers, storage, network – • Setup a PaaS that suits you and your applications • Then move to continuous delivery of applications …
  • 14. IaaS challenges • Not out of the box by itself. • Need a farm of hypervisors – Xen, KVM, Vmware… • Need storage – For image catalogue – For volume/snapshot management • Need flexible network that can be configured on-demand – VLANS, no VLANS, existing net infra, SDN…
  • 15. Let’s make Red Hat happy
  • 17. Coming at 12:00 EST • Apache CloudStack 4.0 • Check out the testing procedure: – https://cwiki.apache.org/CLOUDSTACK/cloudstack -40-test-procedure.html • Or watch my amazing screencast: – http://vimeo.com/52150218
  • 18. A bit of History • Original company VMOPs (2008) – Founded by Sheng Liang former lead dev on JVM • Open source (GPLv3) as CloudStack • Acquired by Citrix (July 2011) • Relicensed under ASL v2 April 3, 2012 • Accepted as Apache Incubating Project April 16, 2012 • First Apache (ACS 4.0) release expected now
  • 20. Apache Process • 100% community driven • New ideas, decisions only taken on mailing lists. Votes taken by community • Project led by Project Management Committee (PMC): – http://www.apache.org/dev/pmc.html – http://incubator.apache.org/guides/committer.html • Non committers get invited as committers: – http://community.apache.org/newcommitter.html
  • 22. Contributions Examples: Sungard: Announced that 6 developers were joining the Apache project Schuberg Philis: Big contribution in building/packaging and Nicira support Go Daddy: Early proto of Maven building Caringo: Support for own object store Basho: Devcloud packaging
  • 23. A Very Flexible IaaS Platform Compute Hypervisor Storage Block & Object Primary Storage Secondary Storage Network Network & Network Services http://www.slideshare.net/cloudstack/cloudstack-architecture
  • 24. Architecture / Language • Java application • Tomcat6, Axis2, Maven build + ant – Ant going away in 4.1 • Collaboration Conference, Nov 30th -Dec 2nd – http://collab12.cloudstack.org/
  • 25. Build and Run in 4.1 • git clone https://git-wip- us.apache.org/repos/asf/incubat or-cloudstack.git • mvn clean • mvn install • mvn –P developer –pl developer –Ddeploydb • mvn –pl :cloud-client-ui jetty:run
  • 26. Cloud Interactions OVM Cluster Primary Storage Ap i Xen vcenter vcenter Monitoring Primary CS API Clu I ste AP vSphere Cluster Storage r M n ter End End gm vCe t User UI User UI Primary XS Cluster Storage Admin Admin Clustered Clustered CloudStack XAPI UI UI CloudStack Domain Domain CS Admin & CloudStack CloudStack CloudStack CloudStack End-user API Primary Admin Admin UI UI Management Management JSON KVM Cluster Storage Server Server NetConf Juniper SRX Cloud user Nitro API {API client (Fog/etc)} VNC JSON ec2 API JSON Netscaler Cloud user Console Console Console {ec2 API client } ProxyConsole VM Proxy VM Proxy VM Proxy VM NFS MySQL MySQL Server Sec. Storage Sec. Storage NFS NFS {Proxied} SSH Sec. VM Storage Sec. Storage VM Ajax Ajax HTTPS VM VM Console Console Router VM Router VM HTTP (Template Download) Router VM Router VM HTTP (Template Copy) Router VM Router VM Cloud user HTTP (Swift) http://www.slideshare.net/cloudstack/cloudstack-architecture
  • 27. Terminology Zone: Availability zone, aka Regions. Could be worldwide. Different data centers Pods: Racks or aisles in a data center Clusters: Group of machines with a common type of Hypervisor Host: A Single server Primary Storage: Shared storage across a cluster Secondary Storage: Shared storage in a single Zone
  • 28. Storage • Primary Storage: – Anything that can be mounted on the node of a cluster. – Cluster LVM…iSCSI… – Holds disk images of running VMs – Support for CEPH with KVM hypervisors • Secondary Storage: – Available across the zone – Holds snapshots and templates (image repo) – Can use Openstack swift or any object store (Gluster FS…) – New support for Caringo • Can use NFS for both to start • Storage Abstraction refactoring underway
  • 29. Networking • Extremely flexible to: – Provide isolation with VLANs – Provide isolation at L3 with shared L2 (scalability) – Support hardware devices that exposes API – Deployed on existing networking infrastructure – Support new networking paradigm (SDN) • Support for Nicira Virtual P • Extensive use of Open VSwitch
  • 30. A customizable GUI AJAX + API
  • 32. API • Not really REST • A set of methods available over http(s) • Unauthenticated on integration port • Authenticated on 8080 using Access and Secret Key • Python/Ruby clients available • Internal Marvin client • Cloudmonkey CLI • Other clouds client support the API
  • 33. Enabling EC2 and S3 • Via the GUI • Via API call on integration API port 8096 http://localhost:8096/client/api? command=updateConfiguration&name=enable.s3.api&value=true http://localhost:8096/client/api? command=updateConfiguration&name=enable.ec2.api&value=true
  • 34. DevCloud • Several use cases: – Try CloudStack in an isolated sandbox. Runs within the appliance – Develop CloudStack on own machine, build locally and deploy new version in DevCloud (Build and test) – Develop and Run locally, use DevCloud as Xen hosts
  • 36. Testing “4.0” code in DevCloud • Deploy new CloudStack code in self-contained DevCloud: • mvn –P deps • ant rdeploy • ant rdeploydb – Wipes database of mgt server, you will need to reconfigure the “data center” • ant rdebug
  • 37. DevCloud: as Host Run CloudStack on local machine Use DevCloud to setup hosts
  • 38. DevCloud to test 4.1 branch • mvn -P developer clean • mvn -P developer install • mvn -P developer -pl developer –Ddeploydb • mvn -P developer -pl tools/devcloud – Ddeploydb • mvn -pl :cloud-client-ui jetty:run Configure infrastructure: • mvn -P developer,deploysvr -pl tools/devcloud -Ddeploysvr
  • 39. Testing Framework –for the PyUG  [environment] • Marvin is a Python dns=10.147.28.6 mshost=10.147.39.69 based framework to run mysql.host=10.147.39.69 tests against a [cloudstack] private.gateway=10.147.40.1 CloudStack install private.pod.startip=10.147.41.121 private.pod.endip=10.147.41.160 • Could be used as a private.netmask=255.255.254.0 public.gateway=10.147.40.1 simulator of a public.vlan.startip=10.147.41.162 datacenter public.vlan.endip=10.147.41.200 public.netmask=255.255.254.0 • Used to configure an hypervisor=XenServer host=10.147.40.10 infrastructure on a mgt host.password=password #storage pools server primary.pool=nfs://10.147.28.7:/expo rt/home/automation/sadhu/primary secondary.pool=nfs://10.147.28.6:/ex port/home/automation/sadhu/secondary
  • 41. Info • Apache incubator project • http://www.cloudstack.org • http://incubator.apache.org/cloudstack/ • #cloudstack on irc.freenode.net • @cloudstack on Twitter • http://www.slideshare.net/cloudstack • http://cloudstack.org/discuss/mailing-lists.html Welcoming contributions and feedback, Join the fun !

Editor's Notes

  1. https://cwiki.apache.org/CLOUDSTACK/building-with-maven.html
  2. http://download.cloud.com/releases/3.0.3/api_3.0.3/TOC_Root_Admin.html
  3. https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+devcloud+environment+setup
  4. https://cwiki.apache.org/confluence/display/CLOUDSTACK/Testing+with+Python