SlideShare a Scribd company logo

Containers for the Enterprise: Delivering OpenShift on OpenStack for Performance and Scale

•
•
Stephen Gordon
Stephen Gordon

Imagine being able to stand up thousands of tenants with thousands of apps, running thousands of Docker-formatted container images and routes, all on a self-healing cluster. Now, take that one step further with all of those images being updatable through a single upload to the registry, and with zero downtime. In this session, Steve Gordon of the Red Hat OpenStack Platform team will show you just that. Steve will walk through a recent benchmarking deployment using the Cloud Native Computing Foundation’s (CNCF) new 1,000 node cluster with OpenStack and Red Hat’s OpenShift Container Platform, the enterprise-ready Kubernetes for developers.

Related slideshows

KubeWHAT!?
KubeWHAT!?KubeWHAT!?
KubeWHAT!?
 
Gluster ovirt integration_gluster_meetup_pune_2015
Gluster ovirt integration_gluster_meetup_pune_2015Gluster ovirt integration_gluster_meetup_pune_2015
Gluster ovirt integration_gluster_meetup_pune_2015
 
Virtualization inside kubernetes
Virtualization inside kubernetesVirtualization inside kubernetes
Virtualization inside kubernetes
 
1 of 22
Download to read offline
DELIVERING OPENSHIFT ON OPENSTACK FOR PERFORMANCE AND SCALE STEVE GORDON (@xsgordon)
GOAL ● 1000 OpenShift Container Platform nodes; hosted on ● 300 physical Red Hat OpenStack Platform nodes 2
3 APPLICATION CONTAINERS 1. Code: mysqld 2. Configuration: /etc/my.cnf 3. Data: /var/lib/mysql 4. Other stuff :-) CODE, CONFIGURATION, DATA
4 EXPOSITION OF RESOURCES
5 CONSUMPTION OF RESOURCES
OPENSHIFT ON OPENSTACK 6 Architectural tenets: ● Technical independence ● Contextual awareness ● Avoiding redundancy ● Simplified management Reference architecture: ● red.ht/2ibNmvX
PERFORMANCE AND SCALE TESTING 7
WHERE TO TEST? 8 The CNCF cluster is made up of 1000 nodes deployed at Switch, Las Vegas by Intel for the use of the CNCF community: Compute Node Spec ● 2x Intel E5-2680v3 12-core ● 256GB RAM ● 2x Intel S3610 400GB SSD ● 1x Intel P3700 800GB NVMe PCIe SSD ● 1x QP Intel X710 Storage Node Spec ● 2x Intel E5-2680v3 12-core ● 128GB RAM ● 2x Intel S3610 400GB SSD ● 10x Intel 2TB NLSAS HDD ● 1x QP Intel X710" Got a nefarious plan for taking over the world using CNCF related open source projects? Head to https://github.com/cncf/cluster.
WHAT TO TEST? Goal: 1000 OpenShift Container Platform nodes, on 300 Red Hat OpenStack Platform nodes ● Push deployment to its limit, identify: ○ Bottlenecks, ○ Config changes, and ○ Best practices ● Document, file, and fix issues as appropriate. 9
ARCHITECTURE 10 ● OpenStack UnderCloud
ARCHITECTURE 11 ● OpenStack UnderCloud ● OpenStack Overcloud control plane
ARCHITECTURE 12 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute
ARCHITECTURE 13 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute ● OpenShift/Kubernetes master node VMs ● OpenShift routing, registry, and metrics ● OpenShift/Kubernetes minions ● Load Generators
ARCHITECTURE 14 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute ● OpenShift/Kubernetes master node VMs ● OpenShift routing, registry, and metrics ● OpenShift/Kubernetes minions
ARCHITECTURE 15 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute ● OpenShift/Kubernetes master node VMs ● OpenShift routing, registry, and metrics ● OpenShift/Kubernetes minions ● Load generators
SYSTEM VERIFICATION TEST SUITE ● Red Hat OpenShift Performance and Scalability team’s upstream test suites ● Main tests are ○ cluster-loader ○ Networking/synthetic ○ Workload Generator ○ Reliability/Longevity ● https://github.com/openshift/svt 16
17 CLUSTER-LOADER ARCHITECTURE Start Parse args & config Config Obj End False Create Namespace True X Exists ? Items < N False Create X Iterate Item Count True True False X can be: ● Quota ● Template ● Service ● User ● Pod ● RC
CLUSTER LOADER TEST 18 OpenShift (Kubernetes) Object Quantity Nodes 1,000 Namespaces (projects) 13,000 Pods 52,000 Build Configs 39,000 Templates 78,000 Image Streams 13,000 Deployment Configs and Services 39,000 (incl. 13,000 Replication Controllers) Secrets 260,000 Routes 39,000 Component # of bugs filed/encountered Kubernetes 8 Installer 8 Images 5 Docker 5 Containers 3 Storage 2 Ansible 2 Management 1 iptables 1 Deployments 1 Command 1 Builds 1 Auth 1
NOTABLE SCALE ISSUES ENCOUNTERED 19 etcd Disk Utilization API Server CPU
NOTABLE SCALE ISSUES ENCOUNTERED 20 Ansible Installer https://github.com/ansible/ansible/issues/16749
FUTURE OPPORTUNITIES ● Reference architecture available at red.ht/2ibNmvX ● Containers and OpenStack: A platform for distributed applications, paper available at http://red.ht/2hSfIPs ● Learn more about Red Hat Summit at redhat.com/summit 21
THANK YOU 22

More Related Content

Containers for the Enterprise: Delivering OpenShift on OpenStack for Performance and Scale

  • 1. DELIVERING OPENSHIFT ON OPENSTACK FOR PERFORMANCE AND SCALE STEVE GORDON (@xsgordon)
  • 2. GOAL ● 1000 OpenShift Container Platform nodes; hosted on ● 300 physical Red Hat OpenStack Platform nodes 2
  • 3. 3 APPLICATION CONTAINERS 1. Code: mysqld 2. Configuration: /etc/my.cnf 3. Data: /var/lib/mysql 4. Other stuff :-) CODE, CONFIGURATION, DATA
  • 6. OPENSHIFT ON OPENSTACK 6 Architectural tenets: ● Technical independence ● Contextual awareness ● Avoiding redundancy ● Simplified management Reference architecture: ● red.ht/2ibNmvX
  • 8. WHERE TO TEST? 8 The CNCF cluster is made up of 1000 nodes deployed at Switch, Las Vegas by Intel for the use of the CNCF community: Compute Node Spec ● 2x Intel E5-2680v3 12-core ● 256GB RAM ● 2x Intel S3610 400GB SSD ● 1x Intel P3700 800GB NVMe PCIe SSD ● 1x QP Intel X710 Storage Node Spec ● 2x Intel E5-2680v3 12-core ● 128GB RAM ● 2x Intel S3610 400GB SSD ● 10x Intel 2TB NLSAS HDD ● 1x QP Intel X710" Got a nefarious plan for taking over the world using CNCF related open source projects? Head to https://github.com/cncf/cluster.
  • 9. WHAT TO TEST? Goal: 1000 OpenShift Container Platform nodes, on 300 Red Hat OpenStack Platform nodes ● Push deployment to its limit, identify: ○ Bottlenecks, ○ Config changes, and ○ Best practices ● Document, file, and fix issues as appropriate. 9
  • 11. ARCHITECTURE 11 ● OpenStack UnderCloud ● OpenStack Overcloud control plane
  • 12. ARCHITECTURE 12 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute
  • 13. ARCHITECTURE 13 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute ● OpenShift/Kubernetes master node VMs ● OpenShift routing, registry, and metrics ● OpenShift/Kubernetes minions ● Load Generators
  • 14. ARCHITECTURE 14 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute ● OpenShift/Kubernetes master node VMs ● OpenShift routing, registry, and metrics ● OpenShift/Kubernetes minions
  • 15. ARCHITECTURE 15 ● OpenStack UnderCloud ● OpenStack Overcloud control plane ● OpenStack Overcloud compute ● OpenShift/Kubernetes master node VMs ● OpenShift routing, registry, and metrics ● OpenShift/Kubernetes minions ● Load generators
  • 16. SYSTEM VERIFICATION TEST SUITE ● Red Hat OpenShift Performance and Scalability team’s upstream test suites ● Main tests are ○ cluster-loader ○ Networking/synthetic ○ Workload Generator ○ Reliability/Longevity ● https://github.com/openshift/svt 16
  • 17. 17 CLUSTER-LOADER ARCHITECTURE Start Parse args & config Config Obj End False Create Namespace True X Exists ? Items < N False Create X Iterate Item Count True True False X can be: ● Quota ● Template ● Service ● User ● Pod ● RC
  • 18. CLUSTER LOADER TEST 18 OpenShift (Kubernetes) Object Quantity Nodes 1,000 Namespaces (projects) 13,000 Pods 52,000 Build Configs 39,000 Templates 78,000 Image Streams 13,000 Deployment Configs and Services 39,000 (incl. 13,000 Replication Controllers) Secrets 260,000 Routes 39,000 Component # of bugs filed/encountered Kubernetes 8 Installer 8 Images 5 Docker 5 Containers 3 Storage 2 Ansible 2 Management 1 iptables 1 Deployments 1 Command 1 Builds 1 Auth 1
  • 19. NOTABLE SCALE ISSUES ENCOUNTERED 19 etcd Disk Utilization API Server CPU
  • 20. NOTABLE SCALE ISSUES ENCOUNTERED 20 Ansible Installer https://github.com/ansible/ansible/issues/16749
  • 21. FUTURE OPPORTUNITIES ● Reference architecture available at red.ht/2ibNmvX ● Containers and OpenStack: A platform for distributed applications, paper available at http://red.ht/2hSfIPs ● Learn more about Red Hat Summit at redhat.com/summit 21