SlideShare a Scribd company logo

Cloud Performance Benchmarking

•Download as PPTX, PDF•
•
Santanu Dey
Santanu Dey

The document discusses benchmarking the performance of compute and database services on the cloud. It outlines procedures to test IOPS, network performance, and CPU usage of a compute instance and database throughput, connections, and queries per second of a DB service. Tests were run varying threads, block sizes, and parallel connections. The compute instance showed optimal IOPS at specific block sizes and higher write than read performance. Database performance increased with more connections but was limited by the server. Issues noted were inability to directly measure IOPS and lack of short-term monitoring data. Finally, a WordPress application was deployed on a compute instance connected to a database service to test performance.

1 of 32
Cloud Performance Benchmarking
The Problem Statement Test Performance of a Compute Instance on the cloud The testing focuses on disk IO, network IO, and some index of CPU. This report outlines the testing procedure ,result data and analysis on the findings Test Performance of a DB Service on the cloud The testing focuses on database IOPS, QPS, Connection number and others, This report outlines the testing procedure ,result data and analysis on findings Application Deployment Try out WordPress based web application is deployed on cloud. The deployment involve 1 instance of Compute and 1 Database service. This presentation outlines the procedure of deployment and any learnings associated with it etc.
Testing Steps
Set up • For the purpose of this tests Alicloud is used • For compute : An ECS instance with 2 virtual cores and 4GB is used • For Database an RDS service used. Both being in the same region.
Steps 1 2 3 4 5 6 Launch an ECS and an RDS instance in the same region Whitelist RDS for accessing from the ECS Install httpd, PHP modules and Wordpress on the ECS instance. Configure Database access in wordpress config file Run CPU, Memory and IO test on ECS. Monitor with top and then with AliCloud default Monitoring Create Database instance for Wordpress app, create user for database access: Using Alicloud Console Install Sysbench tool in ECS instance. Run IOPS and Connections test for the RDS. Test Report
IOPS Test on Compute Node
IOPS on Compute Node: Test Method FIO utility command is used # throw-away: 5 min warm-up fio --runtime=300 --time_based --clocksource=clock_gettime -- name=randread --numjobs=8 --rw=randread --random_distribution=pareto:0.9 --bs=8k -- size=10g --filename=fio.tmp # Sample 1min runs with varying job threads and block size fio --runtime=60 --time_based --clocksource=clock_gettime -- name=randread --numjobs=1 --rw=randread --random_distribution=pareto:0.9 --bs=8k -- size=10g --filename=fio.tmp
IOPS on Compute Node : Monitor from Cloud Console
IOPS on Compute Node: Results and Analysis • Random Writes are optimized at 4k blocks • Random Reads work best at 8k blocks • Writes perform better than reads in this test
Issues • IOPs for the writes were observed higher than that of in cloud console monitoring of Alicloud. This is possibly to due to write buffering ignored by fio • Increasing number of threads does not increase IOPs. This is possibly due to what is supported in the virtualized disk controller/driver in the hypervisor.
CPU Test on Compute Node
CPU Performance Test on Compute Node : Testing Steps Tools Used sysbench and top sysbench --test=cpu --cpu-max-prime=40000 --num-threads=1 run Top is used to observe peak CPU utilization in parallel to the sysbench run
CPU Performance Test on Compute Node : Monitor from Cloud Console
CPU Performance on Compute Node: Results & Analysis # CPU Events Number of Threads Peak CPU Utilization % During Test Total Time Taken in s 40000 1 50.1 99.86 40000 2 99.5 50.19 40000 3 99.8 50.12 40000 4 99.5 50.13 • No performance anomalies observed. Behaviour expected for CPU with 2 cores • Comparing the time taken for the same test with other known CPUs the performance can be benchmarked
Issues • There is no direct relation between CPU events / sec and real CPU clock speed. Only relative estimate indicates benchmarking against other processors • This test is not suitable for observing occasional CPU bursts / CPU hogging due to shared nature of virtual instances. That needs real time monitoring tool for a longer period.
Network Performance Test
Testing Method On Server [root@iZu1iy3602qZ ~]# iperf -s -l 128k ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 85.3 KByte (default) ------------------------------------------------------------ [ 4] local xx.xx.xx.xx port 5001 connected with xx.xx.xx.xx port 42490 [ ID] Interval Transfer Bandwidth [ 4] 0.0-30.0 sec 175 MBytes 48.8 Mbits/sec On Client # iperf -c xx.xx.xx.xx -l 128k -i 4 -t 30 ------------------------------------------------------------ Client connecting to xx.xx.xx.xx, TCP port 5001 TCP window size: 45.0 KByte (default) ------------------------------------------------------------ [ 3] local 172.31.54.251 port 42489 connected with xx.xx.xx.xx 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 4.0 sec 23.2 MBytes 48.8 Mbits/sec [ 3] 0.0-30.2 sec 176 MBytes 49.0 Mbits/sec
Results & Analysis With 1 Connections -> 0.0-30.2 sec 176 MBytes 48.9 Mbits/sec With 4 Connections -> 0.0-30.2 sec 176 MBytes 49.0 Mbits/sec • The bandwidth limit observed at 49 Mbps could be a throttling on the client or the server. The server was the ECS instance on Alicloud. The client was external to Alicloud on another cloud provider. • Within Alicloud the speed was much better - 18 Gbps
RDS Performance
Using sysbench OLTP test mode #Preparation sysbench --test=oltp --oltp-table- size=200000 --mysql-host=<host> -- mysql-db=perfdatabase --mysql- user=perfuser --mysql- password=password1 --db-driver=mysql prepare # Read Test sysbench --test=oltp --oltp-table- size=200000 --mysql-host=<host> -- mysql-db=perfdatabase --mysql- user=perfuser --mysql- password=password1 --db-driver=mysql - -max-time=60 --oltp-read-only=on -- max-requests=0 --num-threads=8 run
Run 1 : 1 Connection # Read only Test read: 55594 write: 0 other: 7942 total: 63536 transactions: 3971 (66.17 per sec.) read/write requests: 55594 (926.38 per sec.) other operations: 7942 (132.34 per sec.) # Read & Write Test read: 43120 write: 15400 other: 6160 total: 64680 transactions: 3080 (51.33 per sec.) read/write requests: 394060 (975.24 per sec.) other operations: 41480 (102.66 per sec.)
Run 2 : 4 Parallel Connections # Read only Test read: 232890 write: 0 other: 33270 total: 266160 transactions: 16635 (277.18 per sec.) read/write requests: 232890 (3880.50 per sec.) other operations: 33270 (554.36 per sec.) # Read & Write Test read: 154042 write: 55015 other: 22006 total: 231063 transactions: 11003 (183.34 per sec.) read/write requests: 209057 (3483.40 per sec.) other operations: 22006 (366.67 per sec.)
Run 3 : 8 Parallel Connections # Read only Test read: 471324 write: 0 other: 67332 total: 538656 transactions: 33666 (560.99 per sec.) read/write requests: 471324 (7853.89 per sec.) other operations: 67332 (1121.98 per sec.) # Read & Write Test read: 290360 write: 103700 other: 41480 total: 435540 transactions: 20740 (345.59 per sec.) read/write requests: 394060 (6566.21 per sec.) other operations: 41480 (691.18 per sec.)
Run 4 : 16 Parallel Connections # Read only Test read: 707126 write: 0 other: 101018 total: 808144 transactions: 50509 (841.57 per sec.) read/write requests: 707126 (11782.05 per sec.) other operations: 101018 (1683.15 per sec.) # Read & Write Test read: 482804 write: 172430 other: 68972 total: 724206 transactions: 34486 (574.49 per sec.) read/write requests: 655234 (10915.38 per sec.) other operations: 68972 (1148.99 per sec.)
Run 5 : 32 Parallel Connections # Read only Test read: 707126 write: 0 other: 101018 total: 808144 transactions: 50509 (841.57 per sec.) read/write requests: 707126 (11782.05 per sec.) other operations: 101018 (1683.15 per sec.) # Read & Write Test read: 708680 write: 253100 other: 101240 total: 1063020 transactions: 50620 (843.33 per sec.) read/write requests: 961780 (16023.35 per sec.) other operations: 101240 (1686.67 per sec.)
Maximum Performance Eventually the following maximums were reached Maximum Number of Queries per second (QPS): 22 k IoPS: Could not measure Concurrent Connections: Tested upto 600 concurrent connections. After which server threw error FATAL: error 1040: Too many connections
RDS Monitoring Metrics
Issues • Mysqlslap could not be run due to lack of database admin privilege : Error: mysqlslap: Cannot drop database 'mysqlslap' ERROR : OPERATION need to be executed set by ADMIN. • IOPs could not be reliably measured without local access to mySQL instance. • Only query performance could be measured • The number of parallel connections can be indirectly measured by running netstat | grep <hostname>| wc -l when the test is going on • Cloud console monitoring collects data every 300 seconds. For short tests running for about 1 min each, those test data are not reflected in the console monitoring graphs.
Default ECS Disk Usage Metrics IOPS Bytes / Sec
Sample Application Deployment on ECS and RDS
Wordpress Deployment • Create WordPress DB in RDS & whitelist the ECS Server • Create Wordpress DB and DB user in RDS • Install HTTPD and PHP module on the ECS instance • Configure Wordpress to use the RDS BD information • Following this use apache ab tool performance test the wordpress site for concurrent requests, max TPS and latency measures. ECS Instance httpd PHP Module Wordpress (php) RDS Service Wordpress DB Connection Whitelisting
Thank You

More Related Content

Cloud Performance Benchmarking

  • 2. The Problem Statement Test Performance of a Compute Instance on the cloud The testing focuses on disk IO, network IO, and some index of CPU. This report outlines the testing procedure ,result data and analysis on the findings Test Performance of a DB Service on the cloud The testing focuses on database IOPS, QPS, Connection number and others, This report outlines the testing procedure ,result data and analysis on findings Application Deployment Try out WordPress based web application is deployed on cloud. The deployment involve 1 instance of Compute and 1 Database service. This presentation outlines the procedure of deployment and any learnings associated with it etc.
  • 4. Set up • For the purpose of this tests Alicloud is used • For compute : An ECS instance with 2 virtual cores and 4GB is used • For Database an RDS service used. Both being in the same region.
  • 5. Steps 1 2 3 4 5 6 Launch an ECS and an RDS instance in the same region Whitelist RDS for accessing from the ECS Install httpd, PHP modules and Wordpress on the ECS instance. Configure Database access in wordpress config file Run CPU, Memory and IO test on ECS. Monitor with top and then with AliCloud default Monitoring Create Database instance for Wordpress app, create user for database access: Using Alicloud Console Install Sysbench tool in ECS instance. Run IOPS and Connections test for the RDS. Test Report
  • 7. IOPS on Compute Node: Test Method FIO utility command is used # throw-away: 5 min warm-up fio --runtime=300 --time_based --clocksource=clock_gettime -- name=randread --numjobs=8 --rw=randread --random_distribution=pareto:0.9 --bs=8k -- size=10g --filename=fio.tmp # Sample 1min runs with varying job threads and block size fio --runtime=60 --time_based --clocksource=clock_gettime -- name=randread --numjobs=1 --rw=randread --random_distribution=pareto:0.9 --bs=8k -- size=10g --filename=fio.tmp
  • 8. IOPS on Compute Node : Monitor from Cloud Console
  • 9. IOPS on Compute Node: Results and Analysis • Random Writes are optimized at 4k blocks • Random Reads work best at 8k blocks • Writes perform better than reads in this test
  • 10. Issues • IOPs for the writes were observed higher than that of in cloud console monitoring of Alicloud. This is possibly to due to write buffering ignored by fio • Increasing number of threads does not increase IOPs. This is possibly due to what is supported in the virtualized disk controller/driver in the hypervisor.
  • 12. CPU Performance Test on Compute Node : Testing Steps Tools Used sysbench and top sysbench --test=cpu --cpu-max-prime=40000 --num-threads=1 run Top is used to observe peak CPU utilization in parallel to the sysbench run
  • 13. CPU Performance Test on Compute Node : Monitor from Cloud Console
  • 14. CPU Performance on Compute Node: Results & Analysis # CPU Events Number of Threads Peak CPU Utilization % During Test Total Time Taken in s 40000 1 50.1 99.86 40000 2 99.5 50.19 40000 3 99.8 50.12 40000 4 99.5 50.13 • No performance anomalies observed. Behaviour expected for CPU with 2 cores • Comparing the time taken for the same test with other known CPUs the performance can be benchmarked
  • 15. Issues • There is no direct relation between CPU events / sec and real CPU clock speed. Only relative estimate indicates benchmarking against other processors • This test is not suitable for observing occasional CPU bursts / CPU hogging due to shared nature of virtual instances. That needs real time monitoring tool for a longer period.
  • 17. Testing Method On Server [root@iZu1iy3602qZ ~]# iperf -s -l 128k ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 85.3 KByte (default) ------------------------------------------------------------ [ 4] local xx.xx.xx.xx port 5001 connected with xx.xx.xx.xx port 42490 [ ID] Interval Transfer Bandwidth [ 4] 0.0-30.0 sec 175 MBytes 48.8 Mbits/sec On Client # iperf -c xx.xx.xx.xx -l 128k -i 4 -t 30 ------------------------------------------------------------ Client connecting to xx.xx.xx.xx, TCP port 5001 TCP window size: 45.0 KByte (default) ------------------------------------------------------------ [ 3] local 172.31.54.251 port 42489 connected with xx.xx.xx.xx 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 4.0 sec 23.2 MBytes 48.8 Mbits/sec [ 3] 0.0-30.2 sec 176 MBytes 49.0 Mbits/sec
  • 18. Results & Analysis With 1 Connections -> 0.0-30.2 sec 176 MBytes 48.9 Mbits/sec With 4 Connections -> 0.0-30.2 sec 176 MBytes 49.0 Mbits/sec • The bandwidth limit observed at 49 Mbps could be a throttling on the client or the server. The server was the ECS instance on Alicloud. The client was external to Alicloud on another cloud provider. • Within Alicloud the speed was much better - 18 Gbps
  • 20. Using sysbench OLTP test mode #Preparation sysbench --test=oltp --oltp-table- size=200000 --mysql-host=<host> -- mysql-db=perfdatabase --mysql- user=perfuser --mysql- password=password1 --db-driver=mysql prepare # Read Test sysbench --test=oltp --oltp-table- size=200000 --mysql-host=<host> -- mysql-db=perfdatabase --mysql- user=perfuser --mysql- password=password1 --db-driver=mysql - -max-time=60 --oltp-read-only=on -- max-requests=0 --num-threads=8 run
  • 21. Run 1 : 1 Connection # Read only Test read: 55594 write: 0 other: 7942 total: 63536 transactions: 3971 (66.17 per sec.) read/write requests: 55594 (926.38 per sec.) other operations: 7942 (132.34 per sec.) # Read & Write Test read: 43120 write: 15400 other: 6160 total: 64680 transactions: 3080 (51.33 per sec.) read/write requests: 394060 (975.24 per sec.) other operations: 41480 (102.66 per sec.)
  • 22. Run 2 : 4 Parallel Connections # Read only Test read: 232890 write: 0 other: 33270 total: 266160 transactions: 16635 (277.18 per sec.) read/write requests: 232890 (3880.50 per sec.) other operations: 33270 (554.36 per sec.) # Read & Write Test read: 154042 write: 55015 other: 22006 total: 231063 transactions: 11003 (183.34 per sec.) read/write requests: 209057 (3483.40 per sec.) other operations: 22006 (366.67 per sec.)
  • 23. Run 3 : 8 Parallel Connections # Read only Test read: 471324 write: 0 other: 67332 total: 538656 transactions: 33666 (560.99 per sec.) read/write requests: 471324 (7853.89 per sec.) other operations: 67332 (1121.98 per sec.) # Read & Write Test read: 290360 write: 103700 other: 41480 total: 435540 transactions: 20740 (345.59 per sec.) read/write requests: 394060 (6566.21 per sec.) other operations: 41480 (691.18 per sec.)
  • 24. Run 4 : 16 Parallel Connections # Read only Test read: 707126 write: 0 other: 101018 total: 808144 transactions: 50509 (841.57 per sec.) read/write requests: 707126 (11782.05 per sec.) other operations: 101018 (1683.15 per sec.) # Read & Write Test read: 482804 write: 172430 other: 68972 total: 724206 transactions: 34486 (574.49 per sec.) read/write requests: 655234 (10915.38 per sec.) other operations: 68972 (1148.99 per sec.)
  • 25. Run 5 : 32 Parallel Connections # Read only Test read: 707126 write: 0 other: 101018 total: 808144 transactions: 50509 (841.57 per sec.) read/write requests: 707126 (11782.05 per sec.) other operations: 101018 (1683.15 per sec.) # Read & Write Test read: 708680 write: 253100 other: 101240 total: 1063020 transactions: 50620 (843.33 per sec.) read/write requests: 961780 (16023.35 per sec.) other operations: 101240 (1686.67 per sec.)
  • 26. Maximum Performance Eventually the following maximums were reached Maximum Number of Queries per second (QPS): 22 k IoPS: Could not measure Concurrent Connections: Tested upto 600 concurrent connections. After which server threw error FATAL: error 1040: Too many connections
  • 28. Issues • Mysqlslap could not be run due to lack of database admin privilege : Error: mysqlslap: Cannot drop database 'mysqlslap' ERROR : OPERATION need to be executed set by ADMIN. • IOPs could not be reliably measured without local access to mySQL instance. • Only query performance could be measured • The number of parallel connections can be indirectly measured by running netstat | grep <hostname>| wc -l when the test is going on • Cloud console monitoring collects data every 300 seconds. For short tests running for about 1 min each, those test data are not reflected in the console monitoring graphs.
  • 29. Default ECS Disk Usage Metrics IOPS Bytes / Sec
  • 31. Wordpress Deployment • Create WordPress DB in RDS & whitelist the ECS Server • Create Wordpress DB and DB user in RDS • Install HTTPD and PHP module on the ECS instance • Configure Wordpress to use the RDS BD information • Following this use apache ab tool performance test the wordpress site for concurrent requests, max TPS and latency measures. ECS Instance httpd PHP Module Wordpress (php) RDS Service Wordpress DB Connection Whitelisting