SlideShare a Scribd company logo

ProxySQL & PXC(Query routing and Failover Test)

•Download as PPTX, PDF•
•
YoungHeon (Roy) Kim
YoungHeon (Roy) Kim

This document describes setting up and testing ProxySQL for query routing and high availability with Percona XtraDB Cluster (PXC). It includes instructions for installing and configuring ProxySQL, adding backend PXC servers, creating query rules for routing, and testing read/write splitting and failover through sysbench tests. Failover is demonstrated by stopping one PXC node, and ProxySQL is shown routing queries to the remaining nodes and marking the failed node as offline in its status.

Related slideshows

MySQL Timeout Variables Explained
MySQL Timeout Variables Explained MySQL Timeout Variables Explained
MySQL Timeout Variables Explained
 
MaxScale이해와활용-2023.11
MaxScale이해와활용-2023.11MaxScale이해와활용-2023.11
MaxScale이해와활용-2023.11
 
The Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication TutorialThe Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication Tutorial
 
1 of 32
ProxySQL & PXC Query routing and failover test
22 dba.kim@gmail.com https://kr.linkedin.com/in/youngheon-roy-kim-23a11181 About me
Architecture 3 DB Servers ProxySQL host1 host2 host3 host4 PXC Proxysql01 Proxysql02 Application App Query ETL Query Replication VIP : 10.xx.xx.xx 10.xx.xx.001 10.xx.xx.002 10.xx.xx.003 10.xx.xx.004
Install & Configuration
ProxySQL Install & Configuration 5 Download https://www.percona.com/downloads/proxysql/LATEST/binary/redhat/6/x86_64/ Install #rpm –ivh proxysql-1.3.6-1.1.x86_64.rpm Proxysql start #service proxysql start You have to install MySQL on proxysql server. At least, you need to install following packages : Percona-Server-shared-56-5.6.34-rel79.1.el6.x86_64 Percona-Server-client-56-5.6.34-rel79.1.el6.x86_64
6 ProxySQL Install & Configuration Proxysql admin connect #mysql –uadmin –h127.0.0.1 –P6032 -p Default password of admin user is 'admin'. Insert server infomation on ProxySQL server mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.001',0,3306,10000000 00 ); mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.002',0,3306,1000); mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.002',1,3306,5000); mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.001',1,3306,1000); mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.003',1,3306,5000); mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.004',2,3306,1000); mysql>LOAD MYSQL SERVERS TO RUNTIME; mysql>SAVE MYSQL SERVERS TO DISK;
7 ProxySQL Install & Configuration Insert application user infomation on ProxySQL server mysql>INSERT INTO mysql_users (username,password) VALUES ('web1','1111'); mysql>INSERT INTO mysql_users (username,password) VALUES ('etl','1111'); mysql>LOAD MYSQL USERS TO RUNTIME; mysql>SAVE MYSQL USERS TO DISK; 'web1' & 'etl' users should be on pxc databases.
ProxySQL Install & Configuration 8 Insert Query rules on ProxySQL server mysql>INSERT INTO mysql_query_rules(active,match_pattern,destination_hostgroup,apply) VALUES(1,'^SELECT.*FOR UPDATE$',0,1); mysql>INSERT INTO mysql_query_rules(active,username,match_pattern,destination_hostgroup,apply) VALUES(1,'etl','^SELECT',2,1); mysql>INSERT INTO mysql_query_rules(active,match_pattern,destination_hostgroup,apply) VALUES(1,'^SELECT',1,1); mysql>LOAD MYSQL QUERY RULES TO RUNTIME; mysql>SAVE MYSQL QUERY RULES TO DISK;
Create monitoring user on pxc servers 9 on PXC servers mysql>GRANT REPLICATION CLIENT ON *.* TO monitor@10.xx.% IDENTIFIED BY 'monitor'; mysql>FLUSH PRIVILEGES; 10.xx.% : proxysql server's ip band
10 ProxySQL Install & Configuration Register schedule for galera check on ProxySQL server mysql>INSERT INTO scheduler(id,interval_ms,filename,arg1,arg2,arg3,arg4,arg5) VALUES (1,'10000','/usr/bin/proxysql_galera_checker',0,1,2,1,'/var/lib/proxysql/proxysql _galera_checker.log'); mysql>LOAD SCHEDULER TO RUNTIME; mysql>SAVE SCEHDULER TO DISK; mysql>SELECT * FROM runtime_schedulerG #tail –f /var/lib/proxysql/proxysql_galera_checker.log
Connection test on pxc servers 11 on ProxySQL server #mysql –uweb1 –p1111 –h127.0.0.1 –P6033 –e"SELECT @@hostname"
Query routing & Failover Test
Confirm ProxySQL's settings 13 on ProxySQL server mysql> select hostgroup_id,hostname,status,weight from mysql_servers; +--------------+---------------+--------+------------+ | hostgroup_id | hostname | status | weight | +--------------+---------------+--------+------------+ | 0 | 10.xx.xx.001 | ONLINE | 1000000000 | | 0 | 10.xx.xx.002 | ONLINE | 1000 | | 1 | 10.xx.xx.002 | ONLINE | 5000 | | 1 | 10.xx.xx.001 | ONLINE | 1000 | | 1 | 10.xx.xx.003 | ONLINE | 5000 | | 2 | 10.xx.xx.004 | ONLINE | 1000 | +--------------+---------------+--------+------------+ 6 rows in set (0.00 sec) Server and Hostgroup hostgroup 0 : read + write hostgroup 1 : read only hostgroup 2 : ETL query(read only)
Confirm ProxySQL's settings on ProxySQL server mysql> select rule_id,active,username,match_pattern,destination_hostgroup,apply from mys ql_query_rules; +---------+--------+----------+----------------------+-----------------------+-------+ | rule_id | active | username | match_pattern | destination_hostgroup | apply | +---------+--------+----------+----------------------+-----------------------+-------+ | 1 | 1 | NULL | ^SELECT.*FOR UPDATE$ | 0 | 1 | | 2 | 1 | etl | NULL | 2 | 1 | | 3 | 1 | NULL | ^SELECT | 1 | 1 | +---------+--------+----------+----------------------+-----------------------+-------+ 3 rows in set (0.00 sec) Query Rule
Sysbench 1.0.7 Install 15 Download http://repo.percona.com/release/6/RPMS/x86_64/sysbench-1.0.7-1.el6.x86_64.rpm Script for test #ls –al /usr/share/sysbench -rwxr-xr-x 1 root root 1452 May 16 15:27 bulk_insert.lua -rw-r--r-- 1 root root 13762 May 16 15:27 oltp_common.lua -rwxr-xr-x 1 root root 1116 May 16 15:27 oltp_delete.lua -rwxr-xr-x 1 root root 2018 May 16 15:27 oltp_insert.lua -rwxr-xr-x 1 root root 1265 May 16 15:27 oltp_point_select.lua -rwxr-xr-x 1 root root 1649 May 16 15:27 oltp_read_only.lua -rwxr-xr-x 1 root root 1824 May 16 15:27 oltp_read_write.lua -rwxr-xr-x 1 root root 1118 May 16 15:27 oltp_update_index.lua -rwxr-xr-x 1 root root 1127 May 16 15:27 oltp_update_non_index.lua -rwxr-xr-x 1 root root 1440 May 16 15:27 oltp_write_only.lua -rwxr-xr-x 1 root root 1919 May 16 15:27 select_random_points.lua -rwxr-xr-x 1 root root 2118 May 16 15:27 select_random_ranges.lua
16 # cat rw.sh sysbench --time=600 --mysql-db=test --mysql-user=web1 --mysql-password=1111 --threads=8 --mysql-host=vip --mysql-port=6033 --db-ps-mode=disable --db-driver=mysql --table-size=1000000 --tables=3 --mysql-ignore-errors=2013,1213,2006,9001 /usr/share/sysbench/oltp_read_write.lua run Write + Read Test script Write + Read Test
Write + Read Test 17 Write : host1 processed whole write transactions. Read : host2 and host3 processed most of read transactions, host1 processed 10% of read transactions. 'replicated_bytes' means this server processed write transaction. 'received_bytes' means this server received data that changed from other servers.
18 # cat r.sh sysbench --time=600 --mysql-db=test --mysql-user=web1 --mysql-password=1111 --threads=8 --mysql-host=vip --mysql-port=6033 --db-ps-mode=disable --db-driver=mysql --table-size=1000000 --tables=3 --mysql-ignore-errors=2013,1213,2006,9001 /usr/share/sysbench/oltp_read_only.lua run Read Only Test script Read Only Test
Read Only Test 19 Read : host2 and host3 processed most of read transactions(5000 QPS each), host1 processed 10% of re ad transactions(800 QPS).
20 # cat r-etl.sh sysbench --time=300 --mysql-db=test --mysql-user=etl --mysql-password=1111 --threads=8 --mysql-host=vip --mysql-port=6033 --db-ps-mode=disable --db-driver=mysql --table-size=1000000 --tables=3 --mysql-ignore-errors=2013,1213,2006,9001 /usr/share/sysbench/oltp_read_only.lua run ETL Query Test script ETL Query Test
21 ETL Query Test Read : host4 processed all queries that etl user executed.
Failover Test 22 Host1 stop and restart during executing rw test script. Write : host1 processed write queries, and host2 processed write queries after host1 was down. host1 processed again write queries after host1 was recovered.
23 on ProxySQL server mysql> SELECT * FROM stats_mysql_connection_pool; +-----------+--------------+----------+---------+----------+----------+--------+---------+---------+--------------- | hostgroup | srv_host | srv_port | status | ConnUsed | ConnFree | ConnOK | ConnERR | Queries | Bytes_data_sent | Bytes_data_recv | Latency_us | +-----------+--------------+----------+---------+----------+----------+--------+---------+---------+--------------- | 0 | 10.xx.xx.001 | 3306 | SHUNNED | 0 | 0 | 99960 | 278 | 6644693 | 963024436 | 0 | 201 | | 0 | 10.xx.xx.002 | 3306 | ONLINE | 9 | 0 | 22400 | 0 | 1058947 | 84360297 | 0 | 176 | | 1 | 10.xx.xx.002 | 3306 | ONLINE | 3 | 3 | 24 | 0 | 6963231 | 313221230 | 16991073582 | 176 | | 1 | 10.xx.xx.001 | 3306 | SHUNNED | 0 | 0 | 47 | 174 | 905312 | 40866635 | 2223688931 | 201 | | 1 | 10.xx.xx.003 | 3306 | ONLINE | 2 | 4 | 22 | 0 | 6577992 | 296003542 | 16081521858 | 196 | | 2 | 10.xx.xx.004 | 3306 | ONLINE | 0 | 0 | 23 | 0 | 248624 | 1367421 | 0 | 158 | +-----------+--------------+----------+---------+----------+----------+--------+---------+---------+---------------6 rows in set (0.00 sec) You can check server status when host1 is down. Failover Test
24 on ProxySQL server mysql> select hostgroup_id,hostname,status,weight from mysql_servers; +--------------+--------------+--------------+---------+ | hostgroup_id | hostname | status | weight | +--------------+--------------+--------------+---------+ | 0 | 10.xx.xx.001 | OFFLINE_SOFT | 1000000 | | 0 | 10.xx.xx.002 | ONLINE | 1000 | | 1 | 10.xx.xx.002 | ONLINE | 5000 | | 1 | 10.xx.xx.001 | OFFLINE_SOFT | 1000 | | 1 | 10.xx.xx.003 | ONLINE | 5000 | | 2 | 10.xx.xx.004 | ONLINE | 1000 | +--------------+--------------+--------------+---------+ 6 rows in set (0.00 sec) You can also check server status when host1 is down. Failover Test
25 on ProxySQL server mysql> select hostgroup_id,hostname,status,weight from mysql_servers; +--------------+--------------+--------------+---------+ | hostgroup_id | hostname | status | weight | +--------------+--------------+--------------+---------+ | 0 | 10.xx.xx.001 | OFFLINE_HARD | 1000 | | 0 | 10.xx.xx.002 | ONLINE | 1000 | | 1 | 10.xx.xx.002 | ONLINE | 5000 | | 1 | 10.xx.xx.001 | OFFLINE_HARD | 1000 | | 1 | 10.xx.xx.003 | ONLINE | 5000 | | 2 | 10.xx.xx.004 | ONLINE | 1000 | +--------------+--------------+--------------+---------+ 6 rows in set (0.00 sec) You can see status of host1 change 'offline_hard' and value of weight is changed from 1000k to 1000. Failover Test
26 on ProxySQL server mysql> select hostgroup_id,hostname,status,weight from mysql_servers; +--------------+--------------+--------------+---------+ | hostgroup_id | hostname | status | weight | +--------------+--------------+--------------+---------+ | 0 | 10.xx.xx.001 | ONLINE | 1000 | | 0 | 10.xx.xx.002 | ONLINE | 1000 | | 1 | 10.xx.xx.002 | ONLINE | 5000 | | 1 | 10.xx.xx.001 | ONLINE | 1000 | | 1 | 10.xx.xx.003 | ONLINE | 5000 | | 2 | 10.xx.xx.004 | ONLINE | 1000 | +--------------+--------------+--------------+---------+ 6 rows in set (0.00 sec) Status of host1 was changed 'ONLINE' when it recovered, but value of weight was not changed. Actually host1 processed all write queries when it recovered, so it worked as value of weight was 1000 k although value of weight looks like 1000. Failover Test
ProxySQL Failover Test using keepalived 27 Install keepalived on proxysql servers #yum install –y keepalived.x86_64
28 Config keepalived # vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { lvs_id proxy_HA notification_email { admin@mail.com } } vrrp_script check_proxy{ script "killall -0 proxysql" interval 2 weight 2 } vrrp_instance VI_01 { state MASTER interface bond0 virtual_router_id 51 priority 100 virtual_ipaddress { 10.xx.xx.vip dev bond0 } track_script{ check_proxy } } ProxySQL Failover Test using keepalived priority 100 : on proxysql01 priority 101 : on proxysql02
29 ProxySQL Failover Test using keepalived add vip on proxysql servers # pwd /etc/sysconfig/network-scripts # vi ifcfg-bond0:1 DEVICE=bond0:1 BOOTPROTO=static BROADCAST=10.xx.xx.255 IPADDR=10.xx.xx.vip NETMASK=255.255.255.0 NETWORK=10.xx.xx.0 ONBOOT=yes BONDING_OPTS="mode=1 miimon=100" #/etc/init.d/network restart #ip addr ## you can check a vip using this command
30 The network of proxysql01 server went down during executing rw test script. Vip moved to proxysql02 server when network of proxysql01 server went down. ProxySQL Failover Test using keepalived
Conclusion 31 Query routing function is working well as I intend. Order of query rules is important, you should check order of query rules. Failover function is working well though one server is down. But weight of value seems to show default value although it works correctly when host recovers. You can modify weight of value correctly using below command: on ProxySQL server mysql>load mysql servers to memory;
Thank You

More Related Content

ProxySQL & PXC(Query routing and Failover Test)

  • 1. ProxySQL & PXC Query routing and failover test
  • 3. Architecture 3 DB Servers ProxySQL host1 host2 host3 host4 PXC Proxysql01 Proxysql02 Application App Query ETL Query Replication VIP : 10.xx.xx.xx 10.xx.xx.001 10.xx.xx.002 10.xx.xx.003 10.xx.xx.004
  • 5. ProxySQL Install & Configuration 5 Download https://www.percona.com/downloads/proxysql/LATEST/binary/redhat/6/x86_64/ Install #rpm –ivh proxysql-1.3.6-1.1.x86_64.rpm Proxysql start #service proxysql start You have to install MySQL on proxysql server. At least, you need to install following packages : Percona-Server-shared-56-5.6.34-rel79.1.el6.x86_64 Percona-Server-client-56-5.6.34-rel79.1.el6.x86_64
  • 6. 6 ProxySQL Install & Configuration Proxysql admin connect #mysql –uadmin –h127.0.0.1 –P6032 -p Default password of admin user is 'admin'. Insert server infomation on ProxySQL server mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.001',0,3306,10000000 00 ); mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.002',0,3306,1000); mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.002',1,3306,5000); mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.001',1,3306,1000); mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.003',1,3306,5000); mysql>INSERT INTO mysql_servers(hostname,hostgroup_id,port,weight) VALUES('10.xx.xx.004',2,3306,1000); mysql>LOAD MYSQL SERVERS TO RUNTIME; mysql>SAVE MYSQL SERVERS TO DISK;
  • 7. 7 ProxySQL Install & Configuration Insert application user infomation on ProxySQL server mysql>INSERT INTO mysql_users (username,password) VALUES ('web1','1111'); mysql>INSERT INTO mysql_users (username,password) VALUES ('etl','1111'); mysql>LOAD MYSQL USERS TO RUNTIME; mysql>SAVE MYSQL USERS TO DISK; 'web1' & 'etl' users should be on pxc databases.
  • 8. ProxySQL Install & Configuration 8 Insert Query rules on ProxySQL server mysql>INSERT INTO mysql_query_rules(active,match_pattern,destination_hostgroup,apply) VALUES(1,'^SELECT.*FOR UPDATE$',0,1); mysql>INSERT INTO mysql_query_rules(active,username,match_pattern,destination_hostgroup,apply) VALUES(1,'etl','^SELECT',2,1); mysql>INSERT INTO mysql_query_rules(active,match_pattern,destination_hostgroup,apply) VALUES(1,'^SELECT',1,1); mysql>LOAD MYSQL QUERY RULES TO RUNTIME; mysql>SAVE MYSQL QUERY RULES TO DISK;
  • 9. Create monitoring user on pxc servers 9 on PXC servers mysql>GRANT REPLICATION CLIENT ON *.* TO monitor@10.xx.% IDENTIFIED BY 'monitor'; mysql>FLUSH PRIVILEGES; 10.xx.% : proxysql server's ip band
  • 10. 10 ProxySQL Install & Configuration Register schedule for galera check on ProxySQL server mysql>INSERT INTO scheduler(id,interval_ms,filename,arg1,arg2,arg3,arg4,arg5) VALUES (1,'10000','/usr/bin/proxysql_galera_checker',0,1,2,1,'/var/lib/proxysql/proxysql _galera_checker.log'); mysql>LOAD SCHEDULER TO RUNTIME; mysql>SAVE SCEHDULER TO DISK; mysql>SELECT * FROM runtime_schedulerG #tail –f /var/lib/proxysql/proxysql_galera_checker.log
  • 11. Connection test on pxc servers 11 on ProxySQL server #mysql –uweb1 –p1111 –h127.0.0.1 –P6033 –e"SELECT @@hostname"
  • 12. Query routing & Failover Test
  • 13. Confirm ProxySQL's settings 13 on ProxySQL server mysql> select hostgroup_id,hostname,status,weight from mysql_servers; +--------------+---------------+--------+------------+ | hostgroup_id | hostname | status | weight | +--------------+---------------+--------+------------+ | 0 | 10.xx.xx.001 | ONLINE | 1000000000 | | 0 | 10.xx.xx.002 | ONLINE | 1000 | | 1 | 10.xx.xx.002 | ONLINE | 5000 | | 1 | 10.xx.xx.001 | ONLINE | 1000 | | 1 | 10.xx.xx.003 | ONLINE | 5000 | | 2 | 10.xx.xx.004 | ONLINE | 1000 | +--------------+---------------+--------+------------+ 6 rows in set (0.00 sec) Server and Hostgroup hostgroup 0 : read + write hostgroup 1 : read only hostgroup 2 : ETL query(read only)
  • 14. Confirm ProxySQL's settings on ProxySQL server mysql> select rule_id,active,username,match_pattern,destination_hostgroup,apply from mys ql_query_rules; +---------+--------+----------+----------------------+-----------------------+-------+ | rule_id | active | username | match_pattern | destination_hostgroup | apply | +---------+--------+----------+----------------------+-----------------------+-------+ | 1 | 1 | NULL | ^SELECT.*FOR UPDATE$ | 0 | 1 | | 2 | 1 | etl | NULL | 2 | 1 | | 3 | 1 | NULL | ^SELECT | 1 | 1 | +---------+--------+----------+----------------------+-----------------------+-------+ 3 rows in set (0.00 sec) Query Rule
  • 15. Sysbench 1.0.7 Install 15 Download http://repo.percona.com/release/6/RPMS/x86_64/sysbench-1.0.7-1.el6.x86_64.rpm Script for test #ls –al /usr/share/sysbench -rwxr-xr-x 1 root root 1452 May 16 15:27 bulk_insert.lua -rw-r--r-- 1 root root 13762 May 16 15:27 oltp_common.lua -rwxr-xr-x 1 root root 1116 May 16 15:27 oltp_delete.lua -rwxr-xr-x 1 root root 2018 May 16 15:27 oltp_insert.lua -rwxr-xr-x 1 root root 1265 May 16 15:27 oltp_point_select.lua -rwxr-xr-x 1 root root 1649 May 16 15:27 oltp_read_only.lua -rwxr-xr-x 1 root root 1824 May 16 15:27 oltp_read_write.lua -rwxr-xr-x 1 root root 1118 May 16 15:27 oltp_update_index.lua -rwxr-xr-x 1 root root 1127 May 16 15:27 oltp_update_non_index.lua -rwxr-xr-x 1 root root 1440 May 16 15:27 oltp_write_only.lua -rwxr-xr-x 1 root root 1919 May 16 15:27 select_random_points.lua -rwxr-xr-x 1 root root 2118 May 16 15:27 select_random_ranges.lua
  • 16. 16 # cat rw.sh sysbench --time=600 --mysql-db=test --mysql-user=web1 --mysql-password=1111 --threads=8 --mysql-host=vip --mysql-port=6033 --db-ps-mode=disable --db-driver=mysql --table-size=1000000 --tables=3 --mysql-ignore-errors=2013,1213,2006,9001 /usr/share/sysbench/oltp_read_write.lua run Write + Read Test script Write + Read Test
  • 17. Write + Read Test 17 Write : host1 processed whole write transactions. Read : host2 and host3 processed most of read transactions, host1 processed 10% of read transactions. 'replicated_bytes' means this server processed write transaction. 'received_bytes' means this server received data that changed from other servers.
  • 18. 18 # cat r.sh sysbench --time=600 --mysql-db=test --mysql-user=web1 --mysql-password=1111 --threads=8 --mysql-host=vip --mysql-port=6033 --db-ps-mode=disable --db-driver=mysql --table-size=1000000 --tables=3 --mysql-ignore-errors=2013,1213,2006,9001 /usr/share/sysbench/oltp_read_only.lua run Read Only Test script Read Only Test
  • 19. Read Only Test 19 Read : host2 and host3 processed most of read transactions(5000 QPS each), host1 processed 10% of re ad transactions(800 QPS).
  • 20. 20 # cat r-etl.sh sysbench --time=300 --mysql-db=test --mysql-user=etl --mysql-password=1111 --threads=8 --mysql-host=vip --mysql-port=6033 --db-ps-mode=disable --db-driver=mysql --table-size=1000000 --tables=3 --mysql-ignore-errors=2013,1213,2006,9001 /usr/share/sysbench/oltp_read_only.lua run ETL Query Test script ETL Query Test
  • 21. 21 ETL Query Test Read : host4 processed all queries that etl user executed.
  • 22. Failover Test 22 Host1 stop and restart during executing rw test script. Write : host1 processed write queries, and host2 processed write queries after host1 was down. host1 processed again write queries after host1 was recovered.
  • 23. 23 on ProxySQL server mysql> SELECT * FROM stats_mysql_connection_pool; +-----------+--------------+----------+---------+----------+----------+--------+---------+---------+--------------- | hostgroup | srv_host | srv_port | status | ConnUsed | ConnFree | ConnOK | ConnERR | Queries | Bytes_data_sent | Bytes_data_recv | Latency_us | +-----------+--------------+----------+---------+----------+----------+--------+---------+---------+--------------- | 0 | 10.xx.xx.001 | 3306 | SHUNNED | 0 | 0 | 99960 | 278 | 6644693 | 963024436 | 0 | 201 | | 0 | 10.xx.xx.002 | 3306 | ONLINE | 9 | 0 | 22400 | 0 | 1058947 | 84360297 | 0 | 176 | | 1 | 10.xx.xx.002 | 3306 | ONLINE | 3 | 3 | 24 | 0 | 6963231 | 313221230 | 16991073582 | 176 | | 1 | 10.xx.xx.001 | 3306 | SHUNNED | 0 | 0 | 47 | 174 | 905312 | 40866635 | 2223688931 | 201 | | 1 | 10.xx.xx.003 | 3306 | ONLINE | 2 | 4 | 22 | 0 | 6577992 | 296003542 | 16081521858 | 196 | | 2 | 10.xx.xx.004 | 3306 | ONLINE | 0 | 0 | 23 | 0 | 248624 | 1367421 | 0 | 158 | +-----------+--------------+----------+---------+----------+----------+--------+---------+---------+---------------6 rows in set (0.00 sec) You can check server status when host1 is down. Failover Test
  • 24. 24 on ProxySQL server mysql> select hostgroup_id,hostname,status,weight from mysql_servers; +--------------+--------------+--------------+---------+ | hostgroup_id | hostname | status | weight | +--------------+--------------+--------------+---------+ | 0 | 10.xx.xx.001 | OFFLINE_SOFT | 1000000 | | 0 | 10.xx.xx.002 | ONLINE | 1000 | | 1 | 10.xx.xx.002 | ONLINE | 5000 | | 1 | 10.xx.xx.001 | OFFLINE_SOFT | 1000 | | 1 | 10.xx.xx.003 | ONLINE | 5000 | | 2 | 10.xx.xx.004 | ONLINE | 1000 | +--------------+--------------+--------------+---------+ 6 rows in set (0.00 sec) You can also check server status when host1 is down. Failover Test
  • 25. 25 on ProxySQL server mysql> select hostgroup_id,hostname,status,weight from mysql_servers; +--------------+--------------+--------------+---------+ | hostgroup_id | hostname | status | weight | +--------------+--------------+--------------+---------+ | 0 | 10.xx.xx.001 | OFFLINE_HARD | 1000 | | 0 | 10.xx.xx.002 | ONLINE | 1000 | | 1 | 10.xx.xx.002 | ONLINE | 5000 | | 1 | 10.xx.xx.001 | OFFLINE_HARD | 1000 | | 1 | 10.xx.xx.003 | ONLINE | 5000 | | 2 | 10.xx.xx.004 | ONLINE | 1000 | +--------------+--------------+--------------+---------+ 6 rows in set (0.00 sec) You can see status of host1 change 'offline_hard' and value of weight is changed from 1000k to 1000. Failover Test
  • 26. 26 on ProxySQL server mysql> select hostgroup_id,hostname,status,weight from mysql_servers; +--------------+--------------+--------------+---------+ | hostgroup_id | hostname | status | weight | +--------------+--------------+--------------+---------+ | 0 | 10.xx.xx.001 | ONLINE | 1000 | | 0 | 10.xx.xx.002 | ONLINE | 1000 | | 1 | 10.xx.xx.002 | ONLINE | 5000 | | 1 | 10.xx.xx.001 | ONLINE | 1000 | | 1 | 10.xx.xx.003 | ONLINE | 5000 | | 2 | 10.xx.xx.004 | ONLINE | 1000 | +--------------+--------------+--------------+---------+ 6 rows in set (0.00 sec) Status of host1 was changed 'ONLINE' when it recovered, but value of weight was not changed. Actually host1 processed all write queries when it recovered, so it worked as value of weight was 1000 k although value of weight looks like 1000. Failover Test
  • 27. ProxySQL Failover Test using keepalived 27 Install keepalived on proxysql servers #yum install –y keepalived.x86_64
  • 28. 28 Config keepalived # vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { lvs_id proxy_HA notification_email { admin@mail.com } } vrrp_script check_proxy{ script "killall -0 proxysql" interval 2 weight 2 } vrrp_instance VI_01 { state MASTER interface bond0 virtual_router_id 51 priority 100 virtual_ipaddress { 10.xx.xx.vip dev bond0 } track_script{ check_proxy } } ProxySQL Failover Test using keepalived priority 100 : on proxysql01 priority 101 : on proxysql02
  • 29. 29 ProxySQL Failover Test using keepalived add vip on proxysql servers # pwd /etc/sysconfig/network-scripts # vi ifcfg-bond0:1 DEVICE=bond0:1 BOOTPROTO=static BROADCAST=10.xx.xx.255 IPADDR=10.xx.xx.vip NETMASK=255.255.255.0 NETWORK=10.xx.xx.0 ONBOOT=yes BONDING_OPTS="mode=1 miimon=100" #/etc/init.d/network restart #ip addr ## you can check a vip using this command
  • 30. 30 The network of proxysql01 server went down during executing rw test script. Vip moved to proxysql02 server when network of proxysql01 server went down. ProxySQL Failover Test using keepalived
  • 31. Conclusion 31 Query routing function is working well as I intend. Order of query rules is important, you should check order of query rules. Failover function is working well though one server is down. But weight of value seems to show default value although it works correctly when host recovers. You can modify weight of value correctly using below command: on ProxySQL server mysql>load mysql servers to memory;