Repair & Recovery for your MySQL, MariaDB & MongoDB / TokuMX Clusters - Webinar Slides
•
11 likes•4,765 views
This document discusses redundancy models for MySQL, MariaDB, MongoDB and TokuMX databases. It covers asynchronous replication used in MySQL replication and MongoDB/TokuMX compared to synchronous replication in Galera and NDB Cluster. The document then zooms into recovery procedures for Galera clusters and discusses how to prevent split-brain situations in multi-datacenter setups through the use of additional nodes and assigning node weights.
Report
Share
More Related Content
What's hot
What's hot (20)
Similar to Repair & Recovery for your MySQL, MariaDB & MongoDB / TokuMX Clusters - Webinar Slides
Similar to Repair & Recovery for your MySQL, MariaDB & MongoDB / TokuMX Clusters - Webinar Slides (20)
More from Severalnines
More from Severalnines (20)
Recently uploaded
Recently uploaded (20)
Repair & Recovery for your MySQL, MariaDB & MongoDB / TokuMX Clusters - Webinar Slides
- 1. Repair and Recovery for your MySQL, MariaDB and MongoDB/ TokuMX clusters January 2013 Johan Andersson Severalnines AB johan@severalnines.com Mobile +46 73 073 60 99
- 2. Copyright 2014 Severalnines AB Agenda ! Redundancy models for Galera, NDB and MongoDB/ TokuMX ! Failover & Recovery (Automatic vs Manual) ! Zooming into Galera recovery procedures ! Split brains in multi-datacenter setups 2
- 3. Copyright 2014 Severalnines AB Redundancy models for Galera, NDB and MongoDB/TokuMX ! Database Clustering can be divided in two areas ! Asynchronously replicated data ! MySQL Replication ! TokuMX/MongoDB ! Synchronously replicated data ! Galera for MySQL ! NDB Cluster (aka MySQL Cluster) 3
- 4. Copyright 2014 Severalnines AB Asynchronous 1 2 3
- 5. Copyright 2014 Severalnines AB Asynchronous Pros Cons ! Read scaling is easy ! Master handles all writes ! Read write splitting ! Lower write latency ! Automated failover is hard: ! Elect a new master ! Slave lag & Data loss ! Has everything been sent to the slave when the master crashes? ! Has the slave applied all data? 5
- 6. Copyright 2014 Severalnines AB Synchronous 1 4 2 3
- 7. Copyright 2014 Severalnines AB Synchronous Pros Cons ! All servers are masters ! Higher write latency ! Application can read and write on any server ! Sensitive to link latency ! No data loss ! Network partitioning ! Simple failover for apps 7
- 8. Copyright 2014 Severalnines AB Redundancy Models TokuMX/MongoDB ReplicaSet (1/2) ! Up to 12 members ! Max 7 voting numbers ! Primary, Secondary, Arbiter ! Slave Delay ! Chaining Allowed ! Heartbeats every 2 seconds ! 5 missed heartbeats: node dead Primary Secondary Arbiter 8
- 9. Copyright 2014 Severalnines AB Redundancy Models TokuMX/MongoDB ReplicaSet (2/2) ! Automatic failover ! Priority (between 0 and 100.0) ! Primary has highest priority ! Influences election of new primary Data Center 1 Primary Secondary Data Center 2 Secondary 9
- 10. Copyright 2014 Severalnines AB Redundancy Models TokuMX/MongoDB Sharded Cluster ! Built-in support for sharding ! Min 2 Query Routers ! 3 Config Servers Primary Secondary Arbiter Primary Secondary Arbiter 10
- 11. Copyright 2014 Severalnines AB Redundancy Models MySQL Replication ! No hard limits on no. of slaves ! Replication chaining ! Delayed slave ! Manual failover Slave Master Slave 11
- 12. Copyright 2014 Severalnines AB Redundancy Models MySQL Cluster ! In-built sharding ! Separate SQL and Data layers ! Up to 4 replicas/node group ! Max 48 data nodes ! Auto-failover Node Group 1 Node Group 2 12
- 13. Copyright 2014 Severalnines AB Redundancy Models Galera Cluster ! Synchronous replication plugin ! All masters have same dataset ! Slowest node determines speed of cluster ! No hard limits on no. of nodes ! Auto-failover ! Galera arbitrator (garbd) Master Master Master 13
- 14. Copyright 2014 Severalnines AB Synchronous Replication Linear Two-phase commit ! Used by NDB data nodes ! Pessimistic locking ! Works well for cases with update contention 1 4 6 3 2 5 1. Prepare transaction 4. commit 14
- 15. Copyright 2014 Severalnines AB Synchronous Replication Certificate-based ! Used by Galera/WSREP ! Optimistic locking ! Not optimal for cases with conflicting updates 1 2 3 4 1. begin transaction 2. commit 15
- 16. Copyright 204Severalnines AB Zooming into Galera recovery procedures By Seppo Jaakola @ Codership 16
- 17. Copyright 2014 Severalnines AB Split-brain ! Network partitioning (aka) split brain. N nodes divided into M equal parts Galera Servers - SYNCED DC1 WAN Galera Servers - SYNCED DC2 17
- 18. Copyright 2014 Severalnines AB Split-brain ! WAN breaks - 2 equal splits (Network partitioning) ! Non-Primary – cannot handle transactions Galera – Non-Primary DC1 WAN Galera – Non-Primary DC2 18
- 19. Copyright 2014 Severalnines AB Split-brain prevention ! Solution: The side with the majority of nodes will win: ! Add garbd or a galera node to DC1 or DC2 ! Add weight to existing node in DC1: wsrep_provider_options=“pc.weight=N;” (default = 1 ) Galera – Synced DC1 WAN Galera – Synced DC2 19
- 20. Copyright 2014 Severalnines AB Split-brain prevention ! Add weight to existing node in DC1: wsrep_provider_options=“pc.weight=2;” Galera – Synced DC1 pc.weight=2 WAN 1 3 Galera – Synced DC2 1 1 2 20
- 21. Copyright 2014 Severalnines AB Split-brain prevention ! Final Two Data center setup with an additional node. Galera – Synced DC1 WAN Galera – Synced DC2 21
- 22. Copyright 2014 Severalnines AB Split-brain prevention ! Three data centers, split brain unlikely. ! Galera 3.0 supports segments (gmcast.segment) ! Only one node in each segment will relay to only one other node in another segment ! Segments also applies to Two Data centers, and between racks in one DC. WAN Galera – Segment 3 DC2 Galera – Segment 1 DC1 Galera – Segment 2 DC3
- 23. Copyright 2014 Severalnines AB Q&A 23
- 24. Copyright 2014 Severalnines AB Thank you. For further questions please contact John-Jerome Schmidt: jj@severalnines.com 24