SlideShare a Scribd company logo

Running Galera Cluster on Microsoft Azure

•
•
Codership Oy - Creators of Galera Cluster
Codership Oy - Creators of Galera Cluster

Running Galera Cluster in Microsoft Azure involves setting up virtual machines and installing Galera Cluster software. This provides more control than Azure Database for MySQL, which uses asynchronous replication. While Azure Database for MySQL is fully managed, Galera Cluster in VMs supports the virtually synchronous replication that is its core feature. Cost estimates show running three Galera Cluster nodes in VMs costs less monthly than three hosted MySQL instances in Azure Database for MySQL.

Related slideshows

MySQL 5.7 InnoDB Cluster (Jan 2018)
MySQL 5.7 InnoDB Cluster (Jan 2018)MySQL 5.7 InnoDB Cluster (Jan 2018)
MySQL 5.7 InnoDB Cluster (Jan 2018)
 
Introduction to Databases - query optimizations for MySQL
Introduction to Databases - query optimizations for MySQLIntroduction to Databases - query optimizations for MySQL
Introduction to Databases - query optimizations for MySQL
 
What is new in PostgreSQL 14?
What is new in PostgreSQL 14?What is new in PostgreSQL 14?
What is new in PostgreSQL 14?
 
1 of 33
Download to read offline
Running Galera Cluster Microsoft Azure and comparing it to their hosted services Colin Charles, colin.charles@galeracluster.com 10 September 2019 https://twitter.com/galeracluster | www.galeracluster.com Codership Webinar
Agenda • Running Galera Cluster in the the Microsoft Azure cloud • Galera Cluster vs Azure Database for MySQL / MariaDB Server
WARNING: NO DEMO! • We will have a recorded video up on the website as a resource • https://galeracluster.com/videos/
Codership • The developers and experts of Galera Cluster • Established in 2007, 3 founders, all engineers • Seppo Jaakola, CEO, Teemu Ollakka, CTO, Alex Yurchenko, architect • Services business model, producing 100% open source software • Thousands of users in various industries: e-commerce, betting/gambling, telecoms, banking, insurance, gaming, healthcare, media, marketing, advertising, travel, education, software as a service, PaaS, IaaS, etc. !4
!5
!6
Galera Cluster is all about multi-master replication • Can be described as virtually synchronous replication • High Availability with no data loss, and consistent data across all nodes — no Single Point of Failure (SPoF) • Quorum based failure handling • Optimistic concurrency control • 100% multi-master cluster (all nodes are equal in terms of having the data, so no lagging secondaries, 24/7 availability, etc.) • This is a core feature of the product by design, has automatic transaction conflict detection and management, and your application can issue any transaction to any Galera Cluster node. Works well in WAN/Clouds • Parallel replication • You do not need automatic failovers via a framework, no need to designate single nodes for writes and the rest for reads, configuration is simple, easier handling of scheduled downtime !7
Galera Cluster optimised for the cloud • Optimised network protocol as packets are only exchanged over the WAN at transaction commit time • Topology aware replication, so each transaction is sent to the data centre only once • Detection and automatic eviction of unreliable nodes • eviction if due to network flaps or node failure, will not be able to rejoin without manual intervention • Split brain recovery/management • Traffic encryption (key in the cloud) !8
Regions • Region: a data centre location, containing multiple Availability Zones !9
Availability Zones !10 • Availability Zone (AZ): isolated from failures from other AZs + low-latency network connectivity to other zones in same region
Running Galera Cluster in the Azure Compute Virtual Machines cloud • The why’s tend to be simple: Galera Cluster is not available on Azure • For regular MySQL, you tend to get more configurability out of Virtual Machines since it is a base OS Linux • You may however have to pay a little more than just a standard Azure Database instance !11
Other considerations • Location, location, location — Microsoft Azure seems to really be available in many locations, currently at 54 regions! • SLAs: "Monthly Uptime Percentage" for the Azure Database for MySQL/MariaDB is calculated as Maximum Available Minutes less Downtime divided by Maximum Available Minutes. • < 99.99% (4.38 min/mo) gives you 10% service credit, < 99% (7.31 hours/mo) gives you 25% service credit and < 95% (36.53 hours/mo) gives you 100% service credit • Support: forums are free, developer support starts at $29/mo, goes up to $100, $1000, and more for Premier support. Response times differ • Management: self/partners !12
Backup and Restore • Backups • full backups occur weekly, differential backups occur twice a day, and transaction log backups occur every five minutes • local or geo-redundant backup options • Azure Database for MySQL provides up to 100% of your provisioned server storage as backup storage at no additional cost. (so provision a 50GB database and you have 50GB of backup storage available gratis; easily good for a week of backups) • Restores? • Point-in-time restore creates a new server in same region • Geo-restore allows you to restore in a different region  !13
Monitoring • Azure metrics have a one-minute frequency, and each metric provides 30 days of history. Alerts can be configured on metrics. • There is Query Store to track query performance over time! • Query Performance Insights to help track long running queries over time, and how they change !14
Costs! • D4 v3 — 4 vCPUs, 16 GB RAM • $0.234/hour (not including disks) • Gen5 - 4 vCores, 5GB RAM/vCore • $0.3504/hour (not including disks) !15
Understanding the different pricing structures !16
Costs II • Monthly cost of running 3 Galera Cluster nodes with 128GB of storage: $541.46 • Monthly cost of running 3 hosted MySQL instances with 128GB of storage: $792.10 !17
So… • Who runs/manages Galera Cluster in an Azure Compute instance? You do. • What does it take to run Galera Cluster in an Azure Compute instance? Not much beyond the usual setup. • Where do you run Galera Cluster in Azure? In an Azure Computer Virtual Machine instance. • When do you run Galera Cluster in Azure? When you feel the need for virtually synchronous replication, automatic node management, etc. • Why do you run Galera Cluster in Azure? For the features, of course! • How do you run Galera Cluster in Azure? Just setup VMs !18
Types of VMs available !19
Picking an OS !20
!21
Storage !22
Storage II !23
Networking !24
Networking • SSH for login • TCP for MySQL is 3306 • TCP ports for Galera Cluster is 4444 (incremental state transfers), 4567 (communications), 4568 (state snapshot transfers) • UDP port for Galera Cluster is 4567 (communications) !25
SELinux & Firewalls • setenforce 0 • However you can open ports for Galera Cluster: • semanage port -a -t mysqld_port_t -p tcp 3306 / semange permissive - a mysqld_t • Similar for the firewall • firewall-cmd --zone=public --add-service=mysql --permanent (add the ports via —add-port=3306/tcp) / firewall-cmd —reload !26
Proxies like ProxySQL • This will take another instance • Has native Galera Cluster hostgroup support • Works with Amazon RDS & Aurora (but nothing natively Azure) • https://aws.amazon.com/blogs/database/how-to-use- proxysql-with-open-source-platforms-to-split-sql-reads- and-writes-on-amazon-aurora-clusters/ • https://aws.amazon.com/blogs/database/supercharge- your-amazon-rds-for-mysql-deployment-with-proxysql- and-percona-monitoring-and-management/ !27
Azure Database for MySQL/MariaDB • Offers MySQL (5.6/5.7, 8.0 in preview) and MariaDB Server (10.2/10.3, missing 10.4) • Upgrades to major versions are dump/restore operations • Fully managed for you • It has an automatic upgrade window • Note limitations (no SUPER access), connection limits • Everything can be done easily within a GUI… you tradeoff control of the database for ease of use & management !28
Underlying replication types • Azure Database for MySQL / MariaDB Server — asynchronous replication (or… is it?) • The read replica feature allows you to replicate data from an Azure Database for MySQL server to a read-only server. You can replicate from the master server to up to five replicas. Replicas are updated asynchronously using the MySQL engine's native binary log (binlog) file position-based replication technology.  • roll-your-own Galera Cluster — virtually synchronous replication • (only Google Cloud SQL uses semi-synchronous replication) • automated services with asynchronous replication? !29
High Availability with Azure MySQL/MariaDB • The high availability (HA) model is based on built-in fail-over mechanisms when a node-level interruption occurs. A node-level interruption could occur because of a hardware failure or in response to a service deployment. • At all times, changes made to an Azure Database for MySQL database server occur in the context of a transaction. Changes are recorded synchronously in Azure storage when the transaction is committed. If a node-level interruption occurs, the database server automatically creates a new node and attaches data storage to the new node. Any active connections are dropped and any inflight transactions are not committed. !30
Resources • Azure Cost Calculator: https://azure.microsoft.com/en-in/pricing/ calculator/ !31
Things we should think about in the future… • Microsoft Azure Marketplace image to roll this out easier? • Kubernetes support? • What else would you like? • Benchmarks? !32
Questions? Colin Charles, colin.charles@galeracluster.com https://twitter.com/galeracluster | www.galeracluster.com !33

More Related Content

Running Galera Cluster on Microsoft Azure

  • 1. Running Galera Cluster Microsoft Azure and comparing it to their hosted services Colin Charles, colin.charles@galeracluster.com 10 September 2019 https://twitter.com/galeracluster | www.galeracluster.com Codership Webinar
  • 2. Agenda • Running Galera Cluster in the the Microsoft Azure cloud • Galera Cluster vs Azure Database for MySQL / MariaDB Server
  • 3. WARNING: NO DEMO! • We will have a recorded video up on the website as a resource • https://galeracluster.com/videos/
  • 4. Codership • The developers and experts of Galera Cluster • Established in 2007, 3 founders, all engineers • Seppo Jaakola, CEO, Teemu Ollakka, CTO, Alex Yurchenko, architect • Services business model, producing 100% open source software • Thousands of users in various industries: e-commerce, betting/gambling, telecoms, banking, insurance, gaming, healthcare, media, marketing, advertising, travel, education, software as a service, PaaS, IaaS, etc. !4
  • 5. !5
  • 6. !6
  • 7. Galera Cluster is all about multi-master replication • Can be described as virtually synchronous replication • High Availability with no data loss, and consistent data across all nodes — no Single Point of Failure (SPoF) • Quorum based failure handling • Optimistic concurrency control • 100% multi-master cluster (all nodes are equal in terms of having the data, so no lagging secondaries, 24/7 availability, etc.) • This is a core feature of the product by design, has automatic transaction conflict detection and management, and your application can issue any transaction to any Galera Cluster node. Works well in WAN/Clouds • Parallel replication • You do not need automatic failovers via a framework, no need to designate single nodes for writes and the rest for reads, configuration is simple, easier handling of scheduled downtime !7
  • 8. Galera Cluster optimised for the cloud • Optimised network protocol as packets are only exchanged over the WAN at transaction commit time • Topology aware replication, so each transaction is sent to the data centre only once • Detection and automatic eviction of unreliable nodes • eviction if due to network flaps or node failure, will not be able to rejoin without manual intervention • Split brain recovery/management • Traffic encryption (key in the cloud) !8
  • 9. Regions • Region: a data centre location, containing multiple Availability Zones !9
  • 10. Availability Zones !10 • Availability Zone (AZ): isolated from failures from other AZs + low-latency network connectivity to other zones in same region
  • 11. Running Galera Cluster in the Azure Compute Virtual Machines cloud • The why’s tend to be simple: Galera Cluster is not available on Azure • For regular MySQL, you tend to get more configurability out of Virtual Machines since it is a base OS Linux • You may however have to pay a little more than just a standard Azure Database instance !11
  • 12. Other considerations • Location, location, location — Microsoft Azure seems to really be available in many locations, currently at 54 regions! • SLAs: "Monthly Uptime Percentage" for the Azure Database for MySQL/MariaDB is calculated as Maximum Available Minutes less Downtime divided by Maximum Available Minutes. • < 99.99% (4.38 min/mo) gives you 10% service credit, < 99% (7.31 hours/mo) gives you 25% service credit and < 95% (36.53 hours/mo) gives you 100% service credit • Support: forums are free, developer support starts at $29/mo, goes up to $100, $1000, and more for Premier support. Response times differ • Management: self/partners !12
  • 13. Backup and Restore • Backups • full backups occur weekly, differential backups occur twice a day, and transaction log backups occur every five minutes • local or geo-redundant backup options • Azure Database for MySQL provides up to 100% of your provisioned server storage as backup storage at no additional cost. (so provision a 50GB database and you have 50GB of backup storage available gratis; easily good for a week of backups) • Restores? • Point-in-time restore creates a new server in same region • Geo-restore allows you to restore in a different region  !13
  • 14. Monitoring • Azure metrics have a one-minute frequency, and each metric provides 30 days of history. Alerts can be configured on metrics. • There is Query Store to track query performance over time! • Query Performance Insights to help track long running queries over time, and how they change !14
  • 15. Costs! • D4 v3 — 4 vCPUs, 16 GB RAM • $0.234/hour (not including disks) • Gen5 - 4 vCores, 5GB RAM/vCore • $0.3504/hour (not including disks) !15
  • 16. Understanding the different pricing structures !16
  • 17. Costs II • Monthly cost of running 3 Galera Cluster nodes with 128GB of storage: $541.46 • Monthly cost of running 3 hosted MySQL instances with 128GB of storage: $792.10 !17
  • 18. So… • Who runs/manages Galera Cluster in an Azure Compute instance? You do. • What does it take to run Galera Cluster in an Azure Compute instance? Not much beyond the usual setup. • Where do you run Galera Cluster in Azure? In an Azure Computer Virtual Machine instance. • When do you run Galera Cluster in Azure? When you feel the need for virtually synchronous replication, automatic node management, etc. • Why do you run Galera Cluster in Azure? For the features, of course! • How do you run Galera Cluster in Azure? Just setup VMs !18
  • 19. Types of VMs available !19
  • 21. !21
  • 25. Networking • SSH for login • TCP for MySQL is 3306 • TCP ports for Galera Cluster is 4444 (incremental state transfers), 4567 (communications), 4568 (state snapshot transfers) • UDP port for Galera Cluster is 4567 (communications) !25
  • 26. SELinux & Firewalls • setenforce 0 • However you can open ports for Galera Cluster: • semanage port -a -t mysqld_port_t -p tcp 3306 / semange permissive - a mysqld_t • Similar for the firewall • firewall-cmd --zone=public --add-service=mysql --permanent (add the ports via —add-port=3306/tcp) / firewall-cmd —reload !26
  • 27. Proxies like ProxySQL • This will take another instance • Has native Galera Cluster hostgroup support • Works with Amazon RDS & Aurora (but nothing natively Azure) • https://aws.amazon.com/blogs/database/how-to-use- proxysql-with-open-source-platforms-to-split-sql-reads- and-writes-on-amazon-aurora-clusters/ • https://aws.amazon.com/blogs/database/supercharge- your-amazon-rds-for-mysql-deployment-with-proxysql- and-percona-monitoring-and-management/ !27
  • 28. Azure Database for MySQL/MariaDB • Offers MySQL (5.6/5.7, 8.0 in preview) and MariaDB Server (10.2/10.3, missing 10.4) • Upgrades to major versions are dump/restore operations • Fully managed for you • It has an automatic upgrade window • Note limitations (no SUPER access), connection limits • Everything can be done easily within a GUI… you tradeoff control of the database for ease of use & management !28
  • 29. Underlying replication types • Azure Database for MySQL / MariaDB Server — asynchronous replication (or… is it?) • The read replica feature allows you to replicate data from an Azure Database for MySQL server to a read-only server. You can replicate from the master server to up to five replicas. Replicas are updated asynchronously using the MySQL engine's native binary log (binlog) file position-based replication technology.  • roll-your-own Galera Cluster — virtually synchronous replication • (only Google Cloud SQL uses semi-synchronous replication) • automated services with asynchronous replication? !29
  • 30. High Availability with Azure MySQL/MariaDB • The high availability (HA) model is based on built-in fail-over mechanisms when a node-level interruption occurs. A node-level interruption could occur because of a hardware failure or in response to a service deployment. • At all times, changes made to an Azure Database for MySQL database server occur in the context of a transaction. Changes are recorded synchronously in Azure storage when the transaction is committed. If a node-level interruption occurs, the database server automatically creates a new node and attaches data storage to the new node. Any active connections are dropped and any inflight transactions are not committed. !30
  • 31. Resources • Azure Cost Calculator: https://azure.microsoft.com/en-in/pricing/ calculator/ !31
  • 32. Things we should think about in the future… • Microsoft Azure Marketplace image to roll this out easier? • Kubernetes support? • What else would you like? • Benchmarks? !32