SlideShare a Scribd company logo
Scaling & High Availability MySQL
learnings from the past decade+
Colin Charles, Chief Evangelist, Percona Inc.

colin.charles@percona.com / byte@bytebot.net

http://www.bytebot.net/blog/ | @bytebot on Twitter

OSDC.de, Berlin, Germany

12 June 2018
whoami
• Chief Evangelist, Percona Inc

• Focusing on the MySQL ecosystem (MySQL, Percona Server, MariaDB
Server), as well as the MongoDB ecosystem (Percona Server for
MongoDB) + 100% open source tools from Percona like Percona
Monitoring & Management, Percona xtrabackup, Percona Toolkit, etc.

• Founding team of MariaDB Server (2009-2016), previously at Monty
Program Ab, merged with SkySQL Ab, now MariaDB Corporation

• Formerly MySQL AB (exit: Sun Microsystems)

• Past lives include The Fedora Project (FESCO), OpenOffice.org

• MySQL Community Contributor of the Year Award winner 2014
License
• Creative Commons BY-NC-SA 4.0

• https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode 

OSDC 2018 | Scaling & High Availability MySQL learnings from the past decade+ by Colin Charles

Recommended for you

OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and S...
OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and  S...OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and  S...
OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and S...

VMWare is a common hypervisor choice in large organizations, and it comes with a zoo of additional tools, options and licenses. But once you add a little bit of OpenSource to the mix, things start to get interesting. There are open APIs which integrate well with Tools like Ansible and InSpec. So it is easy to write your own scripts to verify a configuration or harden a ESX host. We will show you, how we automated our deployment, what problems we encountered and how we added some nice features.

martin schurzsebastian gumprichvmware
Building stateful applications on Kubernetes with Rook
Building stateful applications on Kubernetes with RookBuilding stateful applications on Kubernetes with Rook
Building stateful applications on Kubernetes with Rook

Deploying stateful applications such a Wordpress and Jenkins on top of Kubernetes or any other container orchestrator can be a challenging task. In this context, Rook will be used to showcase how to automatically manage the volume's lifecycle through the its Kubernetes operators (operator pattern approach) by leveraging the recently added CSI GA support.

dockerkubernetesrook
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.

Opcito Technologies is a proud partner with Kubernetes, an open-source system for container orchestration. We will be talking about: • Features of Kubernetes 1.6 • RBAC Configurations • RBAC Use Cases • Running Prometheus in Kubernetes • Prometheus Operator - Deployment, Cluster & Service Monitoring

rbackubernetesprometheus
OSDC 2018 | Scaling & High Availability MySQL learnings from the past decade+ by Colin Charles
OSDC 2018 | Scaling & High Availability MySQL learnings from the past decade+ by Colin Charles
Uptime
Percentile target Max downtime per year
90% 36 days
99% 3.65 days
99.5% 1.83 days
99.9% 8.76 hours
99.99% 52.56 minutes
99.999% 5.25 minutes
99.9999% 31.5 seconds
Estimates of levels of availability
Method
Level of
Availability
Simple replication 98-99.9%
Master-Master/MMM 99%
SAN 99.5-99.9%
DRBD, MHA 99.9%
NDBCluster, Galera Cluster,
InnoDB Cluster
99.999%

Recommended for you

How to Prepare for CKA Exam
How to Prepare for CKA ExamHow to Prepare for CKA Exam
How to Prepare for CKA Exam

The document provides information and tips for preparing for the CKA (Certified Kubernetes Administrator) exam. It outlines what to know prior to taking the exam such as it being offered by CNCF, the exam format, curriculum covered, and required equipment. It recommends practicing Linux CLI, kubectl commands, getting familiar with Kubernetes documentation, examples and APIs. Suggested study methods include using Minikube, Kube-Ansible, online courses, and YouTube videos. Registering for the exam and free learning materials are also mentioned.

kubernetesckainwinstack
Fabric8 - Being devOps doesn't suck anymore
Fabric8 - Being devOps doesn't suck anymoreFabric8 - Being devOps doesn't suck anymore
Fabric8 - Being devOps doesn't suck anymore

Fabric8 is a tool that aims to reduce the gap between development and operations by allowing developers to deploy and manage applications from development through production. It uses profiles and containers to deploy applications in a unified way across environments. Fabric8 supports various containers like Tomcat, Docker, and OpenShift. It includes integrated monitoring via Hawt.io and a shell to help with scripting and diagnostics. The goal is to make developers more self-sufficient by enabling them to take on more operations tasks in their own sandboxes.

Scale Kubernetes to support 50000 services
Scale Kubernetes to support 50000 servicesScale Kubernetes to support 50000 services
Scale Kubernetes to support 50000 services

Kubernetes currently has two load balancing mode: userspace and IPTables. They both have limitation on scalability and performance. We introduced IPVS as third kube-proxy mode which scales kubernetes load balancer to support 50,000 services. Beyond that, control plane needs to be optimized in order to deploy 50,000 services. We will introduce alternative solutions and our prototypes with detailed performance data.

networkingcontainersopen source
HA is Redundancy
• RAID: disk crashes? Another works

• Clustering: server crashes? Another works

• Power: fuse blows? Redundant power supplies

• Network: Switch/NIC crashes? 2nd network route

• Geographical: Datacenter offline/destroyed? Computation to another
DC
Durability
• Data stored on disks

• Is it really written to the disk?

▪ being durable means calling fsync() on each commit

• Is it written in a transactional way to guarantee atomicity, crash
safety, integrity?
High Availability for databases
• HA is harder for databases

• Hardware resources and data need to be redundant

• Remember, this isn’t just data - constantly changing data

• HA means the operation can continue uninterrupted, not by
restoring a new/backup server

• uninterrupted: measured in percentiles
Redundancy through client-side XA
transactions
• Client writes to 2 independent but identical databases

• HA-JDBC (http://ha-jdbc.github.io/) 

• No replication anywhere

Recommended for you

Kubernetes Introduction & Whats new in Kubernetes 1.6
Kubernetes Introduction & Whats new in Kubernetes 1.6Kubernetes Introduction & Whats new in Kubernetes 1.6
Kubernetes Introduction & Whats new in Kubernetes 1.6

- Introduction to Kubernetes features - A look at Kubernetes Networking and Service Discovery - New features in Kubernetes 1.6 - Kubernetes Installation options To know more about our Kubernetes expertise, visit our center of excellence at: http://www.opcito.com/kubernetes/

containeropsmicroservicesautomation
Linuxcon secureefficientcontainerimagemanagementharbor
Linuxcon secureefficientcontainerimagemanagementharborLinuxcon secureefficientcontainerimagemanagementharbor
Linuxcon secureefficientcontainerimagemanagementharbor

This document discusses container image management in enterprises and introduces Project Harbor, an open source container registry. It covers key topics like container image basics, Project Harbor features, maintaining consistency of images between environments, security and access control of images, image distribution strategies, and high availability of container registries. Project Harbor provides an enterprise-grade private registry with features for user management, image replication, security, and integration with systems like LDAP. It aims to help organizations securely manage the distribution and storage of container images.

open sourcesecuritycontainers
Best practices in Deploying SUSE CaaS Platform v3
Best practices in Deploying SUSE CaaS Platform v3Best practices in Deploying SUSE CaaS Platform v3
Best practices in Deploying SUSE CaaS Platform v3

The document provides best practices for deploying SUSE CaaS Platform. It discusses requirements like hardware needs, software subscriptions required, and support options. It covers planning and sizing considerations like cluster topology and disk space needs. Deployment best practices include steps like preparing the infrastructure, installing base software, verifying the infrastructure, installing CaaS Platform, and deploying Kubernetes addons. Testing and operations topics like monitoring, logging, backups are also covered.

susecaaspkubernetes
InnoDB “recovery” time
• innodb_log_file_size
• larger = longer recovery times

• In 2007, Wikipedia reported 40 minutes with 256MB logs;
sometimes it takes 5-10 minutes, sometimes hours

• Percona Server 5.5 (XtraDB) - innodb_recovery_stats
• But today, there has been paradigm change…
Redundancy through shared storage
• Requires specialist hardware, like a SAN

• Complex to operate

• One set of data is your single point of failure

• Cold standby

• failover 1-30 minutes

• this isn’t scale-out

• Active/Active solutions: Oracle RAC
Redundancy through disk replication
• DRBD

• Linux administration vs. DBA skills

• Synchronous

• Second set of data inaccessible for use

• Passive server acting as hot standby

• Failover: 1-30 minutes

• Performance hit compared to single node performance, with higher
average latencies
Redundancy through MySQL replication
• MySQL replication

• Tungsten Replicator

• Galera Cluster

• MySQL InnoDB Cluster

• MySQL Cluster (NDBCLUSTER)

• Storage requirements are multiplied

• Huge potential for scaling out

Recommended for you

Open shift enterprise 3.1 paas on kubernetes
Open shift enterprise 3.1   paas on kubernetesOpen shift enterprise 3.1   paas on kubernetes
Open shift enterprise 3.1 paas on kubernetes

The document provides an overview of Red Hat OpenShift Enterprise, which is Red Hat's container application platform. It discusses key OpenShift concepts like Docker containers, Kubernetes cluster management, projects for sandboxing environments, image builds, deployments, services and routes. It also summarizes added value of OpenShift over plain Docker and Kubernetes through features like templates, web console, multi-language support, automated builds and deployments, and enterprise capabilities.

dockerkubernetesopenshift
Service Discovery In Kubernetes
Service Discovery In KubernetesService Discovery In Kubernetes
Service Discovery In Kubernetes

Service Discovery in kubernetes is all about how services of kubernetes get discovered internally and externally. How does a single POD communicate to another POD the within the cluster and how does a user request reach to a specific POD in the cluster? These are some questions that are answered by this TOPIC.

devopskubernetescontainers
HPC in a Box - Docker Workshop at ISC 2015
HPC in a Box - Docker Workshop at ISC 2015HPC in a Box - Docker Workshop at ISC 2015
HPC in a Box - Docker Workshop at ISC 2015

In this video from the Docker Workshop at ISC 2015, Christian Kniep from QNIB Solutions shows how he uses Docker in his efforts to provide a HPC software stack in a box, encapsulating each layer in the HPC stack within a Linux Container. Watch the video presentation: http://wp.me/p3RLHQ-eos Learn more: http://qnib.org/about/

dockerinsidehpcdocker workshop
MySQL Replication
• Statement based generally

• Row based became available in 5.1, and the default in 5.7

• mixed-mode, resulting in STATEMENT except if calling

▪ UUID function, UDF, CURRENT_USER/USER function, LOAD_FILE
function

▪ 2 or more AUTO_INCREMENT columns updated with same statement

▪ server variable used in statement

▪ storage engine doesn’t allow statement based replication, like
NDBCLUSTER

▪ default in MariaDB Server 10.2
MySQL Replication II
• Asynchronous by default

• Semi-synchronous plugin in 5.5+

• However the holy grail of fully synchronous replication is not part of
standard MySQL replication (yet?)

• MariaDB Galera Cluster is built-in to MariaDB Server 10.1

• MySQL InnoDB Cluster is available in MySQL 8.0, combining the
likes of group replication (the Galera Cluster equivalent), InnoDB,
and mysqlsh for maintaining it; it also uses MySQL Router for load
balancing purposes
Semi-synchronous replication
• semi-sync capable slave acknowledges transaction event only after
written to relay log & flushed to disk

• timeout occurs? master reverts to async replication; resumes when
slaves catch up

• at scale, Facebook runs semi-sync: http://
yoshinorimatsunobu.blogspot.com/2014/04/semi-synchronous-
replication-at-facebook.html
MySQL Replication in 5.6
• Global Transaction ID (GTID)

• Server UUID

• Ignore (master) server IDs (filtering)

• Per-schema multi-threaded slave

• Group commit in the binary log

• Binary log (binlog) checksums

• Crash safe binlog and relay logs

• Time delayed replication

• Parallel replication (per database)

Recommended for you

Rethinking the OS
Rethinking the OSRethinking the OS
Rethinking the OS

Presentation delivered at LinuxCon China 2017. Rethinking the Operating System. A new wave of Operating Systems optimized for containers appeared on the horizon making us excited and puzzeled at the same time. "Why do we need anything different for containers when traditional OSs served us well in the last 25+ years?" "Isn't Kubernetes just another package to install on top of my favorite distro?"" Will this obsolete my whole infrastructure?" are some of the questions this talk will shed some light on. Explore the journey SUSE made in rethinking the OS: From a conservative linux distribution to a platform that goes hand in hand with the needs of Microservices. You will get an insight at what lessons were learned during the intense development effort that lead to SUSE Containers as a Service Platform, how the obstacles along the way were lifted and why "Upstream first" is - and should always be - the rule.

open sourcechina
Docker for HPC in a Nutshell
Docker for HPC in a NutshellDocker for HPC in a Nutshell
Docker for HPC in a Nutshell

In this deck from the Docker Workshop at ISC 2015, Andreas Schmidt from Cassini Consulting describes Docker in a Nutshell "As the newest flavor of Linux Containers, Docker gained a lot of momentum in the last 12 months. With a very convenient and open API-driven architecture Docker is able to help decrease the complexity of operations and increase the productivity of computation. During the last two years Andreas, Christian, and Wolfgang gained a lot of experience with Docker and were thrilled by its possible impact early on. Andreas started working with Docker in mid-2013 and is interested in developing tools for solving Enterprise IT requirements on networking and security. In 2014 he held talks and workshops about these topics. Christian started using Docker in 2013 to virtualize a complete HPC cluster stack and since then held multiple talks about how Docker might impact HPC. Wolfgang and his partner Burak Yenier introduced Docker as a corner-stone of the UberCloud Marketplace to drastically improve and simplify access to HPC cloud resources. UberCloud just announced their new containers for computational fluid dynamics software like Fluent, STAR-CCM+ and OpenFOAM." Watch the video presentation: http://wp.me/p3RLHQ-enP Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

isc 2015insidehpcdocker hpc
Releasing a Distribution in the Age of DevOps.
Releasing a Distribution in the Age of DevOps. Releasing a Distribution in the Age of DevOps.
Releasing a Distribution in the Age of DevOps.

Presentation delivered at LinuxCon China 2017. Operating systems need to move faster without sacrificing stability. New hardware, new software features, and bugfixes are making it into distribution components every day. To maintain stability, packagers and distribution developers are looking toward lessons learned in the DevOps movement to implement Continuous Integration/Continuous Delivery (CI/CD) workflows that provide quicker test feedback to developers. This talk highlights some of the coming trends in Fedora such as: streamlined base package sets, userspace applications delivered as containers, continuous validation of individual distro components and the distro as a whole, and collaboration with the CentOS Project.

open sourcelinux
Replication: START TRANSACTION WITH
CONSISTENT SNAPSHOT
• Works with the binlog, possible to obtain the binlog position corresponding to
a transactional snapshot of the database without blocking any other queries. 

• by-product of group commit in the binlog to view commit ordering

• Used by the command mysqldump--single-transaction --master-
data to do a fully non-blocking backup 

• Works consistently between transactions involving more than one storage
engine

• https://kb.askmonty.org/en/enhancements-for-start-transaction-with-
consistent/

• Percona Server improved it, by session ID, and also introducing backup locks
Multi-source replication
• Multi-source replication - (real-time) analytics, shard provisioning,
backups, etc.

• @@default_master_connection contains current connection name
(used if connection name is not given)

• All master/slave commands take a connection name now (like CHANGE
MASTER “connection_name”, SHOW SLAVE “connection_name”
STATUS, etc.)
Global Transaction ID (GTID)
• Supports multi-source replication

• GTID can be enabled or disabled independently and online for masters or
slaves

• Slaves using GTID do not have to have binary logging enabled.

• (MariaDB) Supports multiple replication domains (independent binlog streams)

• Queries in different domains can be run in parallel on the slave.
Why MariaDB GTID is different compared to
5.6?
• MySQL 5.6 GTID does not support multi-source replication (only 5.7
supports this)

• Supports —log-slave-updates=0 for efficiency (like 5.7)

• Enabled by default

• Turn it on without having to restart the topology (just like 5.7)

Recommended for you

AWS Lambda and serverless Java | DevNation Live
AWS Lambda and serverless Java | DevNation LiveAWS Lambda and serverless Java | DevNation Live
AWS Lambda and serverless Java | DevNation Live

Have you ever tried Java on AWS Lambda but found that the cold-start latency and memory usage were far too high? In this session, we will show how we optimized Java for serverless applications by leveraging GraalVM with Quarkus to provide both supersonic startup speed and a subatomic memory footprint.

developerred hatquarkus
SUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on Kubernetes
SUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on KubernetesSUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on Kubernetes
SUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on Kubernetes

This document discusses potential use cases for Kubernetes and provides examples of deploying serverless/Function as a Service (FaaS) workloads and blockchain databases on Kubernetes. It introduces OpenFaaS as an easy way to deploy FaaS on Kubernetes and deploy a demo Ethereum blockchain on Kubernetes to illustrate how blockchain concepts map to Kubernetes components. The document encourages finding a new use case in your organization to start using Kubernetes and provides resources for learning more about deploying Kubernetes.

susekubernetesethereum
Best practices for MySQL High Availability Tutorial
Best practices for MySQL High Availability TutorialBest practices for MySQL High Availability Tutorial
Best practices for MySQL High Availability Tutorial

Given at O'Reilly Velocity London in October 2017, this tutorial focuses on the current best practices for MySQL High Availability.

mysqlhigh availabilitymariadb
Parallel replication
• Multi-source replication from different masters executed in parallel 

• Queries from different domains are executed in parallel 

• Queries that are run in parallel on the master are run in parallel on the
slave (based on group commit).

• Transactions modifying the same table can be updated in parallel on
the slave! 

• Supports both statement based and row based replication.
All in… sometimes it can get out of sync
• Changed information on slave directly

• Statement based replication

• non-deterministic SQL (UPDATE/DELETE with LIMIT and without ORDER BY)

• triggers & stored procedures

• Master in MyISAM, slave in InnoDB (deadlocks)

• --replication-ignore-db with fully qualified queries

• Binlog corruption on master

• PURGE BINARY LOGS issued and not enough files to update slave

• read_buffer_size larger than max_allowed_packet

• Bugs?
Replication Monitoring
• Percona Toolkit is important

• pt-slave-find: find slave information from master

• pt-table-checksum: online replication consistency check

• executes checksum queries on master

• pt-table-sync: synchronise table data efficiently

• changes data, so backups important
Replication Monitoring with PMM
•http://pmmdemo.percona.com/

Recommended for you

Best practices for MySQL/MariaDB Server/Percona Server High Availability
Best practices for MySQL/MariaDB Server/Percona Server High AvailabilityBest practices for MySQL/MariaDB Server/Percona Server High Availability
Best practices for MySQL/MariaDB Server/Percona Server High Availability

Best practices for MySQL/MariaDB Server/Percona Server High Availability - presented at Percona Live Amsterdam 2016. The focus is on picking the right High Availability solution, discussing replication, handling failure (yes, you can achieve a quick automatic failover), proxies (there are plenty), HA in the cloud/geographical redundancy, sharding solutions, how newer versions of MySQL help you, and what to watch for next.

high availabilityfailoverplams16
Best practices for MySQL High Availability
Best practices for MySQL High AvailabilityBest practices for MySQL High Availability
Best practices for MySQL High Availability

The MariaDB/MySQL world is full of tradeoffs, and choosing a high availability (HA) solution is no exception. This session aims to look at all the alternatives in an unbiased way. Preference is of course only given to open source solutions. How do you choose between: asynchronous/semi-synchronous/synchronous replication, MHA (MySQL high availability tools), DRBD, Tungsten Replicator, or Galera Cluster? Do you integrate Pacemaker and Heartbeat like Percona Replication Manager? The cloud brings even more fun, especially if you are dealing with a hybrid cloud and must think about geographical redundancy. What about newer solutions like using Consul for MySQL HA? When you’ve decided on your solution, how do you provision and monitor these solutions? This and more will be covered in a walkthrough of MySQL HA options and when to apply them.

mysqlmariadbhigh availability
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)

MariaDB is like the "new" MySQL, and its available everywhere. This talk was given at LinuxCon Europe in Dublin in October 2015. Learn about all the new features, considering the release was just around the corner. Changes in replication are also very interesting

mysqlmariadb
mysqlbinlog versions
• ERROR: Error in Log_event::read_log_event(): 'Found invalid event in
binary log', data_len: 56, event_type: 30

• 5.6 ships with a “streaming binlog backup server” - v.3.4; MariaDB
10 doesn’t - v.3.3 (fixed in 10.2 - MDEV-8713)

• GTID variances!

• Beware mysql-client from your Linux distribution
Slave prefetching
• Replication Booster

• https://github.com/yoshinorim/replication-booster-for-mysql

• Prefetch MySQL relay logs to make the SQL thread faster

• Tungsten has slave prefetch

• Percona Server till 5.6 + MariaDB till 10.1 have InnoDB fake changes
Changing paradigm: What replaces slave
prefetching?
• In Percona Server 5.7, slave prefetching has been replaced by doing
intra-schema parallel replication

• Feature removed from XtraDB

• MariaDB Server 10.2 also has this feature removed, as they switched
to InnoDB!
Galera Cluster
• Inside MySQL, a replication plugin (wsrep)

• Replaces MySQL replication (but can work alongside it too)

• True multi-master, active-active solution

• Virtually Synchronous

• WAN performance: 100-300ms/commit, works in parallel

• No slave lag or integrity issues

• Automatic node provisioning

Recommended for you

MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
MariaDB 10.1   what's new and what's coming in 10.2 - Tokyo MariaDB MeetupMariaDB 10.1   what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup

Presented at the Tokyo MariaDB Server meetup in July 2016, this is an overview of what you can see and use in MariaDB Server 10.1, but more importantly what is planned to arrive in 10.2

mariadb servermysqlmariadb
[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale by ...
[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale  by ...[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale  by ...
[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale by ...

Scalability with MariaDB and MaxScale talks about MariaDB 10, and MaxScale, a pluggable router for your queries. These are technologies developed at MariaDB Corporation, made opensource, and will help scale your MariaDB and MySQL workloads

dbts-tokyo-2014
Meet MariaDB 10.1 at the Bulgaria Web Summit
Meet MariaDB 10.1 at the Bulgaria Web SummitMeet MariaDB 10.1 at the Bulgaria Web Summit
Meet MariaDB 10.1 at the Bulgaria Web Summit

Meet MariaDB 10.1 at the Bulgaria Web Summit, held in Sofia in February 2016. Learn all about MariaDB Server, and the new features like encryption, audit plugins, and more.

mariadb servermysqlmariadb
OSDC 2018 | Scaling & High Availability MySQL learnings from the past decade+ by Colin Charles
Percona XtraDB Cluster 5.7
• Engineering within Percona

• Load balancing with ProxySQL (bundled)

• PMM integration

• Benefits of all the MySQL 5.7 feature-set

• ProxySQL admin tool

• Safety features enabled (e.g. no accidentally using MyISAM, etc.)
Group replication
• Fully synchronous replication (update everywhere), self-healing, with
elasticity, redundancy

• Single primary mode supported

• MySQL InnoDB Cluster - a combination of group replication, Router, to
make magic!

• Recent blogs:

• https://www.percona.com/blog/2017/02/24/battle-for-synchronous-
replication-in-mysql-galera-vs-group-replication/

• https://www.percona.com/blog/2017/02/15/group-replication-shipped-
early/
Summary of Replication Performance
• SAN has "some" latency overhead compared to local disk. Can be
great for throughput.

• DRBD has a performance penalty

• Replication, when implemented correctly, has no performance penalty

• But MySQL replication with disk bound data set has single-threaded
issues!

• Semi-sync is poorer on WAN compared to async

• Galera & InnoDB Cluster provide read/write scale-out, thus more
performance

Recommended for you

MariaDB 10: A MySQL Replacement - HKOSC
MariaDB 10: A MySQL Replacement - HKOSC MariaDB 10: A MySQL Replacement - HKOSC
MariaDB 10: A MySQL Replacement - HKOSC

MariaDB 10: A MySQL Replacement. Current up to 10.0.9, right before the 10.0.10 GA release presented the weekend before the release in Hong Kong, at the Hong Kong Open Source Conference.

mysqlhkoscmariadb
Why MariaDB?
Why MariaDB?Why MariaDB?
Why MariaDB?

The myriad reasons why you want to use MariaDB over stock MySQL. Current up to MariaDB 5.3, and presented at Percona Live London 2011.

percona livemariadbpercona live london
Maria db 10 and the mariadb foundation(colin)
Maria db 10 and the mariadb foundation(colin)Maria db 10 and the mariadb foundation(colin)
Maria db 10 and the mariadb foundation(colin)

This document provides an overview of MariaDB 10 and the MariaDB Foundation. It discusses the history and development of MariaDB, including key features added in versions 5.1 through 10.0 such as new storage engines, performance improvements, and features backported from MySQL. It outlines the goals of MariaDB to be compatible with MySQL while adding new features, and describes the community-led development model. The roadmap aims to have MariaDB be a drop-in replacement for MySQL 5.6 by releasing version 10.1.

Handling failure
• How do we find out about failure?

• Polling, monitoring, alerts...

• Error returned to and handled in client side

• What should we do about it?

• Direct requests to the spare nodes (or DCs)

• How to protect data integrity?

• Master-slave is unidirectional: Must ensure there is only one master at all times.

• DRBD and SAN have cold-standby: Must mount disks and start mysqld.

• In all cases must ensure that 2 disconnected replicas cannot both commit
independently. (split brain)
Frameworks to handle failure
• MySQL-MMM

• Severalnines ClusterControl

• Orchestrator

• MySQL MHA

• Percona Replication Manager

• Tungsten Replicator

• 5.6: mysqlfailover, mysqlrpladmin

• (MariaDB) Replication Manager
Orchestrator
• Reads replication topologies, keeps state, continuous polling

• Modify your topology — move slaves around

• Nice GUI, JSON API, CLI
MySQL MHA
• Like MMM, specialized solution for MySQL replication

• Developed by Yoshinori Matsunobu at DeNA

• Automated and manual failover options

• Topology: 1 master, many slaves

• Choose new master by comparing slave binlog positions

• Can be used in conjunction with other solutions

• http://code.google.com/p/mysql-master-ha/

Recommended for you

Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...
Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...
Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...

There are many Galera Cluster distributions and sometimes differences are well worth noting. We get a lot of queries about which Galera Cluster to use, or why one should use one distribution over the other. Learn about Galera Cluster with MySQL 5.7 from Codership, and we’ll compare it with Galera Cluster 4 with MariaDB 10.4, and Percona XtraDB Cluster 5.7 with Galera 3. This is also the webinar where we preview Galera Cluster 4 with MySQL 8.0 as well as compare it with the preview release of Percona XtraDB Cluster 8.0. Overall, learn why distributions exists, and how you can get the most out of your Galera Cluster experience.

galera clustermariadb galera clusterpercona xtradb cluster
Lessons from database failures
Lessons from database failuresLessons from database failures
Lessons from database failures

This is my third iteration of the talk presented in Tokyo, Japan - first was at a keynote at rootconf.in in April 2016, then at the MySQL meetup in New York, and now for dbtechshowcase. The focus is on database failures of the past, and how modern MySQL / MariaDB Server technologies could have helped them avoid such failure. The focus is on backups and verification, replication and failover, and security and encryption.

mysqlsecurityencrypytion
The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016

The document summarizes the history and current state of the MySQL database server ecosystem. It discusses the origins and development of MySQL, MariaDB, Percona Server, and other related projects. It also describes some of the key features and innovations in recent versions of these database servers. The ecosystem is very active with contributions from many organizations and the future remains promising with ongoing work.

Pacemaker
• Heartbeat, Corosync, Pacemaker

• Resource Agents, Percona-PRM

• Percona Replication Manager - cluster, geographical disaster
recovery options

• Pacemaker agent specialised on MySQL replication

• https://github.com/percona/percona-pacemaker-agents/ 

• Pacemaker Resource Agents 3.9.3+ include Percona Replication
Manager (PRM)
Load Balancers for multi-master clusters
• Synchronous multi-master clusters like Galera require load balancers

• HAProxy

• Galera Load Balancer (GLB)

• MaxScale

• ProxySQL
MySQL Router
• Routing between applications and any backend MySQL servers

• Failover

• Load Balancing

• Pluggable architecture (connection routing)
MariaDB MaxScale
• “Pluggable router” that offers
connection & statement based
load balancing

• Possibilities are endless - use it
for logging, writing to other
databases (besides MySQL),
preventing SQL injections via
regex filtering, route via hints,
query rewriting, have a binlog
relay, etc.

Recommended for you

The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016

MySQL is a unique adult (now 21 years old) in many ways. It supports plugins. It supports storage engines. It is also owned by Oracle, thus birthing two branches of the popular opensource database: Percona Server and MariaDB Server. It also once spawned a fork: Drizzle. Lately a consortium of web scale users (think a chunk of the top 10 sites out there) have spawned WebScaleSQL. You're a busy DBA having to maintain a mix of this. Or you're a CIO planning to choose one branch. How do you go about picking? Supporting multiple databases? Find out more in this talk. Also covered is a deep-dive into what feature differences exist between MySQL/Percona Server/MariaDB/WebScaleSQL, how distributions package the various databases differently. Within the hour, you'll be informed about the past, the present, and hopefully be knowledgeable enough to know what to pick in the future. Note, there will also be coverage of the various trees around WebScaleSQL, like the Facebook tree, the Alibaba tree as well as the Twitter tree.

ecosystemscale14xscale
OSDC 2017 | Lessons from database failures by Colin Charles
OSDC 2017 | Lessons from database failures by Colin CharlesOSDC 2017 | Lessons from database failures by Colin Charles
OSDC 2017 | Lessons from database failures by Colin Charles

Lets learn from MySQL failures at scale, because we tie in the topic of High Availability, in where people are thinking about geographical redundancy, and even things like automatic failover. In the talk there will be case study material, e.g. where automatic failure caused a site to go offline, where a social network started of with not using fully automated failovers but evolved, etc. How is the MySQL world making things better, for example by allowing you to use semi-synchronous replication to run fully scalable services. The talk starts off with an even almost stupid example of how a business died due to incorrect MySQL backup procedures. It will go on to talk about security and encryption at rest as well. So a mix of problems from the field, big “fail whales”, and how you should avoid them by properly architecting solutions.

osdcopen sourcedata
A beginners guide to MariaDB
A beginners guide to MariaDBA beginners guide to MariaDB
A beginners guide to MariaDB

MariaDB started life as a database to host the Maria storage engine in 2009. Not long after its inception, the MySQL community went through yet another change in ownership, and it was deemed that MariaDB will be a complete database branch developed to extend MySQL, but with constant merging of upstream changes. The goal of the MariaDB project is to ensure that everyone is part of the community, including employees of the major steering companies. MariaDB also features enhanced features, some of which are common with the Percona Performance Server. Most importantly, MariaDB is a drop-in replacement and is completely backward compatible with MySQL. In 2010, MariaDB released 5.1 in February, and 5.2 in November – two major releases in a span of one calendar year is a feat that was achieved! DBAs and developers alike will gain an introduction to MariaDB, what is different with MySQL, how to make use of the feature enhancements, and more.

mariadbmysql
ProxySQL
• High Performance MySQL proxy with a GPL license

• Performance is a priority - the numbers prove it

• Can query rewrite

• Sharding by host/schema or both, with rule engine + modification to
SQL + application logic
JDBC/PHP drivers
• JDBC - multi-host failover feature (just specify master/slave hosts in
the properties)

• true for MariaDB Java Connector too

• PHP handles this too - mysqlnd_ms

• Can handle read-write splitting, round robin or random host
selection, and more
Clustering: solution or part of problem?
• "Causes of Downtime in Production MySQL Servers" whitepaper,
Baron Schwartz VividCortex

• Human error

• SAN

• Clustering framework + SAN = more problems

• Galera/group replication is replication based, has no false positives
as there’s no “failover” moment, you don’t need a clustering
framework (JDBC or PHP can load balance), and is relatively elegant
overall
Replication type
• Competence choices

• Replication: MySQL DBA manages

• DRBD: Linux admin manages

• SAN: requires domain controller

• Operations

• DRBD (disk level) = cold standby =
longer failover

• Replication = hot standby =
shorter failover

• GTID helps tremendously

• Performance

• SAN has higher latency than local
disk

• DRBD has higher latency than
local disk

• Replication has little overhead

• Redundancy

• Shared disk = SPoF

• Shared nothing = redundant

Recommended for you

The MySQL ecosystem - understanding it, not running away from it!
The MySQL ecosystem - understanding it, not running away from it! The MySQL ecosystem - understanding it, not running away from it!
The MySQL ecosystem - understanding it, not running away from it!

You're a busy DBA thinking about having to maintain a mix of this. Or you're a CIO planning to choose one branch over another. How do you go about picking? Supporting multiple databases? Find out more in this talk. Also covered is a deep-dive into what feature differences exist between MySQL/Percona Server/MariaDB Server. Within 20 minutes, you'll leave informed and knowledgable on what to pick. A base blog post to get started: https://www.percona.com/blog/2017/11/02/mysql-vs-mariadb-reality-check/

mysqlmariadbmariadb server
The MySQL Server Ecosystem in 2016
The MySQL Server Ecosystem in 2016The MySQL Server Ecosystem in 2016
The MySQL Server Ecosystem in 2016

This was a short 25 minute talk, but we go into a bit of a history of MySQL, how the branches and forks appeared, what's sticking around today (branch? Percona Server. Fork? MariaDB Server). What should you use? Think about what you need today and what the roadmap holds.

percona livemariadb serverpercona server
MariaDB: The 2012 Edition
MariaDB: The 2012 EditionMariaDB: The 2012 Edition
MariaDB: The 2012 Edition

MariaDB is a community developed branch of MySQL that is feature enhanced and backward compatible. It aims to be a 100% drop-in replacement for MySQL that is stable, bug-free, and released under the GPLv2 license. Major releases of MariaDB include new storage engines like XtraDB and Aria, as well as new features for performance, scalability, and compatibility. MariaDB is developed as an open source project and supported by Monty Program and other community contributors and service providers.

mysqlpercona livemariadb
SBR vs RBR? Async vs sync?
• row based: deterministic

• statement based: dangerous

• GTID: easier setup & failover of complex topologies

• async: data loss in failover

• sync: best

• multi-threaded slaves: scalability (hello 5.6+, Tungsten)
What about the cloud?
• Usually scalability & high availability is more or less “built-in”

• e.g. RDS has multi-AZ (synchronous data replication), but doesn’t
give you a read replica; Cloud SQL uses semi-sync replication

• Watch out for the SLAs (and automatic upgrades)

• Monitoring via PERFORMANCE_SCHEMA

• “Bad” nodes do exist; do not assume node provisioning is quick
Conclusion
• MySQL replication is amazing if you know it (and monitor it) well
enough

• Large sites run just fine with semi-sync + tooling for automated
failover

• Galera Cluster/MySQL InnoDB Cluster is great for virtually fully
synchronous replication

• Don’t forget the need for a load balancer: ProxySQL is nifty

• When thinking scaling, think scale out (it is more efficient, and fits
modern cloud mantras too!)
At Percona, we care about your High
Availability
• Percona XtraDB Cluster 5.7 with support for ProxySQL and Percona
Monitoring & Management (PMM)

• Percona Monitoring & Management (PMM) with Orchestrator

• Percona Toolkit

• Percona Server for MySQL 5.7

• Percona XtraBackup

Recommended for you

MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdfMySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf

MySQL is still hot, with Percona XtraDB Cluster (PXC) and MariaDB Server. Welcome back post-pandemic to see what is on offer in the current ecosystem. Did you know that Amazon RDS now uses semi-sync replication rather than DRBD for multi-AZ deployments? Did you know that Galera Cluster for MySQL 8 is much more efficient with CLONE SST rather than using the xtrabackup method for SST? Did you know that Percona Server continues to extend MyRocks? Did you know that MariaDB Server has more Oracle syntax compatibility? This and more will be covered in the session, while short and quick, should leave you wandering to discover new features for production.

#mysql #mysql8 #mariadb #opensource #databases
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...

AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introduction.pdf

awscloudpractitioner
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理

原版一模一样【微信:741003700 】【英国牛津大学毕业证(oxon毕业证书)成绩单】【微信:741003700 】学位证,留信学历认证(真实可查,永久存档)原件一模一样纸张工艺/offer、在读证明、外壳等材料/诚信可靠,可直接看成品样本,帮您解决无法毕业带来的各种难题!外壳,原版制作,诚信可靠,可直接看成品样本。行业标杆!精益求精,诚心合作,真诚制作!多年品质 ,按需精细制作,24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题,包您满意。 本公司拥有海外各大学样板无数,能完美还原。 1:1完美还原海外各大学毕业材料上的工艺:水印,阴影底纹,钢印LOGO烫金烫银,LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700 【主营项目】 一.毕业证【q微741003700】成绩单、使馆认证、教育部认证、雅思托福成绩单、学生卡等! 二.真实使馆公证(即留学回国人员证明,不成功不收费) 三.真实教育部学历学位认证(教育部存档!教育部留服网站永久可查) 四.办理各国各大学文凭(一对一专业服务,可全程监控跟踪进度) 如果您处于以下几种情况: ◇在校期间,因各种原因未能顺利毕业……拿不到官方毕业证【q/微741003700】 ◇面对父母的压力,希望尽快拿到; ◇不清楚认证流程以及材料该如何准备; ◇回国时间很长,忘记办理; ◇回国马上就要找工作,办给用人单位看; ◇企事业单位必须要求办理的 ◇需要报考公务员、购买免税车、落转户口 ◇申请留学生创业基金 留信网认证的作用: 1:该专业认证可证明留学生真实身份 2:同时对留学生所学专业登记给予评定 3:国家专业人才认证中心颁发入库证书 4:这个认证书并且可以归档倒地方 5:凡事获得留信网入网的信息将会逐步更新到个人身份内,将在公安局网内查询个人身份证信息后,同步读取人才网入库信息 6:个人职称评审加20分 7:个人信誉贷款加10分 8:在国家人才网主办的国家网络招聘大会中纳入资料,供国家高端企业选择人才 办理英国牛津大学毕业证(oxon毕业证书)【微信:741003700 】外观非常简单,由纸质材料制成,上面印有校徽、校名、毕业生姓名、专业等信息。 办理英国牛津大学毕业证(oxon毕业证书)【微信:741003700 】格式相对统一,各专业都有相应的模板。通常包括以下部分: 校徽:象征着学校的荣誉和传承。 校名:学校英文全称 授予学位:本部分将注明获得的具体学位名称。 毕业生姓名:这是最重要的信息之一,标志着该证书是由特定人员获得的。 颁发日期:这是毕业正式生效的时间,也代表着毕业生学业的结束。 其他信息:根据不同的专业和学位,可能会有一些特定的信息或章节。 办理英国牛津大学毕业证(oxon毕业证书)【微信:741003700 】价值很高,需要妥善保管。一般来说,应放置在安全、干燥、防潮的地方,避免长时间暴露在阳光下。如需使用,最好使用复印件而不是原件,以免丢失。 综上所述,办理英国牛津大学毕业证(oxon毕业证书)【微信:741003700 】是证明身份和学历的高价值文件。外观简单庄重,格式统一,包括重要的个人信息和发布日期。对持有人来说,妥善保管是非常重要的。

英国牛津大学毕业证(oxon毕业证书)
Thank you!
Colin Charles
colin.charles@percona.com / byte@bytebot.net
http://bytebot.net/blog | @bytebot on twitter
slides: slideshare.net/bytebot

More Related Content

What's hot

Container Orchestration
Container OrchestrationContainer Orchestration
Container Orchestration
dfilppi
 
Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1
Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1
Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1
Etsuji Nakai
 
OSDC 2018 | Highly Available Cloud Foundry on Kubernetes by Cornelius Schumacher
OSDC 2018 | Highly Available Cloud Foundry on Kubernetes by Cornelius SchumacherOSDC 2018 | Highly Available Cloud Foundry on Kubernetes by Cornelius Schumacher
OSDC 2018 | Highly Available Cloud Foundry on Kubernetes by Cornelius Schumacher
NETWAYS
 
OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and S...
OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and  S...OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and  S...
OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and S...
NETWAYS
 
Building stateful applications on Kubernetes with Rook
Building stateful applications on Kubernetes with RookBuilding stateful applications on Kubernetes with Rook
Building stateful applications on Kubernetes with Rook
Roberto Hashioka
 
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
Opcito Technologies
 
How to Prepare for CKA Exam
How to Prepare for CKA ExamHow to Prepare for CKA Exam
How to Prepare for CKA Exam
Alfie Chen
 
Fabric8 - Being devOps doesn't suck anymore
Fabric8 - Being devOps doesn't suck anymoreFabric8 - Being devOps doesn't suck anymore
Fabric8 - Being devOps doesn't suck anymore
Henryk Konsek
 
Scale Kubernetes to support 50000 services
Scale Kubernetes to support 50000 servicesScale Kubernetes to support 50000 services
Scale Kubernetes to support 50000 services
LinuxCon ContainerCon CloudOpen China
 
Kubernetes Introduction & Whats new in Kubernetes 1.6
Kubernetes Introduction & Whats new in Kubernetes 1.6Kubernetes Introduction & Whats new in Kubernetes 1.6
Kubernetes Introduction & Whats new in Kubernetes 1.6
Opcito Technologies
 
Linuxcon secureefficientcontainerimagemanagementharbor
Linuxcon secureefficientcontainerimagemanagementharborLinuxcon secureefficientcontainerimagemanagementharbor
Linuxcon secureefficientcontainerimagemanagementharbor
LinuxCon ContainerCon CloudOpen China
 
Best practices in Deploying SUSE CaaS Platform v3
Best practices in Deploying SUSE CaaS Platform v3Best practices in Deploying SUSE CaaS Platform v3
Best practices in Deploying SUSE CaaS Platform v3
Juan Herrera Utande
 
Open shift enterprise 3.1 paas on kubernetes
Open shift enterprise 3.1   paas on kubernetesOpen shift enterprise 3.1   paas on kubernetes
Open shift enterprise 3.1 paas on kubernetes
Samuel Terburg
 
Service Discovery In Kubernetes
Service Discovery In KubernetesService Discovery In Kubernetes
Service Discovery In Kubernetes
Knoldus Inc.
 
HPC in a Box - Docker Workshop at ISC 2015
HPC in a Box - Docker Workshop at ISC 2015HPC in a Box - Docker Workshop at ISC 2015
HPC in a Box - Docker Workshop at ISC 2015
inside-BigData.com
 
Rethinking the OS
Rethinking the OSRethinking the OS
Docker for HPC in a Nutshell
Docker for HPC in a NutshellDocker for HPC in a Nutshell
Docker for HPC in a Nutshell
inside-BigData.com
 
Releasing a Distribution in the Age of DevOps.
Releasing a Distribution in the Age of DevOps. Releasing a Distribution in the Age of DevOps.
Releasing a Distribution in the Age of DevOps.
LinuxCon ContainerCon CloudOpen China
 
AWS Lambda and serverless Java | DevNation Live
AWS Lambda and serverless Java | DevNation LiveAWS Lambda and serverless Java | DevNation Live
AWS Lambda and serverless Java | DevNation Live
Red Hat Developers
 
SUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on Kubernetes
SUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on KubernetesSUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on Kubernetes
SUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on Kubernetes
Juan Herrera Utande
 

What's hot (20)

Container Orchestration
Container OrchestrationContainer Orchestration
Container Orchestration
 
Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1
Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1
Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1
 
OSDC 2018 | Highly Available Cloud Foundry on Kubernetes by Cornelius Schumacher
OSDC 2018 | Highly Available Cloud Foundry on Kubernetes by Cornelius SchumacherOSDC 2018 | Highly Available Cloud Foundry on Kubernetes by Cornelius Schumacher
OSDC 2018 | Highly Available Cloud Foundry on Kubernetes by Cornelius Schumacher
 
OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and S...
OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and  S...OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and  S...
OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and S...
 
Building stateful applications on Kubernetes with Rook
Building stateful applications on Kubernetes with RookBuilding stateful applications on Kubernetes with Rook
Building stateful applications on Kubernetes with Rook
 
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
Securing & Monitoring Your K8s Cluster with RBAC and Prometheus”.
 
How to Prepare for CKA Exam
How to Prepare for CKA ExamHow to Prepare for CKA Exam
How to Prepare for CKA Exam
 
Fabric8 - Being devOps doesn't suck anymore
Fabric8 - Being devOps doesn't suck anymoreFabric8 - Being devOps doesn't suck anymore
Fabric8 - Being devOps doesn't suck anymore
 
Scale Kubernetes to support 50000 services
Scale Kubernetes to support 50000 servicesScale Kubernetes to support 50000 services
Scale Kubernetes to support 50000 services
 
Kubernetes Introduction & Whats new in Kubernetes 1.6
Kubernetes Introduction & Whats new in Kubernetes 1.6Kubernetes Introduction & Whats new in Kubernetes 1.6
Kubernetes Introduction & Whats new in Kubernetes 1.6
 
Linuxcon secureefficientcontainerimagemanagementharbor
Linuxcon secureefficientcontainerimagemanagementharborLinuxcon secureefficientcontainerimagemanagementharbor
Linuxcon secureefficientcontainerimagemanagementharbor
 
Best practices in Deploying SUSE CaaS Platform v3
Best practices in Deploying SUSE CaaS Platform v3Best practices in Deploying SUSE CaaS Platform v3
Best practices in Deploying SUSE CaaS Platform v3
 
Open shift enterprise 3.1 paas on kubernetes
Open shift enterprise 3.1   paas on kubernetesOpen shift enterprise 3.1   paas on kubernetes
Open shift enterprise 3.1 paas on kubernetes
 
Service Discovery In Kubernetes
Service Discovery In KubernetesService Discovery In Kubernetes
Service Discovery In Kubernetes
 
HPC in a Box - Docker Workshop at ISC 2015
HPC in a Box - Docker Workshop at ISC 2015HPC in a Box - Docker Workshop at ISC 2015
HPC in a Box - Docker Workshop at ISC 2015
 
Rethinking the OS
Rethinking the OSRethinking the OS
Rethinking the OS
 
Docker for HPC in a Nutshell
Docker for HPC in a NutshellDocker for HPC in a Nutshell
Docker for HPC in a Nutshell
 
Releasing a Distribution in the Age of DevOps.
Releasing a Distribution in the Age of DevOps. Releasing a Distribution in the Age of DevOps.
Releasing a Distribution in the Age of DevOps.
 
AWS Lambda and serverless Java | DevNation Live
AWS Lambda and serverless Java | DevNation LiveAWS Lambda and serverless Java | DevNation Live
AWS Lambda and serverless Java | DevNation Live
 
SUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on Kubernetes
SUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on KubernetesSUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on Kubernetes
SUSE CaaSP: deploy OpenFaaS and Ethereum Blockchain on Kubernetes
 

Similar to OSDC 2018 | Scaling & High Availability MySQL learnings from the past decade+ by Colin Charles

Best practices for MySQL High Availability Tutorial
Best practices for MySQL High Availability TutorialBest practices for MySQL High Availability Tutorial
Best practices for MySQL High Availability Tutorial
Colin Charles
 
Best practices for MySQL/MariaDB Server/Percona Server High Availability
Best practices for MySQL/MariaDB Server/Percona Server High AvailabilityBest practices for MySQL/MariaDB Server/Percona Server High Availability
Best practices for MySQL/MariaDB Server/Percona Server High Availability
Colin Charles
 
Best practices for MySQL High Availability
Best practices for MySQL High AvailabilityBest practices for MySQL High Availability
Best practices for MySQL High Availability
Colin Charles
 
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)
Colin Charles
 
MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
MariaDB 10.1   what's new and what's coming in 10.2 - Tokyo MariaDB MeetupMariaDB 10.1   what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
Colin Charles
 
[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale by ...
[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale  by ...[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale  by ...
[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale by ...
Insight Technology, Inc.
 
Meet MariaDB 10.1 at the Bulgaria Web Summit
Meet MariaDB 10.1 at the Bulgaria Web SummitMeet MariaDB 10.1 at the Bulgaria Web Summit
Meet MariaDB 10.1 at the Bulgaria Web Summit
Colin Charles
 
MariaDB 10: A MySQL Replacement - HKOSC
MariaDB 10: A MySQL Replacement - HKOSC MariaDB 10: A MySQL Replacement - HKOSC
MariaDB 10: A MySQL Replacement - HKOSC
Colin Charles
 
Why MariaDB?
Why MariaDB?Why MariaDB?
Why MariaDB?
Colin Charles
 
Maria db 10 and the mariadb foundation(colin)
Maria db 10 and the mariadb foundation(colin)Maria db 10 and the mariadb foundation(colin)
Maria db 10 and the mariadb foundation(colin)
kayokogoto
 
Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...
Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...
Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...
Codership Oy - Creators of Galera Cluster
 
Lessons from database failures
Lessons from database failuresLessons from database failures
Lessons from database failures
Colin Charles
 
The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016
sys army
 
The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016
Colin Charles
 
OSDC 2017 | Lessons from database failures by Colin Charles
OSDC 2017 | Lessons from database failures by Colin CharlesOSDC 2017 | Lessons from database failures by Colin Charles
OSDC 2017 | Lessons from database failures by Colin Charles
NETWAYS
 
A beginners guide to MariaDB
A beginners guide to MariaDBA beginners guide to MariaDB
A beginners guide to MariaDB
Colin Charles
 
The MySQL ecosystem - understanding it, not running away from it!
The MySQL ecosystem - understanding it, not running away from it! The MySQL ecosystem - understanding it, not running away from it!
The MySQL ecosystem - understanding it, not running away from it!
Colin Charles
 
The MySQL Server Ecosystem in 2016
The MySQL Server Ecosystem in 2016The MySQL Server Ecosystem in 2016
The MySQL Server Ecosystem in 2016
Colin Charles
 
MariaDB: The 2012 Edition
MariaDB: The 2012 EditionMariaDB: The 2012 Edition
MariaDB: The 2012 Edition
Colin Charles
 
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdfMySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
Alkin Tezuysal
 

Similar to OSDC 2018 | Scaling & High Availability MySQL learnings from the past decade+ by Colin Charles (20)

Best practices for MySQL High Availability Tutorial
Best practices for MySQL High Availability TutorialBest practices for MySQL High Availability Tutorial
Best practices for MySQL High Availability Tutorial
 
Best practices for MySQL/MariaDB Server/Percona Server High Availability
Best practices for MySQL/MariaDB Server/Percona Server High AvailabilityBest practices for MySQL/MariaDB Server/Percona Server High Availability
Best practices for MySQL/MariaDB Server/Percona Server High Availability
 
Best practices for MySQL High Availability
Best practices for MySQL High AvailabilityBest practices for MySQL High Availability
Best practices for MySQL High Availability
 
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)
 
MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
MariaDB 10.1   what's new and what's coming in 10.2 - Tokyo MariaDB MeetupMariaDB 10.1   what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
MariaDB 10.1 what's new and what's coming in 10.2 - Tokyo MariaDB Meetup
 
[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale by ...
[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale  by ...[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale  by ...
[db tech showcase Tokyo 2014] B15: Scalability with MariaDB and MaxScale by ...
 
Meet MariaDB 10.1 at the Bulgaria Web Summit
Meet MariaDB 10.1 at the Bulgaria Web SummitMeet MariaDB 10.1 at the Bulgaria Web Summit
Meet MariaDB 10.1 at the Bulgaria Web Summit
 
MariaDB 10: A MySQL Replacement - HKOSC
MariaDB 10: A MySQL Replacement - HKOSC MariaDB 10: A MySQL Replacement - HKOSC
MariaDB 10: A MySQL Replacement - HKOSC
 
Why MariaDB?
Why MariaDB?Why MariaDB?
Why MariaDB?
 
Maria db 10 and the mariadb foundation(colin)
Maria db 10 and the mariadb foundation(colin)Maria db 10 and the mariadb foundation(colin)
Maria db 10 and the mariadb foundation(colin)
 
Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...
Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...
Choosing between Codership's MySQL Galera, MariaDB Galera Cluster and Percona...
 
Lessons from database failures
Lessons from database failuresLessons from database failures
Lessons from database failures
 
The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016
 
The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016The MySQL Server ecosystem in 2016
The MySQL Server ecosystem in 2016
 
OSDC 2017 | Lessons from database failures by Colin Charles
OSDC 2017 | Lessons from database failures by Colin CharlesOSDC 2017 | Lessons from database failures by Colin Charles
OSDC 2017 | Lessons from database failures by Colin Charles
 
A beginners guide to MariaDB
A beginners guide to MariaDBA beginners guide to MariaDB
A beginners guide to MariaDB
 
The MySQL ecosystem - understanding it, not running away from it!
The MySQL ecosystem - understanding it, not running away from it! The MySQL ecosystem - understanding it, not running away from it!
The MySQL ecosystem - understanding it, not running away from it!
 
The MySQL Server Ecosystem in 2016
The MySQL Server Ecosystem in 2016The MySQL Server Ecosystem in 2016
The MySQL Server Ecosystem in 2016
 
MariaDB: The 2012 Edition
MariaDB: The 2012 EditionMariaDB: The 2012 Edition
MariaDB: The 2012 Edition
 
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdfMySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
 

Recently uploaded

AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
karim wahed
 
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
avufu
 
NYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdfNYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdf
AUGNYC
 
dachnug51 - Whats new in domino 14 .pdf
dachnug51 - Whats new in domino 14  .pdfdachnug51 - Whats new in domino 14  .pdf
dachnug51 - Whats new in domino 14 .pdf
DNUG e.V.
 
Break data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud ConnectorsBreak data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud Connectors
confluent
 
ThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and DjangoThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and Django
akshesh doshi
 
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
Hironori Washizaki
 
Google ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learningGoogle ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learning
VishrutGoyani1
 
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdfdachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
DNUG e.V.
 
Safe Work Permit Management Software for Hot Work Permits
Safe Work Permit Management Software for Hot Work PermitsSafe Work Permit Management Software for Hot Work Permits
Safe Work Permit Management Software for Hot Work Permits
sheqnetworkmarketing
 
Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.
shivamt017
 
introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...
sachin chaurasia
 
Discover the Power of ONEMONITAR: The Ultimate Mobile Spy App for Android Dev...
Discover the Power of ONEMONITAR: The Ultimate Mobile Spy App for Android Dev...Discover the Power of ONEMONITAR: The Ultimate Mobile Spy App for Android Dev...
Discover the Power of ONEMONITAR: The Ultimate Mobile Spy App for Android Dev...
onemonitarsoftware
 
ANSYS Mechanical APDL Introductory Tutorials.pdf
ANSYS Mechanical APDL Introductory Tutorials.pdfANSYS Mechanical APDL Introductory Tutorials.pdf
ANSYS Mechanical APDL Introductory Tutorials.pdf
sachin chaurasia
 
Leading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptxLeading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptx
taskroupseo
 
How we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hoursHow we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hours
Ortus Solutions, Corp
 
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
ThousandEyes
 
dachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdfdachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdf
DNUG e.V.
 
WhatsApp Tracker - Tracking WhatsApp to Boost Online Safety.pdf
WhatsApp Tracker -  Tracking WhatsApp to Boost Online Safety.pdfWhatsApp Tracker -  Tracking WhatsApp to Boost Online Safety.pdf
WhatsApp Tracker - Tracking WhatsApp to Boost Online Safety.pdf
onemonitarsoftware
 
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTIONBITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
ssuser2b426d1
 

Recently uploaded (20)

AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
AWS Cloud Practitioner Essentials (Second Edition) (Arabic) Course Introducti...
 
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
一比一原版英国牛津大学毕业证(oxon毕业证书)如何办理
 
NYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdfNYC 26-Jun-2024 Combined Presentations.pdf
NYC 26-Jun-2024 Combined Presentations.pdf
 
dachnug51 - Whats new in domino 14 .pdf
dachnug51 - Whats new in domino 14  .pdfdachnug51 - Whats new in domino 14  .pdf
dachnug51 - Whats new in domino 14 .pdf
 
Break data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud ConnectorsBreak data silos with real-time connectivity using Confluent Cloud Connectors
Break data silos with real-time connectivity using Confluent Cloud Connectors
 
ThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and DjangoThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and Django
 
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...
 
Google ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learningGoogle ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learning
 
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdfdachnug51 - HCL Sametime 12 as a Software Appliance.pdf
dachnug51 - HCL Sametime 12 as a Software Appliance.pdf
 
Safe Work Permit Management Software for Hot Work Permits
Safe Work Permit Management Software for Hot Work PermitsSafe Work Permit Management Software for Hot Work Permits
Safe Work Permit Management Software for Hot Work Permits
 
Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.Shivam Pandit working on Php Web Developer.
Shivam Pandit working on Php Web Developer.
 
introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...introduction of Ansys software and basic and advance knowledge of modelling s...
introduction of Ansys software and basic and advance knowledge of modelling s...
 
Discover the Power of ONEMONITAR: The Ultimate Mobile Spy App for Android Dev...
Discover the Power of ONEMONITAR: The Ultimate Mobile Spy App for Android Dev...Discover the Power of ONEMONITAR: The Ultimate Mobile Spy App for Android Dev...
Discover the Power of ONEMONITAR: The Ultimate Mobile Spy App for Android Dev...
 
ANSYS Mechanical APDL Introductory Tutorials.pdf
ANSYS Mechanical APDL Introductory Tutorials.pdfANSYS Mechanical APDL Introductory Tutorials.pdf
ANSYS Mechanical APDL Introductory Tutorials.pdf
 
Leading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptxLeading Project Management Tool Taskruop.pptx
Leading Project Management Tool Taskruop.pptx
 
How we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hoursHow we built TryBoxLang in under 48 hours
How we built TryBoxLang in under 48 hours
 
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
Cisco Live Announcements: New ThousandEyes Release Highlights - July 2024
 
dachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdfdachnug51 - All you ever wanted to know about domino licensing.pdf
dachnug51 - All you ever wanted to know about domino licensing.pdf
 
WhatsApp Tracker - Tracking WhatsApp to Boost Online Safety.pdf
WhatsApp Tracker -  Tracking WhatsApp to Boost Online Safety.pdfWhatsApp Tracker -  Tracking WhatsApp to Boost Online Safety.pdf
WhatsApp Tracker - Tracking WhatsApp to Boost Online Safety.pdf
 
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTIONBITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
BITCOIN HEIST RANSOMEWARE ATTACK PREDICTION
 

OSDC 2018 | Scaling & High Availability MySQL learnings from the past decade+ by Colin Charles

  • 1. Scaling & High Availability MySQL learnings from the past decade+ Colin Charles, Chief Evangelist, Percona Inc. colin.charles@percona.com / byte@bytebot.net http://www.bytebot.net/blog/ | @bytebot on Twitter OSDC.de, Berlin, Germany 12 June 2018
  • 2. whoami • Chief Evangelist, Percona Inc • Focusing on the MySQL ecosystem (MySQL, Percona Server, MariaDB Server), as well as the MongoDB ecosystem (Percona Server for MongoDB) + 100% open source tools from Percona like Percona Monitoring & Management, Percona xtrabackup, Percona Toolkit, etc. • Founding team of MariaDB Server (2009-2016), previously at Monty Program Ab, merged with SkySQL Ab, now MariaDB Corporation • Formerly MySQL AB (exit: Sun Microsystems) • Past lives include The Fedora Project (FESCO), OpenOffice.org • MySQL Community Contributor of the Year Award winner 2014
  • 3. License • Creative Commons BY-NC-SA 4.0 • https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode 

  • 7. Uptime Percentile target Max downtime per year 90% 36 days 99% 3.65 days 99.5% 1.83 days 99.9% 8.76 hours 99.99% 52.56 minutes 99.999% 5.25 minutes 99.9999% 31.5 seconds
  • 8. Estimates of levels of availability Method Level of Availability Simple replication 98-99.9% Master-Master/MMM 99% SAN 99.5-99.9% DRBD, MHA 99.9% NDBCluster, Galera Cluster, InnoDB Cluster 99.999%
  • 9. HA is Redundancy • RAID: disk crashes? Another works • Clustering: server crashes? Another works • Power: fuse blows? Redundant power supplies • Network: Switch/NIC crashes? 2nd network route • Geographical: Datacenter offline/destroyed? Computation to another DC
  • 10. Durability • Data stored on disks • Is it really written to the disk? ▪ being durable means calling fsync() on each commit • Is it written in a transactional way to guarantee atomicity, crash safety, integrity?
  • 11. High Availability for databases • HA is harder for databases • Hardware resources and data need to be redundant • Remember, this isn’t just data - constantly changing data • HA means the operation can continue uninterrupted, not by restoring a new/backup server • uninterrupted: measured in percentiles
  • 12. Redundancy through client-side XA transactions • Client writes to 2 independent but identical databases • HA-JDBC (http://ha-jdbc.github.io/) • No replication anywhere
  • 13. InnoDB “recovery” time • innodb_log_file_size • larger = longer recovery times • In 2007, Wikipedia reported 40 minutes with 256MB logs; sometimes it takes 5-10 minutes, sometimes hours • Percona Server 5.5 (XtraDB) - innodb_recovery_stats • But today, there has been paradigm change…
  • 14. Redundancy through shared storage • Requires specialist hardware, like a SAN • Complex to operate • One set of data is your single point of failure • Cold standby • failover 1-30 minutes • this isn’t scale-out • Active/Active solutions: Oracle RAC
  • 15. Redundancy through disk replication • DRBD • Linux administration vs. DBA skills • Synchronous • Second set of data inaccessible for use • Passive server acting as hot standby • Failover: 1-30 minutes • Performance hit compared to single node performance, with higher average latencies
  • 16. Redundancy through MySQL replication • MySQL replication • Tungsten Replicator • Galera Cluster • MySQL InnoDB Cluster • MySQL Cluster (NDBCLUSTER) • Storage requirements are multiplied • Huge potential for scaling out
  • 17. MySQL Replication • Statement based generally • Row based became available in 5.1, and the default in 5.7 • mixed-mode, resulting in STATEMENT except if calling ▪ UUID function, UDF, CURRENT_USER/USER function, LOAD_FILE function ▪ 2 or more AUTO_INCREMENT columns updated with same statement ▪ server variable used in statement ▪ storage engine doesn’t allow statement based replication, like NDBCLUSTER ▪ default in MariaDB Server 10.2
  • 18. MySQL Replication II • Asynchronous by default • Semi-synchronous plugin in 5.5+ • However the holy grail of fully synchronous replication is not part of standard MySQL replication (yet?) • MariaDB Galera Cluster is built-in to MariaDB Server 10.1 • MySQL InnoDB Cluster is available in MySQL 8.0, combining the likes of group replication (the Galera Cluster equivalent), InnoDB, and mysqlsh for maintaining it; it also uses MySQL Router for load balancing purposes
  • 19. Semi-synchronous replication • semi-sync capable slave acknowledges transaction event only after written to relay log & flushed to disk • timeout occurs? master reverts to async replication; resumes when slaves catch up • at scale, Facebook runs semi-sync: http:// yoshinorimatsunobu.blogspot.com/2014/04/semi-synchronous- replication-at-facebook.html
  • 20. MySQL Replication in 5.6 • Global Transaction ID (GTID) • Server UUID • Ignore (master) server IDs (filtering) • Per-schema multi-threaded slave • Group commit in the binary log • Binary log (binlog) checksums • Crash safe binlog and relay logs • Time delayed replication • Parallel replication (per database)
  • 21. Replication: START TRANSACTION WITH CONSISTENT SNAPSHOT • Works with the binlog, possible to obtain the binlog position corresponding to a transactional snapshot of the database without blocking any other queries. • by-product of group commit in the binlog to view commit ordering • Used by the command mysqldump--single-transaction --master- data to do a fully non-blocking backup • Works consistently between transactions involving more than one storage engine • https://kb.askmonty.org/en/enhancements-for-start-transaction-with- consistent/ • Percona Server improved it, by session ID, and also introducing backup locks
  • 22. Multi-source replication • Multi-source replication - (real-time) analytics, shard provisioning, backups, etc. • @@default_master_connection contains current connection name (used if connection name is not given) • All master/slave commands take a connection name now (like CHANGE MASTER “connection_name”, SHOW SLAVE “connection_name” STATUS, etc.)
  • 23. Global Transaction ID (GTID) • Supports multi-source replication • GTID can be enabled or disabled independently and online for masters or slaves • Slaves using GTID do not have to have binary logging enabled. • (MariaDB) Supports multiple replication domains (independent binlog streams) • Queries in different domains can be run in parallel on the slave.
  • 24. Why MariaDB GTID is different compared to 5.6? • MySQL 5.6 GTID does not support multi-source replication (only 5.7 supports this) • Supports —log-slave-updates=0 for efficiency (like 5.7) • Enabled by default • Turn it on without having to restart the topology (just like 5.7)
  • 25. Parallel replication • Multi-source replication from different masters executed in parallel • Queries from different domains are executed in parallel • Queries that are run in parallel on the master are run in parallel on the slave (based on group commit). • Transactions modifying the same table can be updated in parallel on the slave! • Supports both statement based and row based replication.
  • 26. All in… sometimes it can get out of sync • Changed information on slave directly • Statement based replication • non-deterministic SQL (UPDATE/DELETE with LIMIT and without ORDER BY) • triggers & stored procedures • Master in MyISAM, slave in InnoDB (deadlocks) • --replication-ignore-db with fully qualified queries • Binlog corruption on master • PURGE BINARY LOGS issued and not enough files to update slave • read_buffer_size larger than max_allowed_packet • Bugs?
  • 27. Replication Monitoring • Percona Toolkit is important • pt-slave-find: find slave information from master • pt-table-checksum: online replication consistency check • executes checksum queries on master • pt-table-sync: synchronise table data efficiently • changes data, so backups important
  • 28. Replication Monitoring with PMM •http://pmmdemo.percona.com/
  • 29. mysqlbinlog versions • ERROR: Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 56, event_type: 30 • 5.6 ships with a “streaming binlog backup server” - v.3.4; MariaDB 10 doesn’t - v.3.3 (fixed in 10.2 - MDEV-8713) • GTID variances! • Beware mysql-client from your Linux distribution
  • 30. Slave prefetching • Replication Booster • https://github.com/yoshinorim/replication-booster-for-mysql • Prefetch MySQL relay logs to make the SQL thread faster • Tungsten has slave prefetch • Percona Server till 5.6 + MariaDB till 10.1 have InnoDB fake changes
  • 31. Changing paradigm: What replaces slave prefetching? • In Percona Server 5.7, slave prefetching has been replaced by doing intra-schema parallel replication • Feature removed from XtraDB • MariaDB Server 10.2 also has this feature removed, as they switched to InnoDB!
  • 32. Galera Cluster • Inside MySQL, a replication plugin (wsrep) • Replaces MySQL replication (but can work alongside it too) • True multi-master, active-active solution • Virtually Synchronous • WAN performance: 100-300ms/commit, works in parallel • No slave lag or integrity issues • Automatic node provisioning
  • 34. Percona XtraDB Cluster 5.7 • Engineering within Percona • Load balancing with ProxySQL (bundled) • PMM integration • Benefits of all the MySQL 5.7 feature-set • ProxySQL admin tool • Safety features enabled (e.g. no accidentally using MyISAM, etc.)
  • 35. Group replication • Fully synchronous replication (update everywhere), self-healing, with elasticity, redundancy • Single primary mode supported • MySQL InnoDB Cluster - a combination of group replication, Router, to make magic! • Recent blogs: • https://www.percona.com/blog/2017/02/24/battle-for-synchronous- replication-in-mysql-galera-vs-group-replication/ • https://www.percona.com/blog/2017/02/15/group-replication-shipped- early/
  • 36. Summary of Replication Performance • SAN has "some" latency overhead compared to local disk. Can be great for throughput. • DRBD has a performance penalty • Replication, when implemented correctly, has no performance penalty • But MySQL replication with disk bound data set has single-threaded issues! • Semi-sync is poorer on WAN compared to async • Galera & InnoDB Cluster provide read/write scale-out, thus more performance
  • 37. Handling failure • How do we find out about failure? • Polling, monitoring, alerts... • Error returned to and handled in client side • What should we do about it? • Direct requests to the spare nodes (or DCs) • How to protect data integrity? • Master-slave is unidirectional: Must ensure there is only one master at all times. • DRBD and SAN have cold-standby: Must mount disks and start mysqld. • In all cases must ensure that 2 disconnected replicas cannot both commit independently. (split brain)
  • 38. Frameworks to handle failure • MySQL-MMM • Severalnines ClusterControl • Orchestrator • MySQL MHA • Percona Replication Manager • Tungsten Replicator • 5.6: mysqlfailover, mysqlrpladmin • (MariaDB) Replication Manager
  • 39. Orchestrator • Reads replication topologies, keeps state, continuous polling • Modify your topology — move slaves around • Nice GUI, JSON API, CLI
  • 40. MySQL MHA • Like MMM, specialized solution for MySQL replication • Developed by Yoshinori Matsunobu at DeNA • Automated and manual failover options • Topology: 1 master, many slaves • Choose new master by comparing slave binlog positions • Can be used in conjunction with other solutions • http://code.google.com/p/mysql-master-ha/
  • 41. Pacemaker • Heartbeat, Corosync, Pacemaker • Resource Agents, Percona-PRM • Percona Replication Manager - cluster, geographical disaster recovery options • Pacemaker agent specialised on MySQL replication • https://github.com/percona/percona-pacemaker-agents/ • Pacemaker Resource Agents 3.9.3+ include Percona Replication Manager (PRM)
  • 42. Load Balancers for multi-master clusters • Synchronous multi-master clusters like Galera require load balancers • HAProxy • Galera Load Balancer (GLB) • MaxScale • ProxySQL
  • 43. MySQL Router • Routing between applications and any backend MySQL servers • Failover • Load Balancing • Pluggable architecture (connection routing)
  • 44. MariaDB MaxScale • “Pluggable router” that offers connection & statement based load balancing • Possibilities are endless - use it for logging, writing to other databases (besides MySQL), preventing SQL injections via regex filtering, route via hints, query rewriting, have a binlog relay, etc.
  • 45. ProxySQL • High Performance MySQL proxy with a GPL license • Performance is a priority - the numbers prove it • Can query rewrite • Sharding by host/schema or both, with rule engine + modification to SQL + application logic
  • 46. JDBC/PHP drivers • JDBC - multi-host failover feature (just specify master/slave hosts in the properties) • true for MariaDB Java Connector too • PHP handles this too - mysqlnd_ms • Can handle read-write splitting, round robin or random host selection, and more
  • 47. Clustering: solution or part of problem? • "Causes of Downtime in Production MySQL Servers" whitepaper, Baron Schwartz VividCortex • Human error • SAN • Clustering framework + SAN = more problems • Galera/group replication is replication based, has no false positives as there’s no “failover” moment, you don’t need a clustering framework (JDBC or PHP can load balance), and is relatively elegant overall
  • 48. Replication type • Competence choices • Replication: MySQL DBA manages • DRBD: Linux admin manages • SAN: requires domain controller • Operations • DRBD (disk level) = cold standby = longer failover • Replication = hot standby = shorter failover • GTID helps tremendously • Performance • SAN has higher latency than local disk • DRBD has higher latency than local disk • Replication has little overhead • Redundancy • Shared disk = SPoF • Shared nothing = redundant
  • 49. SBR vs RBR? Async vs sync? • row based: deterministic • statement based: dangerous • GTID: easier setup & failover of complex topologies • async: data loss in failover • sync: best • multi-threaded slaves: scalability (hello 5.6+, Tungsten)
  • 50. What about the cloud? • Usually scalability & high availability is more or less “built-in” • e.g. RDS has multi-AZ (synchronous data replication), but doesn’t give you a read replica; Cloud SQL uses semi-sync replication • Watch out for the SLAs (and automatic upgrades) • Monitoring via PERFORMANCE_SCHEMA • “Bad” nodes do exist; do not assume node provisioning is quick
  • 51. Conclusion • MySQL replication is amazing if you know it (and monitor it) well enough • Large sites run just fine with semi-sync + tooling for automated failover • Galera Cluster/MySQL InnoDB Cluster is great for virtually fully synchronous replication • Don’t forget the need for a load balancer: ProxySQL is nifty • When thinking scaling, think scale out (it is more efficient, and fits modern cloud mantras too!)
  • 52. At Percona, we care about your High Availability • Percona XtraDB Cluster 5.7 with support for ProxySQL and Percona Monitoring & Management (PMM) • Percona Monitoring & Management (PMM) with Orchestrator • Percona Toolkit • Percona Server for MySQL 5.7 • Percona XtraBackup
  • 53. Thank you! Colin Charles colin.charles@percona.com / byte@bytebot.net http://bytebot.net/blog | @bytebot on twitter slides: slideshare.net/bytebot