Presented at OSCON 2018. A review of what is available from MySQL, MariaDB Server, MongoDB, PostgreSQL, and more. Covering your choices, considerations, versions, access methods, cost, a deeper look at RDS and if you should run your own instances or not.
This document summarizes a talk given by Michael "Monty" Widenius about reasons to switch to MariaDB 10.0 from MySQL 5.5 or MariaDB 5.5. The talk addresses why MariaDB was created, features of MariaDB releases, benchmarks, the role of the MariaDB foundation, and reasons to switch. It provides information on the MariaDB foundation goals of developing and distributing MariaDB openly. It outlines many new features in MariaDB 10.0 including new storage engines, replication features, functionality, and improvements in areas like speed, optimization, and usability.
* If you see the screen is not good condition, downloading please. *
Introduction to MariaDB
- mariadb oracle mysql comparison
- mariadb install step by step
- mariadb basic query
MySQL is a unique adult (now 21 years old) in many ways. It supports plugins. It supports storage engines. It is also owned by Oracle, thus birthing two branches of the popular opensource database: Percona Server and MariaDB Server. It also once spawned a fork: Drizzle. Lately a consortium of web scale users (think a chunk of the top 10 sites out there) have spawned WebScaleSQL.
You're a busy DBA having to maintain a mix of this. Or you're a CIO planning to choose one branch. How do you go about picking? Supporting multiple databases? Find out more in this talk. Also covered is a deep-dive into what feature differences exist between MySQL/Percona Server/MariaDB/WebScaleSQL, how distributions package the various databases differently. Within the hour, you'll be informed about the past, the present, and hopefully be knowledgeable enough to know what to pick in the future.
Note, there will also be coverage of the various trees around WebScaleSQL, like the Facebook tree, the Alibaba tree as well as the Twitter tree.
This is my third iteration of the talk presented in Tokyo, Japan - first was at a keynote at rootconf.in in April 2016, then at the MySQL meetup in New York, and now for dbtechshowcase. The focus is on database failures of the past, and how modern MySQL / MariaDB Server technologies could have helped them avoid such failure. The focus is on backups and verification, replication and failover, and security and encryption.
MariaDB 10: A MySQL Replacement. Current up to 10.0.9, right before the 10.0.10 GA release presented the weekend before the release in Hong Kong, at the Hong Kong Open Source Conference.
MariaDB is a community-developed fork of MySQL that aims to be a drop-in replacement. It focuses on being compatible, stable with no regressions, and feature-enhanced compared to MySQL. The presentation covered MariaDB's architecture including connections, query caching, storage engines, and tools for administration and development like mysql, mysqldump, and EXPLAIN.
Co-presented alongside Ronald Bradford, this covers MySQL, Percona Server, and MariaDB Server (since the latter occasionally can be different enough). Go thru insecure practices, focus on communication security, connection security, data security, user accounts and server access security.
MariaDB started life as a database to host the Maria storage engine in 2009. Not long after its inception, the MySQL community went through yet another change in ownership, and it was deemed that MariaDB will be a complete database branch developed to extend MySQL, but with constant merging of upstream changes.
The goal of the MariaDB project is to ensure that everyone is part of the community, including employees of the major steering companies. MariaDB also features enhanced features, some of which are common with the Percona Performance Server. Most importantly, MariaDB is a drop-in replacement and is completely backward compatible with MySQL. In 2010, MariaDB released 5.1 in February, and 5.2 in November – two major releases in a span of one calendar year is a feat that was achieved!
DBAs and developers alike will gain an introduction to MariaDB, what is different with MySQL, how to make use of the feature enhancements, and more.
MariaDB is a community developed branch of MySQL that is feature enhanced and backward compatible. It aims to be a 100% drop-in replacement for MySQL that is stable, bug-free, and released under the GPLv2 license. Major releases of MariaDB include new storage engines like XtraDB and Aria, as well as new features for performance, scalability, and compatibility. MariaDB is developed as an open source project and supported by Monty Program and other community contributors and service providers.
This document summarizes a presentation on MariaDB/MySQL security essentials. The presentation covered historically insecure default configurations, privilege escalation vulnerabilities, access control best practices like limiting privileges to only what users need and removing unnecessary accounts. It also discussed authentication methods like SSL, PAM, Kerberos and audit plugins. Encryption at the table, tablespace and binary log level was explained as well. Preventing SQL injections and available security assessment tools were also mentioned.
Meet MariaDB 10.1 at the Bulgaria Web Summit, held in Sofia in February 2016. Learn all about MariaDB Server, and the new features like encryption, audit plugins, and more.
- MariaDB Corporation was founded by original developers of MySQL and provides commercial support for MariaDB and MySQL. It has over 400 enterprise customers globally.
- MariaDB is an enhanced, drop-in replacement for MySQL that is open source and offers additional features like improved performance, security, and scalability. It has been adopted by several Linux distributions as the default database.
- MariaDB offers several advantages over MySQL for applications like Drupal, including its XtraDB storage engine, SphinxSE search engine, thread pool feature for handling many concurrent queries efficiently, and Galera Cluster for high availability.
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)
MariaDB is like the "new" MySQL, and its available everywhere. This talk was given at LinuxCon Europe in Dublin in October 2015. Learn about all the new features, considering the release was just around the corner. Changes in replication are also very interesting
At the MariaDB Server Developer's meeting in Amsterdam, Oct 8 2016. This was the deck to talk about what MariaDB Server 10.1/10.2 might be missing from MySQL versions up to 5.7. The focus is on compatibility of MariaDB Server with MySQL.
The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScale
This document discusses MySQL proxy technologies including MySQL Router, ProxySQL, and MariaDB MaxScale. It provides an overview of each technology, including when they were released, key features, and comparisons between them. ProxySQL is highlighted as a popular option currently with integration with Percona tools, while MySQL Router may become more widely used due to its support for MySQL InnoDB Cluster. MariaDB MaxScale is noted for its binlog routing capabilities. Overall the document aims to help people understand and choose between the different MySQL proxy options.
You want to use MySQL in Amazon RDS, Rackspace Cloud, Google Cloud SQL or HP Helion Public Cloud? Check this out, from Percona Live London 2014. (Note that pricing of Google Cloud SQL changed prices on the same day after the presentation)
With a focus on Amazon AWS RDS MySQL and PostgreSQL, Rackspace cloud, Google Cloud SQL, Microsoft Azure for MySQL and PostgreSQL as well as a hint of the other clouds
Today you can use MySQL in several clouds in what is considered using it as a service, a database as a service (DBaaS). Learn the differences, the access methods, and the level of control you have for the various cloud offerings including:
- Amazon RDS
- Google Cloud SQL
- HPCloud DBaaS
- Rackspace Openstack DBaaS
The administration tools and ideologies behind it are completely different, and you are in a "locked-down" environment. Some considerations include:
* Different backup strategies
* Planning for multiple data centres for availability
* Where do you host your application?
* How do you get the most performance out of the solution?
* What does this all cost?
Questions like this will be demystified in the talk.
Colin Charles presented on running MySQL in the hosted cloud. He discussed various database as a service (DBaaS) options like Amazon RDS, Rackspace, and Google Cloud SQL. Key considerations for DBaaS include location, service level agreements, support options, available MySQL/MariaDB versions, access methods, configuration options, costs, and features like high availability and backups. Running MySQL on EC2 is also an option but requires more management of hardware, software, networking, storage and backups. Benchmarking and monitoring tools were recommended to evaluate performance and usage.
Today you can use hosted MySQL/MariaDB/Percona Server in several "cloud providers" in what is considered using it as a service, a database as a service (DBaaS). You can also use hosted PostgreSQL and MongoDB thru various service providers. Learn the differences, the access methods, and the level of control you have for the various public cloud offerings:
- Amazon RDS for MySQL and PostgreSQL
- Google Cloud SQL
- Rackspace OpenStack DBaaS
- The likes of compose.io, MongoLab and Rackspace's offerings around MongoDB
The administration tools and ideologies behind it are completely different, and you are in a "locked-down" environment. Some considerations include:
* Different backup strategies
* Planning for multiple data centres for availability
* Where do you host your application?
* How do you get the most performance out of the solution?
* What does this all cost?
Growth topics include:
* How do you move from one DBaaS to another?
* How do you move all this from DBaaS to your own hosted platform?
Questions like this will be demystified in the talk. This talk will benefit experienced database administrators (DBAs) who now also have to deal with cloud deployments as well as application developers in startups that have to rely on "managed services" without the ability of a DBA.
NOSQL Meets Relational - The MySQL Ecosystem Gains More Flexibility
Colin Charles gave a presentation comparing SQL and NoSQL databases. He discussed why organizations adopt NoSQL databases like MongoDB for large, unstructured datasets and rapid development. However, he argued that MySQL can also handle these workloads through features like dynamic columns, memcached integration, and JSON support. MySQL addresses limitations around high availability, scalability, and schema flexibility through tools and plugins that provide sharding, replication, load balancing, and online schema changes. In the end, MySQL with the right tools is capable of fulfilling both transactional and NoSQL-style workloads.
This document provides an overview of MariaDB 10 and the MariaDB Foundation. It discusses the history and development of MariaDB, including key features added in versions 5.1 through 10.0 such as new storage engines, performance improvements, and features backported from MySQL. It outlines the goals of MariaDB to be compatible with MySQL while adding new features, and describes the community-led development model. The roadmap aims to have MariaDB be a drop-in replacement for MySQL 5.6 by releasing version 10.1.
* Use cases of MySQL as well as edge cases of MySQL topologies using real-life examples and "war" stories
* How scalability and proxy wars make MySQL topologies more robust to serve webscale shops
* Open-source tools, utilities, and surrounding MySQL Ecosystem.
The document summarizes the history and current state of the MySQL database server ecosystem. It discusses the origins and development of MySQL, MariaDB, Percona Server, and other related projects. It also describes some of the key features and innovations in recent versions of these database servers. The ecosystem is very active with contributions from many organizations and the future remains promising with ongoing work.
This document provides an overview of migrating applications and workloads to AWS. It discusses key considerations for different migration approaches including "forklift", "embrace", and "optimize". It also covers important AWS services and best practices for architecture design, high availability, disaster recovery, security, storage, databases, auto-scaling, and cost optimization. Real-world customer examples of migration lessons and benefits are also presented.
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
MySQL is still hot, with Percona XtraDB Cluster (PXC) and MariaDB Server. Welcome back post-pandemic to see what is on offer in the current ecosystem.
Did you know that Amazon RDS now uses semi-sync replication rather than DRBD for multi-AZ deployments? Did you know that Galera Cluster for MySQL 8 is much more efficient with CLONE SST rather than using the xtrabackup method for SST? Did you know that Percona Server continues to extend MyRocks? Did you know that MariaDB Server has more Oracle syntax compatibility? This and more will be covered in the session, while short and quick, should leave you wandering to discover new features for production.
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
This document provides an overview and agenda for a workshop on patterns for continuous delivery, high availability, DevOps and cloud native development using NetflixOSS open source tools and frameworks. The presenter introduces himself and his background. The content covers Netflix's architecture evolution from monolithic to microservices, how Netflix scales on AWS, and principles and outcomes that enable cloud native development. The workshop then dives into specific NetflixOSS projects like Eureka, Cassandra, Zuul and Hystrix that help with service discovery, data storage, routing and availability. Tools for deployment, configuration, cost analysis and developer productivity are also discussed.
Databases in the Cloud discusses AWS database services for moving workloads to the cloud. It describes Amazon Relational Database Service (RDS) which provides several fully managed relational database options including MySQL, PostgreSQL, MariaDB, Oracle, SQL Server, and Amazon Aurora. It also discusses non-relational database services like DynamoDB, ElastiCache, and Redshift for analytics workloads. The document provides guidance on choosing between SQL and NoSQL databases and discusses benefits of managed database services over hosting databases on-premises or in EC2 instances.
MySQL is commonly used as the default database in OpenStack. It provides high availability through options like Galera and MySQL Group Replication. Galera is a third party active/active cluster that provides synchronous replication, while Group Replication is a native MySQL plugin that also enables active/active clusters with built-in conflict detection. MySQL NDB Cluster is an alternative that provides in-memory data storage with automatic sharding and strong consistency across shards. Both Galera/Group Replication and NDB Cluster can be used to implement highly available MySQL services in OpenStack environments.
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #1: AWS Aurora
AWS Aurora vs. Continuent Tungsten Clusters
Building a Geo-Scale, Multi-Region and Highly Available MySQL Cloud Back-End
This first installment of our High Noon series of on-demand webinars is focused on AWS Aurora. It looks at some of the key characteristics of AWS Aurora and how it fares as a MySQL HA / DR / Geo-Scale solution, especially when compared to Continuent Tungsten Clustering.
Watch this webinar to learn how to do better MySQL HA / DR / Geo-Scale.
AGENDA
- Goals for the High Noon Webinar Series
- AWS Aurora
- Key Characteristics
- Cross Region Requirements
- RDS Proxy
- Limitations Using AWS Aurora
- How to do better MySQL HA / DR / Geo-Scale?
- AWS Aurora vs Tungsten Clustering
- About Continuent & Its Solutions
PRESENTER
Matthew Lang - Customer Success Director – Americas, Continuent - has over 25 years of experience in database administration, database programming, and system architecture, including the creation of a database replication product that is still in use today. He has designed highly available, scaleable systems that have allowed startups to quickly become enterprise organizations, utilizing a variety of technologies including open source projects, virtualization and cloud.
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQL
MS Azure Database for MySQL vs. Continuent Tungsten Clusters
Building a Geo-Scale, Multi-Region and Highly Available MySQL Cloud Back-End
This is the third of our High Noon series covering MySQL clustering solutions for high availability (HA), disaster recovery (DR), and geographic distribution.
Azure Database for MySQL is a managed database cluster within Microsoft Azure Cloud that runs MySQL community edition. There are really two deployment options: “Single Server” and “Flexible Server (Preview).” We will look at the Flexible Server version, even though it is still preview, because most enterprise applications require failover, so this is the relevant comparison for Tungsten Clustering.
You may use Tungsten Clustering with native MySQL, MariaDB or Percona Server for MySQL in GCP, AWS, Azure, and/or on-premises data centers for better technological capabilities, control, and flexibility. But learn about the pros and cons!
Enjoy the webinar!
AGENDA
- Goals for the High Noon Webinar Series
- High Noon Series: Tungsten Clustering vs Others
- Microsoft Azure Database for MySQL
- Key Characteristics
- Certification-based Replication
- Azure MySQL Multi-Site Requirements
- Limitations Using Azure MySQL
- How to do better MySQL HA / DR / Geo-Scale?
- Azure MySQL vs Tungsten Clustering
- About Continuent & Its Solutions
PRESENTER
Matthew Lang - Customer Success Director – Americas, Continuent - has over 25 years of experience in database administration, database programming, and system architecture, including the creation of a database replication product that is still in use today. He has designed highly available, scaleable systems that have allowed startups to quickly become enterprise organizations, utilizing a variety of technologies including open source projects, virtualization and cloud.
This was a short 25 minute talk, but we go into a bit of a history of MySQL, how the branches and forks appeared, what's sticking around today (branch? Percona Server. Fork? MariaDB Server). What should you use? Think about what you need today and what the roadmap holds.
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #5: Oracle’s InnoDB Cluster
Oracle’s InnoDB Cluster vs. Continuent Tungsten Clusters for MySQL
Building a Geo-Distributed, Multi-Region and Highly Available MySQL Cloud Back-End
This is the fifth of our High Noon series covering MySQL clustering solutions for high availability (HA), disaster recovery (DR), and geographic distribution.
InnoDB Cluster uses MySQL’s group replication to handle the replication. It’s also known as semi-synchronous replication. Learn about this and more in this webinar!
You may use Tungsten Clustering with native MySQL, MariaDB or Percona Server for MySQL in GCP, AWS, Azure, and/or on-premises data centers for better technological capabilities, control, and flexibility. But learn about the pros and cons!
AGENDA
- Goals for the High Noon Webinar Series
- High Noon Series: Tungsten Clustering vs Others
- Oracle InnoDB Cluster
- Key Characteristics
- Certification-based Replication
- InnoDB Cluster Multi-Site Requirements
- Limitations Using InnoDB Cluster
- How to do better MySQL HA / DR / Geo-Distribution?
- InnoDB Cluster vs Tungsten Clustering
- About Continuent & Its Solutions
PRESENTER
Matthew Lang - Customer Success Director – Americas, Continuent - has over 25 years of experience in database administration, database programming, and system architecture, including the creation of a database replication product that is still in use today. He has designed highly available, scaleable systems that have allowed startups to quickly become enterprise organizations, utilizing a variety of technologies including open source projects, virtualization and cloud.
Cloudera Impala - Las Vegas Big Data Meetup Nov 5th 2014
Maxime Dumas gives a presentation on Cloudera Impala, which provides fast SQL query capability for Apache Hadoop. Impala allows for interactive queries on Hadoop data in seconds rather than minutes by using a native MPP query engine instead of MapReduce. It offers benefits like SQL support, improved performance of 3-4x up to 90x faster than MapReduce, and flexibility to query existing Hadoop data without needing to migrate or duplicate it. The latest release of Impala 2.0 includes new features like window functions, subqueries, and spilling joins and aggregations to disk when memory is exhausted.
Engineering that goes into making Percona Server for MySQL 5.6 & 5.7 different (and a hint of MongoDB) for dbtechshowcase 2017 - the slides also have some Japanese in it. This should help a Japanese audience to read it. If there are questions due to poor translation, do not hesitate to drop me an email (byte@bytebot.net) or tweet: @bytebot
MariaDB 10 and what's new with the projectColin Charles
This document provides an overview of MariaDB 10.0 and what's new compared to previous versions. Some of the key highlights include backporting features from MySQL 5.6 such as InnoDB, Performance Schema, and online ALTER TABLE. MariaDB 10.0 also includes new features like multi-source replication, persistent statistics, and integration with NoSQL databases. The goals are to have feature parity with MySQL 5.6 and provide an open source alternative to Oracle's MySQL with more active development.
MariaDB 10 Tutorial - 13.11.11 - Percona Live LondonIvan Zoratti
This document provides an overview and summary of MariaDB 10 features presented by Ivan Zoratti. It discusses new features in MariaDB 10 like storage engines, administration improvements, and replication capabilities. The document also summarizes optimization enhancements in MariaDB 10 like the new optimizer, improved indexing techniques, and subquery optimizations. Various agenda topics are outlined for the MariaDB 10 tutorial.
MariaDB - a MySQL Replacement #SELF2014Colin Charles
MariaDB - a MySQL replacement at South East Linux Fest 2014 - SELF2014. Learn about features that are not in MySQL 5.6, some that are only just coming in MySQL 5.7, and some that just don't exist.
This document summarizes a talk given by Michael "Monty" Widenius about reasons to switch to MariaDB 10.0 from MySQL 5.5 or MariaDB 5.5. The talk addresses why MariaDB was created, features of MariaDB releases, benchmarks, the role of the MariaDB foundation, and reasons to switch. It provides information on the MariaDB foundation goals of developing and distributing MariaDB openly. It outlines many new features in MariaDB 10.0 including new storage engines, replication features, functionality, and improvements in areas like speed, optimization, and usability.
* If you see the screen is not good condition, downloading please. *
Introduction to MariaDB
- mariadb oracle mysql comparison
- mariadb install step by step
- mariadb basic query
MySQL is a unique adult (now 21 years old) in many ways. It supports plugins. It supports storage engines. It is also owned by Oracle, thus birthing two branches of the popular opensource database: Percona Server and MariaDB Server. It also once spawned a fork: Drizzle. Lately a consortium of web scale users (think a chunk of the top 10 sites out there) have spawned WebScaleSQL.
You're a busy DBA having to maintain a mix of this. Or you're a CIO planning to choose one branch. How do you go about picking? Supporting multiple databases? Find out more in this talk. Also covered is a deep-dive into what feature differences exist between MySQL/Percona Server/MariaDB/WebScaleSQL, how distributions package the various databases differently. Within the hour, you'll be informed about the past, the present, and hopefully be knowledgeable enough to know what to pick in the future.
Note, there will also be coverage of the various trees around WebScaleSQL, like the Facebook tree, the Alibaba tree as well as the Twitter tree.
This is my third iteration of the talk presented in Tokyo, Japan - first was at a keynote at rootconf.in in April 2016, then at the MySQL meetup in New York, and now for dbtechshowcase. The focus is on database failures of the past, and how modern MySQL / MariaDB Server technologies could have helped them avoid such failure. The focus is on backups and verification, replication and failover, and security and encryption.
MariaDB 10: A MySQL Replacement - HKOSC Colin Charles
MariaDB 10: A MySQL Replacement. Current up to 10.0.9, right before the 10.0.10 GA release presented the weekend before the release in Hong Kong, at the Hong Kong Open Source Conference.
MariaDB: in-depth (hands on training in Seoul)Colin Charles
MariaDB is a community-developed fork of MySQL that aims to be a drop-in replacement. It focuses on being compatible, stable with no regressions, and feature-enhanced compared to MySQL. The presentation covered MariaDB's architecture including connections, query caching, storage engines, and tools for administration and development like mysql, mysqldump, and EXPLAIN.
Securing your MySQL / MariaDB Server dataColin Charles
Co-presented alongside Ronald Bradford, this covers MySQL, Percona Server, and MariaDB Server (since the latter occasionally can be different enough). Go thru insecure practices, focus on communication security, connection security, data security, user accounts and server access security.
MariaDB started life as a database to host the Maria storage engine in 2009. Not long after its inception, the MySQL community went through yet another change in ownership, and it was deemed that MariaDB will be a complete database branch developed to extend MySQL, but with constant merging of upstream changes.
The goal of the MariaDB project is to ensure that everyone is part of the community, including employees of the major steering companies. MariaDB also features enhanced features, some of which are common with the Percona Performance Server. Most importantly, MariaDB is a drop-in replacement and is completely backward compatible with MySQL. In 2010, MariaDB released 5.1 in February, and 5.2 in November – two major releases in a span of one calendar year is a feat that was achieved!
DBAs and developers alike will gain an introduction to MariaDB, what is different with MySQL, how to make use of the feature enhancements, and more.
MariaDB is a community developed branch of MySQL that is feature enhanced and backward compatible. It aims to be a 100% drop-in replacement for MySQL that is stable, bug-free, and released under the GPLv2 license. Major releases of MariaDB include new storage engines like XtraDB and Aria, as well as new features for performance, scalability, and compatibility. MariaDB is developed as an open source project and supported by Monty Program and other community contributors and service providers.
MariaDB Server & MySQL Security Essentials 2016Colin Charles
This document summarizes a presentation on MariaDB/MySQL security essentials. The presentation covered historically insecure default configurations, privilege escalation vulnerabilities, access control best practices like limiting privileges to only what users need and removing unnecessary accounts. It also discussed authentication methods like SSL, PAM, Kerberos and audit plugins. Encryption at the table, tablespace and binary log level was explained as well. Preventing SQL injections and available security assessment tools were also mentioned.
Meet MariaDB 10.1 at the Bulgaria Web SummitColin Charles
Meet MariaDB 10.1 at the Bulgaria Web Summit, held in Sofia in February 2016. Learn all about MariaDB Server, and the new features like encryption, audit plugins, and more.
- MariaDB Corporation was founded by original developers of MySQL and provides commercial support for MariaDB and MySQL. It has over 400 enterprise customers globally.
- MariaDB is an enhanced, drop-in replacement for MySQL that is open source and offers additional features like improved performance, security, and scalability. It has been adopted by several Linux distributions as the default database.
- MariaDB offers several advantages over MySQL for applications like Drupal, including its XtraDB storage engine, SphinxSE search engine, thread pool feature for handling many concurrent queries efficiently, and Galera Cluster for high availability.
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)Colin Charles
MariaDB is like the "new" MySQL, and its available everywhere. This talk was given at LinuxCon Europe in Dublin in October 2015. Learn about all the new features, considering the release was just around the corner. Changes in replication are also very interesting
MariaDB Server Compatibility with MySQLColin Charles
At the MariaDB Server Developer's meeting in Amsterdam, Oct 8 2016. This was the deck to talk about what MariaDB Server 10.1/10.2 might be missing from MySQL versions up to 5.7. The focus is on compatibility of MariaDB Server with MySQL.
The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScaleColin Charles
This document discusses MySQL proxy technologies including MySQL Router, ProxySQL, and MariaDB MaxScale. It provides an overview of each technology, including when they were released, key features, and comparisons between them. ProxySQL is highlighted as a popular option currently with integration with Percona tools, while MySQL Router may become more widely used due to its support for MySQL InnoDB Cluster. MariaDB MaxScale is noted for its binlog routing capabilities. Overall the document aims to help people understand and choose between the different MySQL proxy options.
You want to use MySQL in Amazon RDS, Rackspace Cloud, Google Cloud SQL or HP Helion Public Cloud? Check this out, from Percona Live London 2014. (Note that pricing of Google Cloud SQL changed prices on the same day after the presentation)
With a focus on Amazon AWS RDS MySQL and PostgreSQL, Rackspace cloud, Google Cloud SQL, Microsoft Azure for MySQL and PostgreSQL as well as a hint of the other clouds
Today you can use MySQL in several clouds in what is considered using it as a service, a database as a service (DBaaS). Learn the differences, the access methods, and the level of control you have for the various cloud offerings including:
- Amazon RDS
- Google Cloud SQL
- HPCloud DBaaS
- Rackspace Openstack DBaaS
The administration tools and ideologies behind it are completely different, and you are in a "locked-down" environment. Some considerations include:
* Different backup strategies
* Planning for multiple data centres for availability
* Where do you host your application?
* How do you get the most performance out of the solution?
* What does this all cost?
Questions like this will be demystified in the talk.
MySQL in the Hosted Cloud - Percona Live 2015Colin Charles
Colin Charles presented on running MySQL in the hosted cloud. He discussed various database as a service (DBaaS) options like Amazon RDS, Rackspace, and Google Cloud SQL. Key considerations for DBaaS include location, service level agreements, support options, available MySQL/MariaDB versions, access methods, configuration options, costs, and features like high availability and backups. Running MySQL on EC2 is also an option but requires more management of hardware, software, networking, storage and backups. Benchmarking and monitoring tools were recommended to evaluate performance and usage.
Today you can use hosted MySQL/MariaDB/Percona Server in several "cloud providers" in what is considered using it as a service, a database as a service (DBaaS). You can also use hosted PostgreSQL and MongoDB thru various service providers. Learn the differences, the access methods, and the level of control you have for the various public cloud offerings:
- Amazon RDS for MySQL and PostgreSQL
- Google Cloud SQL
- Rackspace OpenStack DBaaS
- The likes of compose.io, MongoLab and Rackspace's offerings around MongoDB
The administration tools and ideologies behind it are completely different, and you are in a "locked-down" environment. Some considerations include:
* Different backup strategies
* Planning for multiple data centres for availability
* Where do you host your application?
* How do you get the most performance out of the solution?
* What does this all cost?
Growth topics include:
* How do you move from one DBaaS to another?
* How do you move all this from DBaaS to your own hosted platform?
Questions like this will be demystified in the talk. This talk will benefit experienced database administrators (DBAs) who now also have to deal with cloud deployments as well as application developers in startups that have to rely on "managed services" without the ability of a DBA.
NOSQL Meets Relational - The MySQL Ecosystem Gains More FlexibilityIvan Zoratti
Colin Charles gave a presentation comparing SQL and NoSQL databases. He discussed why organizations adopt NoSQL databases like MongoDB for large, unstructured datasets and rapid development. However, he argued that MySQL can also handle these workloads through features like dynamic columns, memcached integration, and JSON support. MySQL addresses limitations around high availability, scalability, and schema flexibility through tools and plugins that provide sharding, replication, load balancing, and online schema changes. In the end, MySQL with the right tools is capable of fulfilling both transactional and NoSQL-style workloads.
Maria db 10 and the mariadb foundation(colin)kayokogoto
This document provides an overview of MariaDB 10 and the MariaDB Foundation. It discusses the history and development of MariaDB, including key features added in versions 5.1 through 10.0 such as new storage engines, performance improvements, and features backported from MySQL. It outlines the goals of MariaDB to be compatible with MySQL while adding new features, and describes the community-led development model. The roadmap aims to have MariaDB be a drop-in replacement for MySQL 5.6 by releasing version 10.1.
* Use cases of MySQL as well as edge cases of MySQL topologies using real-life examples and "war" stories
* How scalability and proxy wars make MySQL topologies more robust to serve webscale shops
* Open-source tools, utilities, and surrounding MySQL Ecosystem.
The document summarizes the history and current state of the MySQL database server ecosystem. It discusses the origins and development of MySQL, MariaDB, Percona Server, and other related projects. It also describes some of the key features and innovations in recent versions of these database servers. The ecosystem is very active with contributions from many organizations and the future remains promising with ongoing work.
This document provides an overview of migrating applications and workloads to AWS. It discusses key considerations for different migration approaches including "forklift", "embrace", and "optimize". It also covers important AWS services and best practices for architecture design, high availability, disaster recovery, security, storage, databases, auto-scaling, and cost optimization. Real-world customer examples of migration lessons and benefits are also presented.
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdfAlkin Tezuysal
MySQL is still hot, with Percona XtraDB Cluster (PXC) and MariaDB Server. Welcome back post-pandemic to see what is on offer in the current ecosystem.
Did you know that Amazon RDS now uses semi-sync replication rather than DRBD for multi-AZ deployments? Did you know that Galera Cluster for MySQL 8 is much more efficient with CLONE SST rather than using the xtrabackup method for SST? Did you know that Percona Server continues to extend MyRocks? Did you know that MariaDB Server has more Oracle syntax compatibility? This and more will be covered in the session, while short and quick, should leave you wandering to discover new features for production.
Yow Conference Dec 2013 Netflix Workshop Slides with NotesAdrian Cockcroft
This document provides an overview and agenda for a workshop on patterns for continuous delivery, high availability, DevOps and cloud native development using NetflixOSS open source tools and frameworks. The presenter introduces himself and his background. The content covers Netflix's architecture evolution from monolithic to microservices, how Netflix scales on AWS, and principles and outcomes that enable cloud native development. The workshop then dives into specific NetflixOSS projects like Eureka, Cassandra, Zuul and Hystrix that help with service discovery, data storage, routing and availability. Tools for deployment, configuration, cost analysis and developer productivity are also discussed.
Databases in the Cloud discusses AWS database services for moving workloads to the cloud. It describes Amazon Relational Database Service (RDS) which provides several fully managed relational database options including MySQL, PostgreSQL, MariaDB, Oracle, SQL Server, and Amazon Aurora. It also discusses non-relational database services like DynamoDB, ElastiCache, and Redshift for analytics workloads. The document provides guidance on choosing between SQL and NoSQL databases and discusses benefits of managed database services over hosting databases on-premises or in EC2 instances.
MySQL is commonly used as the default database in OpenStack. It provides high availability through options like Galera and MySQL Group Replication. Galera is a third party active/active cluster that provides synchronous replication, while Group Replication is a native MySQL plugin that also enables active/active clusters with built-in conflict detection. MySQL NDB Cluster is an alternative that provides in-memory data storage with automatic sharding and strong consistency across shards. Both Galera/Group Replication and NDB Cluster can be used to implement highly available MySQL services in OpenStack environments.
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #1: AWS AuroraContinuent
AWS Aurora vs. Continuent Tungsten Clusters
Building a Geo-Scale, Multi-Region and Highly Available MySQL Cloud Back-End
This first installment of our High Noon series of on-demand webinars is focused on AWS Aurora. It looks at some of the key characteristics of AWS Aurora and how it fares as a MySQL HA / DR / Geo-Scale solution, especially when compared to Continuent Tungsten Clustering.
Watch this webinar to learn how to do better MySQL HA / DR / Geo-Scale.
AGENDA
- Goals for the High Noon Webinar Series
- AWS Aurora
- Key Characteristics
- Cross Region Requirements
- RDS Proxy
- Limitations Using AWS Aurora
- How to do better MySQL HA / DR / Geo-Scale?
- AWS Aurora vs Tungsten Clustering
- About Continuent & Its Solutions
PRESENTER
Matthew Lang - Customer Success Director – Americas, Continuent - has over 25 years of experience in database administration, database programming, and system architecture, including the creation of a database replication product that is still in use today. He has designed highly available, scaleable systems that have allowed startups to quickly become enterprise organizations, utilizing a variety of technologies including open source projects, virtualization and cloud.
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQLContinuent
MS Azure Database for MySQL vs. Continuent Tungsten Clusters
Building a Geo-Scale, Multi-Region and Highly Available MySQL Cloud Back-End
This is the third of our High Noon series covering MySQL clustering solutions for high availability (HA), disaster recovery (DR), and geographic distribution.
Azure Database for MySQL is a managed database cluster within Microsoft Azure Cloud that runs MySQL community edition. There are really two deployment options: “Single Server” and “Flexible Server (Preview).” We will look at the Flexible Server version, even though it is still preview, because most enterprise applications require failover, so this is the relevant comparison for Tungsten Clustering.
You may use Tungsten Clustering with native MySQL, MariaDB or Percona Server for MySQL in GCP, AWS, Azure, and/or on-premises data centers for better technological capabilities, control, and flexibility. But learn about the pros and cons!
Enjoy the webinar!
AGENDA
- Goals for the High Noon Webinar Series
- High Noon Series: Tungsten Clustering vs Others
- Microsoft Azure Database for MySQL
- Key Characteristics
- Certification-based Replication
- Azure MySQL Multi-Site Requirements
- Limitations Using Azure MySQL
- How to do better MySQL HA / DR / Geo-Scale?
- Azure MySQL vs Tungsten Clustering
- About Continuent & Its Solutions
PRESENTER
Matthew Lang - Customer Success Director – Americas, Continuent - has over 25 years of experience in database administration, database programming, and system architecture, including the creation of a database replication product that is still in use today. He has designed highly available, scaleable systems that have allowed startups to quickly become enterprise organizations, utilizing a variety of technologies including open source projects, virtualization and cloud.
This was a short 25 minute talk, but we go into a bit of a history of MySQL, how the branches and forks appeared, what's sticking around today (branch? Percona Server. Fork? MariaDB Server). What should you use? Think about what you need today and what the roadmap holds.
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #5: Oracle’s InnoDB ClusterContinuent
Oracle’s InnoDB Cluster vs. Continuent Tungsten Clusters for MySQL
Building a Geo-Distributed, Multi-Region and Highly Available MySQL Cloud Back-End
This is the fifth of our High Noon series covering MySQL clustering solutions for high availability (HA), disaster recovery (DR), and geographic distribution.
InnoDB Cluster uses MySQL’s group replication to handle the replication. It’s also known as semi-synchronous replication. Learn about this and more in this webinar!
You may use Tungsten Clustering with native MySQL, MariaDB or Percona Server for MySQL in GCP, AWS, Azure, and/or on-premises data centers for better technological capabilities, control, and flexibility. But learn about the pros and cons!
AGENDA
- Goals for the High Noon Webinar Series
- High Noon Series: Tungsten Clustering vs Others
- Oracle InnoDB Cluster
- Key Characteristics
- Certification-based Replication
- InnoDB Cluster Multi-Site Requirements
- Limitations Using InnoDB Cluster
- How to do better MySQL HA / DR / Geo-Distribution?
- InnoDB Cluster vs Tungsten Clustering
- About Continuent & Its Solutions
PRESENTER
Matthew Lang - Customer Success Director – Americas, Continuent - has over 25 years of experience in database administration, database programming, and system architecture, including the creation of a database replication product that is still in use today. He has designed highly available, scaleable systems that have allowed startups to quickly become enterprise organizations, utilizing a variety of technologies including open source projects, virtualization and cloud.
Cloudera Impala - Las Vegas Big Data Meetup Nov 5th 2014cdmaxime
Maxime Dumas gives a presentation on Cloudera Impala, which provides fast SQL query capability for Apache Hadoop. Impala allows for interactive queries on Hadoop data in seconds rather than minutes by using a native MPP query engine instead of MapReduce. It offers benefits like SQL support, improved performance of 3-4x up to 90x faster than MapReduce, and flexibility to query existing Hadoop data without needing to migrate or duplicate it. The latest release of Impala 2.0 includes new features like window functions, subqueries, and spilling joins and aggregations to disk when memory is exhausted.
Percona ServerをMySQL 5.6と5.7用に作るエンジニアリング(そしてMongoDBのヒント)Colin Charles
Engineering that goes into making Percona Server for MySQL 5.6 & 5.7 different (and a hint of MongoDB) for dbtechshowcase 2017 - the slides also have some Japanese in it. This should help a Japanese audience to read it. If there are questions due to poor translation, do not hesitate to drop me an email (byte@bytebot.net) or tweet: @bytebot
Databases require capacity planning (and to those coming from traditional RDBMS solutions, this can be thought of as a sizing guide). Capacity planning prevents resource exhaustion. Capacity planning can be hard. This talk has a heavier leaning on MySQL, but the concepts and addendum will help with any other data store.
Lessons from {distributed,remote,virtual} communities and companiesColin Charles
A last minute talk for the people at DevOps Amsterdam, happening around the same time as O'Reilly Velocity Amsterdam 2016. Here are lessons one can learn from distributed/remote/virtual communities and companies from someone that has spent a long time being remote and distributed.
Forking Successfully - or is a branch better?Colin Charles
Forking Successfully or do you think a branch will work better? Learn from history, see what's current, etc. Presented at OSCON London 2016. This is forking beyond the github generation. And if you're going to do it, some tips on how you could be successful.
Best practices for MySQL/MariaDB Server/Percona Server High AvailabilityColin Charles
Best practices for MySQL/MariaDB Server/Percona Server High Availability - presented at Percona Live Amsterdam 2016. The focus is on picking the right High Availability solution, discussing replication, handling failure (yes, you can achieve a quick automatic failover), proxies (there are plenty), HA in the cloud/geographical redundancy, sharding solutions, how newer versions of MySQL help you, and what to watch for next.
Failure happens, and we can learn from it. We need to think about backups, but also verification of them. We should definitely make use of replication and think about automatic failover. And security is key, but don't forget that encryption is now available in MySQL, Percona Server and MariaDB Server.
Presented at the MySQL Chicago Meetup in August 2016. The focus of the talk is on backups and verification, replication and failover, as well as security and encryption.
An introduction to MongoDB from an experienced MySQL user and developer. There are differences and we go thru the What/Why/Who/Where of MongoDB, the "similarities" to the MySQL world like storage engines, how replication is a little more interesting with built-in sharding and automatic failover, backups, monitoring, DBaaS, going to production and finding out more resources.
Tuning Linux for your database FLOSSUK 2016Colin Charles
Some best practices about tuning Linux for your database workloads. The focus is not just on MySQL or MariaDB Server but also on understanding the OS from hardware/cloud, I/O, filesystems, memory, CPU, network, and resources.
Having spent more than the last decade being the main point of contact for distributions shipping MySQL, then MariaDB Server, it's clear that working with distributions have many challenges. Licensing changes (when MySQL moved the client libraries from LGPL to GPL with a FOSS Exception), ABI changes, speed (or lack thereof) of distribution releases/freezes, supporting the software throughout the lifespan of the distribution, specific bugs due to platforms, and a lot more will be discussed in this talk. Let's not forget the politics. How do we decide "tiers" of importance for distributions? As a bonus, there will be a focus on how much effort it took to "replace" MySQL with MariaDB.
Benefits: if you're making a distribution, this is the point of view of the upstream package makers. Why are distribution statistics important to us? Do we monitor your bugs system or do you have a better escalation to us? How do we test to make sure things are going well before release. This and more will be spoken about.
As an upstream project (package), we love nothing more than being available everywhere. But time and energy goes into making this is so as there are quirks in every distribution.
Meet MariaDB Server 10.1 London MySQL meetup December 2015Colin Charles
Meet MariaDB Server 10.1, the server that got released recently. Presented at the London MySQL Meetup in December 2015. Learn about the new features in MariaDB Server, especially around the focus of what we did to improve security.
This document discusses MariaDB plugins and provides examples of several useful plugins, including authentication plugins, password validation plugins, SQL error logging, audit logging, query analysis, and more. It encourages contributing plugins to help extend MariaDB's functionality.
Better encryption & security with MariaDB 10.1 & MySQL 5.7Colin Charles
Talking about the improvements in MariaDB on MySQL security and encryption features that are so important in today's data landscape. Presented http://www.meetup.com/EffectiveMySQL/events/224828891/
Coordinate Systems in FME 101 - Webinar SlidesSafe Software
If you’ve ever had to analyze a map or GPS data, chances are you’ve encountered and even worked with coordinate systems. As historical data continually updates through GPS, understanding coordinate systems is increasingly crucial. However, not everyone knows why they exist or how to effectively use them for data-driven insights.
During this webinar, you’ll learn exactly what coordinate systems are and how you can use FME to maintain and transform your data’s coordinate systems in an easy-to-digest way, accurately representing the geographical space that it exists within. During this webinar, you will have the chance to:
- Enhance Your Understanding: Gain a clear overview of what coordinate systems are and their value
- Learn Practical Applications: Why we need datams and projections, plus units between coordinate systems
- Maximize with FME: Understand how FME handles coordinate systems, including a brief summary of the 3 main reprojectors
- Custom Coordinate Systems: Learn how to work with FME and coordinate systems beyond what is natively supported
- Look Ahead: Gain insights into where FME is headed with coordinate systems in the future
Don’t miss the opportunity to improve the value you receive from your coordinate system data, ultimately allowing you to streamline your data analysis and maximize your time. See you there!
YOUR RELIABLE WEB DESIGN & DEVELOPMENT TEAM — FOR LASTING SUCCESS
WPRiders is a web development company specialized in WordPress and WooCommerce websites and plugins for customers around the world. The company is headquartered in Bucharest, Romania, but our team members are located all over the world. Our customers are primarily from the US and Western Europe, but we have clients from Australia, Canada and other areas as well.
Some facts about WPRiders and why we are one of the best firms around:
More than 700 five-star reviews! You can check them here.
1500 WordPress projects delivered.
We respond 80% faster than other firms! Data provided by Freshdesk.
We’ve been in business since 2015.
We are located in 7 countries and have 22 team members.
With so many projects delivered, our team knows what works and what doesn’t when it comes to WordPress and WooCommerce.
Our team members are:
- highly experienced developers (employees & contractors with 5 -10+ years of experience),
- great designers with an eye for UX/UI with 10+ years of experience
- project managers with development background who speak both tech and non-tech
- QA specialists
- Conversion Rate Optimisation - CRO experts
They are all working together to provide you with the best possible service. We are passionate about WordPress, and we love creating custom solutions that help our clients achieve their goals.
At WPRiders, we are committed to building long-term relationships with our clients. We believe in accountability, in doing the right thing, as well as in transparency and open communication. You can read more about WPRiders on the About us page.
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsMydbops
This presentation, delivered at the Postgres Bangalore (PGBLR) Meetup-2 on June 29th, 2024, dives deep into connection pooling for PostgreSQL databases. Aakash M, a PostgreSQL Tech Lead at Mydbops, explores the challenges of managing numerous connections and explains how connection pooling optimizes performance and resource utilization.
Key Takeaways:
* Understand why connection pooling is essential for high-traffic applications
* Explore various connection poolers available for PostgreSQL, including pgbouncer
* Learn the configuration options and functionalities of pgbouncer
* Discover best practices for monitoring and troubleshooting connection pooling setups
* Gain insights into real-world use cases and considerations for production environments
This presentation is ideal for:
* Database administrators (DBAs)
* Developers working with PostgreSQL
* DevOps engineers
* Anyone interested in optimizing PostgreSQL performance
Contact info@mydbops.com for PostgreSQL Managed, Consulting and Remote DBA Services
Mitigating the Impact of State Management in Cloud Stream Processing SystemsScyllaDB
Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states.
In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing.
Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.
Measuring the Impact of Network Latency at TwitterScyllaDB
Widya Salim and Victor Ma will outline the causal impact analysis, framework, and key learnings used to quantify the impact of reducing Twitter's network latency.
Kief Morris rethinks the infrastructure code delivery lifecycle, advocating for a shift towards composable infrastructure systems. We should shift to designing around deployable components rather than code modules, use more useful levels of abstraction, and drive design and deployment from applications rather than bottom-up, monolithic architecture and delivery.
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...Toru Tamaki
Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, Philip Torr "A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models" arXiv2023
https://arxiv.org/abs/2307.12980
The Rise of Supernetwork Data Intensive ComputingLarry Smarr
Invited Remote Lecture to SC21
The International Conference for High Performance Computing, Networking, Storage, and Analysis
St. Louis, Missouri
November 18, 2021
Comparison Table of DiskWarrior Alternatives.pdfAndrey Yasko
To help you choose the best DiskWarrior alternative, we've compiled a comparison table summarizing the features, pros, cons, and pricing of six alternatives.
How Social Media Hackers Help You to See Your Wife's Message.pdfHackersList
In the modern digital era, social media platforms have become integral to our daily lives. These platforms, including Facebook, Instagram, WhatsApp, and Snapchat, offer countless ways to connect, share, and communicate.
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc
Six months into 2024, and it is clear the privacy ecosystem takes no days off!! Regulators continue to implement and enforce new regulations, businesses strive to meet requirements, and technology advances like AI have privacy professionals scratching their heads about managing risk.
What can we learn about the first six months of data privacy trends and events in 2024? How should this inform your privacy program management for the rest of the year?
Join TrustArc, Goodwin, and Snyk privacy experts as they discuss the changes we’ve seen in the first half of 2024 and gain insight into the concrete, actionable steps you can take to up-level your privacy program in the second half of the year.
This webinar will review:
- Key changes to privacy regulations in 2024
- Key themes in privacy and data governance in 2024
- How to maximize your privacy program in the second half of 2024
Sustainability requires ingenuity and stewardship. Did you know Pigging Solutions pigging systems help you achieve your sustainable manufacturing goals AND provide rapid return on investment.
How? Our systems recover over 99% of product in transfer piping. Recovering trapped product from transfer lines that would otherwise become flush-waste, means you can increase batch yields and eliminate flush waste. From raw materials to finished product, if you can pump it, we can pig it.
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Chris Swan
Have you noticed the OpenSSF Scorecard badges on the official Dart and Flutter repos? It's Google's way of showing that they care about security. Practices such as pinning dependencies, branch protection, required reviews, continuous integration tests etc. are measured to provide a score and accompanying badge.
You can do the same for your projects, and this presentation will show you how, with an emphasis on the unique challenges that come up when working with Dart and Flutter.
The session will provide a walkthrough of the steps involved in securing a first repository, and then what it takes to repeat that process across an organization with multiple repos. It will also look at the ongoing maintenance involved once scorecards have been implemented, and how aspects of that maintenance can be better automated to minimize toil.
Support en anglais diffusé lors de l'événement 100% IA organisé dans les locaux parisiens d'Iguane Solutions, le mardi 2 juillet 2024 :
- Présentation de notre plateforme IA plug and play : ses fonctionnalités avancées, telles que son interface utilisateur intuitive, son copilot puissant et des outils de monitoring performants.
- REX client : Cyril Janssens, CTO d’ easybourse, partage son expérience d’utilisation de notre plateforme IA plug & play.
1. Databases in the Hosted Cloud
Colin Charles, Chief Evangelist, Percona Inc.
colin.charles@percona.com / byte@bytebot.net
http://bytebot.net/blog/ | @bytebot on Twitter
OSCON, Portland, Oregon
19 July 2018
2. whoami
• Chief Evangelist, Percona Inc
• Focusing on the MySQL ecosystem (MySQL, Percona Server, MariaDB
Server), as well as the MongoDB ecosystem (Percona Server for MongoDB)
+ 100% open source tools from Percona like Percona Monitoring &
Management, Percona xtrabackup, Percona Toolkit, etc. and now
PostgreSQL too!
• Founding team of MariaDB Server (2009-2016), previously at Monty Program
Ab, merged with SkySQL Ab, now MariaDB Corporation
• Formerly MySQL AB (exit: Sun Microsystems)
• Past lives include Fedora Project (FESCO), OpenOffice.org
• MySQL Community Contributor of the Year Award winner 2014
3. Agenda
• MySQL as a service offering (DBaaS)
• Choices
• Considerations
• MySQL versions & access
• Costs
• Deeper into RDS
• Should you run this on EC2 or an equivalent?
• What about PostgreSQL? MongoDB?
• Conclusion
4. Database as a service
• Database as a Service (DBaaS)
• MySQL/PostgreSQL available on-demand, without any installation/
configuration of hardware/software
• Pay-per-usage based
• Provider maintains database, you don’t maintain, upgrade, or
administer the database
5. New way of deployment
• Enter a credit card number
• call API (or use the GUI)
aws ec2 run-instances --image-id ami-xxx -k $
{EC2_KEYPAIR} -t c3.large
nova boot --image centos6-x86_64 --flavor m1.large db1
credit: http://www.flickr.com/photos/68751915@N05/6280507539/
6. Why DBaaS?
• “Couldn’t we just have a few more servers to handle the traffic spike
during the elections?”
• Don’t have a lot of DBAs, optimise for operational ease
• Rapid deployment & scale-out
7. Your choices today
• Amazon Web Services Relational Database Service (RDS)
• Rackspace Cloud Databases
• Google Cloud SQL
• Alibaba Cloud
• Oracle Cloud MySQL Service
• Microsoft Azure Database
8. There are more
• Jelastic - PaaS offering MySQL, MariaDB Server
• ClearDB - MySQL partnered with Heroku, Azure clouds
• Joyent - Image offers Percona MySQL and a Percona SmartMachine
9. Google Compute Engine
• Google Compute Engine offers Percona XtraDB Cluster as a “click-
to-deploy” app
• comes with Galera 3, Percona Toolkit, XtraBackup as well
10. Pivotal CloudFoundry
• Pivotal CloudFoundry
• “MySQL” PaaS which is MariaDB Galera Cluster 10 (v1, legacy)
• MariaDB v10.1.30 and Galera v25.3.20 (latest release, June 12
2018)
• MySQL for PCF v2
• Percona Server v5.7.20-21 (latest release, June 22 2018)
• Works with IaaS platforms: AWS, Azure, GCP, OpenStack, and
vSphere
• Significance of Galera Cluster versus regular replication
11. Red Hat OpenShift
• Variants between Online & Enterprise editions (usually you get
access to modern releases with OpenShift Online)
• MySQL 5.5/5.6/5.7
• MariaDB 10.0/10.1
• PostgreSQL 9.4/9.5
• MongoDB 2.4/2.6/3.2/3.4
12. Beware
• GenieDB - globally distributed MySQL as a service, master-master
replication, works on EC2, Rackspace, Google Compute Engine, HP
Cloud
• Xeround - 2 weeks notice...
14. Thank you HPCloud
• http://www.bytebot.net/blog/archives/2015/10/27/sunsetting-
hpcloud-whom-contributed-to-making-mysql-better
• Utility user
• enforce_storage_engine
• prevent LOAD DATA INFILE/SELECT INTO OUTFILE
• restrict # of binlog files
16. What else do you get?
• PostgreSQL is getting popular in the cloud: AWS RDS, AWS Aurora,
Microsoft Azure, Alibaba Cloud, Google Cloud SQL
• Most MongoDB users run within the cloud — but not hosted, it tends
to be “roll your own” or via ObjectRocket (and lately MongoDB Atlas
c- reasonable free tier!)
• Compose.io (now IBM) & the like (MongoLab) — hosted MongoDB,
Redis, Enhanced PostgreSQL, ElasticSearch, RethinkDB (!?), MySQL
with group replication, ScyllaDB, etc.
17. Regions & Availability Zones
• Region: a data centre location, containing multiple Availability Zones
• Availability Zone (AZ): isolated from failures from other AZs + low-
latency network connectivity to other zones in same region
18. Location, location, location
• AWS RDS: US East (N. Virginia, Ohio), US West (Oregon, Northern
California, California), EU (Ireland, Frankfurt, London, Paris), APAC
(Singapore, Tokyo, Sydney, Seoul, Mumbai), South America (São
Paulo), GovCloud, Canada (Central), China (Beijing)
• Rackspace: USA (Dallas DFW, Chicago ORD, N. Virginia IAD), APAC
(Sydney, Hong Kong), EU (London)*
• Google Cloud SQL:
• 2nd generation instances: Montréal, Iowa, N. Virginia, Oregon, LA,
São Paulo, Finland, London, Frankfurt, Netherlands, Tokyo, Mumbai,
Singapore, Sydney
19. Service Level Agreements (SLA)
• AWS - at least 99.95% in a calendar month, less than, 10% service credit
• Rackspace - 99.9% in a calendar month
• Google - 99.95% in a calendar month, less than 10% service credit
• Alibaba Cloud - no less than 99.95% in a calendar month (some services,
99.9%)
• Microsoft Azure - < 99.99% will give you a 10% service credit
• SLAs exclude “scheduled maintenance” which may have impact on
storage I/O + elevate latency
• e.g. AWS is 30 minutes/week
20. Support
• AWS - active forums; $100+ (or a % of AWS usage) phone #
• Rackspace - live chat, phone #, forums
• Google - forums; $150/mo gets support portal; $400+ (or a % of
usage fees) for phone #
• Microsoft - forums; $100/mo to start
• Alibaba Cloud - $70 (or a % of usage fees)
• Most have developer support plans at $19.99-$29 even; enterprise
support around $8,000-$15,000
21. Who manages this?
• AWS: self-management, Enterprise ($15k+)
• Rackspace Cloud: $100 + 0.04 cents/hr over regular pricing
• Google: self-management, or via partner, Rackspace Managaed
Services
• Microsoft: generally self, premier can give you advisory services
• Alibaba Cloud: self-management
• Rackspace Managed Services: AWS, Alibaba Cloud, Google Cloud
Platform, Microsoft Azure Cloud, OpenStack Public Cloud
22. MySQL versions
• AWS: MySQL Community 5.1, 5.5, 5.6, 5.7 / MariaDB Server 10.0,
10.1, 10.2
• Rackspace: MariaDB Server 10, 10.1, 10.1-enc, MySQL 5.7/5.6/5.1,
Percona Server 5.6
• Google: MySQL Community 5.5, 5.6, 5.7
• Microsoft: MySQL Community 5.6, 5.7
• Alibaba Cloud: MySQL Community 5.6, 5.7
23. Access methods
• AWS - within Amazon, externally via mysql client, API access.
• Rackspace - private hostname within Rackspace network, API
access.
• Google - within AppEngine, a command line Java tool (gcutil),
standard mysql client
• Microsoft - within Azure, externally via mysql client
• Alibaba Cloud - standard mysql client
24. Can you configure MySQL?
• You don’t access my.cnf
naturally
• In AWS you have parameter
groups which allow
configuration of MySQL
source: http://www.mysqlperformanceblog.com/2013/08/21/amazon-rds-with-mysql-5-6-configuration-variables/
25. Cost
• Subscribe to relevant newsletters of your services
• Cost changes rapidly, plus you get new instance types and new
features (IOPS)
• Don’t forget network access costs
• Monitor your costs daily, hourly if possible (spot pricing — if roll your
own)
26. Costs: AWS
• AWS prices vary between regions
• http://aws.amazon.com/rds/pricing/
27. Costs: AWS II
• Standard latest generation: db.m4.large (2vCPU/8GB) at $1,533/yr
[2017]
• Previous generation: Medium instances (3.75GB) useful for testing
($1,577/yr [2014] vs $2,411/yr [2013])
• Large instance (7.5GB) production ready ($3,241/yr vs $4,777/yr
[2013])
• Multi-AZ production ready: db.m4.2xlarge (32GB, 8vCPUs) at
$12,264/yr [2017]
• Previous generation: m3.2XL (30GB, 8vCPUs) ($12,964/yr)
28. Costs: Rackspace
• Option to have regular Cloud Database or Managed Instances
• 4GB instance is $2,111.4 (vs. $2,102/yr in 2015 and $3,504/yr in
2013)
• 8GB instance is $4,213.8 (vs. $4,205/yr in 2015 and $6,658/yr in
2013)
• Consider looking at I/O priority, and the actual TPS you get
• High Available instances with automatic failover available too
29. Costs: Google
• You must enable billing before you create Cloud SQL instances
• https://developers.google.com/cloud-sql/docs/billing
• Previously (first generation):
• Testing (D8 - 4GB RAM) - $3,204.7 vs $4,274.15 (in 2015)
• XL equivalent (D16 - 8GB RAM) - $6413.05 vs $8,548.30 (in 2015)
• Today:
• db-n1-standard-8 (8 vCPUs, 30GB RAM) - $6,762.72
• Packages billing plans are cheaper than per-use billing plans
30. Costs: Microsoft Azure
• I/O isn’t being charged for yet
• Basic tier with 2 vCores gives you 2GB per vCore (=4GB)
• So… 4 vCores, 8GB RAM, $3,069/year
31. Costs: Alibaba Cloud
• rds.mysql.s2.xlarge - 2vCPU/8GB RAM - $4,537 ($3,192 on
subscription, pay it monthly at $266)
• In USA, that could be $3,109 ($2,112, monthly $176)
• In China? $2,487 ($1,632, monthly $136)
• SQL Audit, backup, monitoring, all costs more
• Readonly storage instance pricing exist
32. Where do you host your application?
• Typically within the compute clusters of the service you’re running
the DBaaS in
• You wan’t a multi-cloud strategy? You will have to have your entire
layers of applications replicated on different clouds
33. RDS: Multi-AZ
• Provides enhanced durability (synchronous data replication)
• Increased availability (automatic failover)
• Warning: can be slow with large database size
• Easy GUI administration
• Doesn’t give you another usable “read-replica” though
34. External replication
• MySQL 5.6 you can do RDS -> Non-RDS
• enable backup retention, you now have binlog access
• target: exporting data out of RDS
• This can help you migrate to another platform as well
• Replicate into RDS with 5.5.33 or later
• AWS provides stored procedures like
mysql.rds_set_external_master nowadays
35. Getting started
• Importing data into the cloud?
• mysqldump is a good choice today
• Upgrading from RDS 5.5 to RDS 5.6?
• mysqldump before, but nowadays you can do this via Read
Replicas
36. Handling backups
• You don’t get to use Percona XtraBackup! (or mydumper)
• Google Cloud SQL automates backups (has a backup window - 4h)
• Amazon has automated backups (with point-in-time recovery), with full daily
snapshots (has a backup window).
• Multi-AZ? Backup taken from the standby!
• Backup retention default? 1 day. Increase it
• Aria may not work well with automatic backups, so use InnoDB/XtraDB
• Rackspace allow instance backups too
• Microsoft has automatic backups (with point-in-time recovery). Backup
retention is 7 days (up to 35 days), with redundancy options
37. Monitoring
• AWS has the best options currently available
• Today you have CloudWatch
• Google has improved on their read/write graphs, integration with
Stackdriver monitoring
• Rackspace has started with basic graphs, visuals for MySQL, have a
Cloud Intelligence product
• Percona Monitoring & Management - http://
pmmdemo.percona.com/
• DataDog, VividCortex, etc.
38. Storage Engines
• MySQL (/MariaDB) have many
• cool ones include MyRocks, TokuDB, SPIDER, CONNECT
• You basically use InnoDB (XtraDB) and MyISAM with cloud solutions
• MyISAM on RDS won’t guarantee point-in-time recovery, snapshot
restore
39. High Availability
• Plan for node failures
• Don’t assume node provisioning is quick
• Backup, backup, backup!
• “Bad” nodes exist
• RDS? Multi-AZ.
• HA is not equal across options
• Rackspace has High Availability Database instances built on their
ObjectRocket platform
• Google? Deploys semi-synchronous replication, so you do get a usable replica
• alert for replication lag
40. Unsupported features
• AWS MySQL: GTIDs (but MariaDB Server GTIDs work!), InnoDB Cache
Warming (intra-schema parallel replication in 5.7 works - this was an XtraDB
5.6 feature), InnoDB transportable tablespaces, authentication plugins,
password strength plugin, replication filters, semi-sync replication
• AWS MariaDB: Data at Rest Encryption, MariaDB Galera Cluster,
HandlerSocket, Multi-source Replication, Password validation plugin,
simple_password_check, and cracklib_password_check, Replication Filters,
Storage engine-specific object attributes, table and Tablespace Encryption
• Google: UDFs, PERFORMANCE_SCHEMA, LOAD DATA INFILE, INSTALL
PLUGIN, SELECT ... INTO OUTFILE
• mysqlsh?
42. Provisioned IOPS
• Available on Amazon, and recently Google
• Faster, predictable, consistent I/O performance with low latencies
• Good throughput, RAID on backed
• EBS is more reliable
43. More on running in the hosted cloud…
• log access via API
• no SUPER access to skip replication errors easily
• sync_binlog=0 not available
• no OS access (sar, ps, tcpdump)
44. Warning: automatic upgrades
• Regressions happen even with a minor version upgrade in the
MySQL world (though this is happening a lot less as quality has
increased tremendously)
• InnoDB update that modifies rows PK triggers recursive behaviour
until all disk space is exceeded? 5.5.24->5.5.25 (fixed: 5.5.25a)
• Using query cache for partitioned tables? Disabled since 5.5.22-
>5.5.23!
45. Benchmarking for use
• sysbench
• OLTP test, use tables with 20M rows and 20M transactions, check
1-128 threads/run (run this on RDS, Rackspace)
• June 2013, tps, performance per dollar, Rackspace delivers more
performance across all flavours except 512MB instance
• Yahoo! Cloud Serving Benchmark
• https://github.com/brianfrankcooper/YCSB
• Google’s PerfKit Benchmarker
• https://github.com/GoogleCloudPlatform/PerfKitBenchmarker
46. Roadmaps?
• There don’t seem to be public roadmaps. You find out when there’s a
change!
• Watch out for those events…
48. PostgreSQL in RDS
• loading data?
backup_retention=0
• disable multi-AZ when loading
• disable autovacuum
• dump compressed, restore in
parallel
• don’t disable fsync (really!)
• sync replication using multi-AZ
• you can control the upgrade
time though — this is a bonus
• Use PIOPS
• SSL should be on
49. Running MySQL in EC2
• Can do multiple geographic
regions via replication
• Run just one Percona Server/
MariaDB server/instance
• Use additional EBS volumes for
InnoDB tablespaces
• RAID EBS volumes (RAID1)
• Warm up data partitions, mount
partitions with noatime, nodirtime
• Vertical scaling with SSD-backed
storage
• Monitoring with Icinga/nagios
• Snapshot backups and save to S3
• Can use Elastic Load Balancer
• Can use spot instances
• Can use tools like MHA to provide
automatic failover
• Can use MariaDB Galera Cluster/
Percona XtraDB Cluster
50. AWS Aurora
• Bigger instances work better
• Zero-downtime migration from RDS
• Metrics via CloudWatch, Connectors
via MariaDB
• 99.99% uptime
• MySQL 5.6.10 “fork”, no optimiser,
not traditional replication (but Aurora
<->MySQL works of course)
• MySQL 5.7.12 Aurora launched Feb
2018, with JSON support, spatial
indexes, generated columns, etc.
• Auto scaling - compute, memory,
storage
• Replicas (15) for reads
• Automated backups in S3, DB
snapshots
• Encryption with key server being
Amazon KMS
• Spatial data support - like InnoDB
5.7!
• Lab mode (hash joins, scan
batching, etc.)
51. Looking ahead
• OtterTune: automatically find good settings for a database
configuration - https://github.com/cmu-db/ottertune
• Peloton: self-driving database management system - http://
pelotondb.io/
60% reduction in latency,
22-35% better throughput
https://aws.amazon.com/blogs/ai/tuning-your-dbms-automatically-with-machine-learning/
52. Some closing thoughts
• Hardware varies per region
• Sometimes, software manageability varies per region
• Beware costs on your credit card!
• These things change often, in terms of pricing, instance availability — so regularly
monitor latest news
• Don’t upgrade immediately to the latest new releases
• Always read release notes
• If going the EC2 (or equivalent) route, maybe have other management services in place
(e.g. Ewhat MongoDB provides)
• Sharding — vitess?
• Proxies — ProxySQL, MySQL Router