Today you can use hosted MySQL/MariaDB/Percona Server in several "cloud providers" in what is considered using it as a service, a database as a service (DBaaS). You can also use hosted PostgreSQL and MongoDB thru various service providers. Learn the differences, the access methods, and the level of control you have for the various public cloud offerings:
- Amazon RDS for MySQL and PostgreSQL
- Google Cloud SQL
- Rackspace OpenStack DBaaS
- The likes of compose.io, MongoLab and Rackspace's offerings around MongoDB
The administration tools and ideologies behind it are completely different, and you are in a "locked-down" environment. Some considerations include:
* Different backup strategies
* Planning for multiple data centres for availability
* Where do you host your application?
* How do you get the most performance out of the solution?
* What does this all cost?
Growth topics include:
* How do you move from one DBaaS to another?
* How do you move all this from DBaaS to your own hosted platform?
Questions like this will be demystified in the talk. This talk will benefit experienced database administrators (DBAs) who now also have to deal with cloud deployments as well as application developers in startups that have to rely on "managed services" without the ability of a DBA.
MariaDB Server & MySQL Security Essentials 2016Colin Charles
This document summarizes a presentation on MariaDB/MySQL security essentials. The presentation covered historically insecure default configurations, privilege escalation vulnerabilities, access control best practices like limiting privileges to only what users need and removing unnecessary accounts. It also discussed authentication methods like SSL, PAM, Kerberos and audit plugins. Encryption at the table, tablespace and binary log level was explained as well. Preventing SQL injections and available security assessment tools were also mentioned.
MySQL is a unique adult (now 21 years old) in many ways. It supports plugins. It supports storage engines. It is also owned by Oracle, thus birthing two branches of the popular opensource database: Percona Server and MariaDB Server. It also once spawned a fork: Drizzle. Lately a consortium of web scale users (think a chunk of the top 10 sites out there) have spawned WebScaleSQL.
You're a busy DBA having to maintain a mix of this. Or you're a CIO planning to choose one branch. How do you go about picking? Supporting multiple databases? Find out more in this talk. Also covered is a deep-dive into what feature differences exist between MySQL/Percona Server/MariaDB/WebScaleSQL, how distributions package the various databases differently. Within the hour, you'll be informed about the past, the present, and hopefully be knowledgeable enough to know what to pick in the future.
Note, there will also be coverage of the various trees around WebScaleSQL, like the Facebook tree, the Alibaba tree as well as the Twitter tree.
This is my third iteration of the talk presented in Tokyo, Japan - first was at a keynote at rootconf.in in April 2016, then at the MySQL meetup in New York, and now for dbtechshowcase. The focus is on database failures of the past, and how modern MySQL / MariaDB Server technologies could have helped them avoid such failure. The focus is on backups and verification, replication and failover, and security and encryption.
MariaDB 10 Tutorial - 13.11.11 - Percona Live LondonIvan Zoratti
This document provides an overview and summary of MariaDB 10 features presented by Ivan Zoratti. It discusses new features in MariaDB 10 like storage engines, administration improvements, and replication capabilities. The document also summarizes optimization enhancements in MariaDB 10 like the new optimizer, improved indexing techniques, and subquery optimizations. Various agenda topics are outlined for the MariaDB 10 tutorial.
The Complete MariaDB Server Tutorial - Percona Live 2015Colin Charles
The document provides an overview of the Complete MariaDB Server Tutorial presentation. It introduces MariaDB and discusses what it is, its goals of being compatible with MySQL and having stable releases. It also covers MariaDB architecture, installation, utilities, and storage engines.
Securing your MySQL / MariaDB Server dataColin Charles
Co-presented alongside Ronald Bradford, this covers MySQL, Percona Server, and MariaDB Server (since the latter occasionally can be different enough). Go thru insecure practices, focus on communication security, connection security, data security, user accounts and server access security.
An introduction to MongoDB from an experienced MySQL user and developer. There are differences and we go thru the What/Why/Who/Where of MongoDB, the "similarities" to the MySQL world like storage engines, how replication is a little more interesting with built-in sharding and automatic failover, backups, monitoring, DBaaS, going to production and finding out more resources.
Having spent more than the last decade being the main point of contact for distributions shipping MySQL, then MariaDB Server, it's clear that working with distributions have many challenges. Licensing changes (when MySQL moved the client libraries from LGPL to GPL with a FOSS Exception), ABI changes, speed (or lack thereof) of distribution releases/freezes, supporting the software throughout the lifespan of the distribution, specific bugs due to platforms, and a lot more will be discussed in this talk. Let's not forget the politics. How do we decide "tiers" of importance for distributions? As a bonus, there will be a focus on how much effort it took to "replace" MySQL with MariaDB.
Benefits: if you're making a distribution, this is the point of view of the upstream package makers. Why are distribution statistics important to us? Do we monitor your bugs system or do you have a better escalation to us? How do we test to make sure things are going well before release. This and more will be spoken about.
As an upstream project (package), we love nothing more than being available everywhere. But time and energy goes into making this is so as there are quirks in every distribution.
MySQL in the Hosted Cloud - Percona Live 2015Colin Charles
Colin Charles presented on running MySQL in the hosted cloud. He discussed various database as a service (DBaaS) options like Amazon RDS, Rackspace, and Google Cloud SQL. Key considerations for DBaaS include location, service level agreements, support options, available MySQL/MariaDB versions, access methods, configuration options, costs, and features like high availability and backups. Running MySQL on EC2 is also an option but requires more management of hardware, software, networking, storage and backups. Benchmarking and monitoring tools were recommended to evaluate performance and usage.
MariaDB started life as a database to host the Maria storage engine in 2009. Not long after its inception, the MySQL community went through yet another change in ownership, and it was deemed that MariaDB will be a complete database branch developed to extend MySQL, but with constant merging of upstream changes.
The goal of the MariaDB project is to ensure that everyone is part of the community, including employees of the major steering companies. MariaDB also features enhanced features, some of which are common with the Percona Performance Server. Most importantly, MariaDB is a drop-in replacement and is completely backward compatible with MySQL. In 2010, MariaDB released 5.1 in February, and 5.2 in November – two major releases in a span of one calendar year is a feat that was achieved!
DBAs and developers alike will gain an introduction to MariaDB, what is different with MySQL, how to make use of the feature enhancements, and more.
This document discusses various MySQL high availability solutions and best practices. It begins with an introduction to the presenter and their background and experience. Then it discusses the problems of redundancy, scaling, and high availability that these solutions aim to address. Several specific solutions are covered in detail, including Galera Cluster, master-slave replication, MySQL Cluster, Group Replication, MaxScale, MySQL Router, and MySQL InnoDB Cluster. Key features of each are summarized. The document concludes with an invitation for questions.
Presented at Percona Live Amsterdam 2016, this is an in-depth look at MariaDB Server right up to MariaDB Server 10.1. Learn the differences. See what's already in MySQL. And so on.
MariaDB is a community developed branch of MySQL that is feature enhanced and backward compatible. It aims to be a 100% drop-in replacement for MySQL that is stable, bug-free, and released under the GPLv2 license. Major releases of MariaDB include new storage engines like XtraDB and Aria, as well as new features for performance, scalability, and compatibility. MariaDB is developed as an open source project and supported by Monty Program and other community contributors and service providers.
The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScaleColin Charles
This document discusses MySQL proxy technologies including MySQL Router, ProxySQL, and MariaDB MaxScale. It provides an overview of each technology, including when they were released, key features, and comparisons between them. ProxySQL is highlighted as a popular option currently with integration with Percona tools, while MySQL Router may become more widely used due to its support for MySQL InnoDB Cluster. MariaDB MaxScale is noted for its binlog routing capabilities. Overall the document aims to help people understand and choose between the different MySQL proxy options.
MariaDB: in-depth (hands on training in Seoul)Colin Charles
MariaDB is a community-developed fork of MySQL that aims to be a drop-in replacement. It focuses on being compatible, stable with no regressions, and feature-enhanced compared to MySQL. The presentation covered MariaDB's architecture including connections, query caching, storage engines, and tools for administration and development like mysql, mysqldump, and EXPLAIN.
MariaDB 10 and what's new with the projectColin Charles
This document provides an overview of MariaDB 10.0 and what's new compared to previous versions. Some of the key highlights include backporting features from MySQL 5.6 such as InnoDB, Performance Schema, and online ALTER TABLE. MariaDB 10.0 also includes new features like multi-source replication, persistent statistics, and integration with NoSQL databases. The goals are to have feature parity with MySQL 5.6 and provide an open source alternative to Oracle's MySQL with more active development.
MariaDB 10: A MySQL Replacement - HKOSC Colin Charles
MariaDB 10: A MySQL Replacement. Current up to 10.0.9, right before the 10.0.10 GA release presented the weekend before the release in Hong Kong, at the Hong Kong Open Source Conference.
You want to use MySQL in Amazon RDS, Rackspace Cloud, Google Cloud SQL or HP Helion Public Cloud? Check this out, from Percona Live London 2014. (Note that pricing of Google Cloud SQL changed prices on the same day after the presentation)
Today you can use MySQL in several clouds in what is considered using it as a service, a database as a service (DBaaS). Learn the differences, the access methods, and the level of control you have for the various cloud offerings including:
- Amazon RDS
- Google Cloud SQL
- HPCloud DBaaS
- Rackspace Openstack DBaaS
The administration tools and ideologies behind it are completely different, and you are in a "locked-down" environment. Some considerations include:
* Different backup strategies
* Planning for multiple data centres for availability
* Where do you host your application?
* How do you get the most performance out of the solution?
* What does this all cost?
Questions like this will be demystified in the talk.
Presented at OSCON 2018. A review of what is available from MySQL, MariaDB Server, MongoDB, PostgreSQL, and more. Covering your choices, considerations, versions, access methods, cost, a deeper look at RDS and if you should run your own instances or not.
Maria db 10 and the mariadb foundation(colin)kayokogoto
This document provides an overview of MariaDB 10 and the MariaDB Foundation. It discusses the history and development of MariaDB, including key features added in versions 5.1 through 10.0 such as new storage engines, performance improvements, and features backported from MySQL. It outlines the goals of MariaDB to be compatible with MySQL while adding new features, and describes the community-led development model. The roadmap aims to have MariaDB be a drop-in replacement for MySQL 5.6 by releasing version 10.1.
With a focus on Amazon AWS RDS MySQL and PostgreSQL, Rackspace cloud, Google Cloud SQL, Microsoft Azure for MySQL and PostgreSQL as well as a hint of the other clouds
Google Cloud Platform, Compute Engine, and App EngineCsaba Toth
Introduction to Google Cloud Platform's compute section, Google Compute Engine, Google App Engine. Place these technologies into the cloud service stack, and later show how Google blurs the boundaries of IaaS and PaaS.
The document summarizes the history and current state of the MySQL database server ecosystem. It discusses the origins and development of MySQL, MariaDB, Percona Server, and other related projects. It also describes some of the key features and innovations in recent versions of these database servers. The ecosystem is very active with contributions from many organizations and the future remains promising with ongoing work.
MariaDB - a MySQL Replacement #SELF2014Colin Charles
MariaDB - a MySQL replacement at South East Linux Fest 2014 - SELF2014. Learn about features that are not in MySQL 5.6, some that are only just coming in MySQL 5.7, and some that just don't exist.
This document discusses different cloud platforms for hosting Grails applications. It provides an overview of infrastructure as a service (IaaS) models like Amazon EC2 and shared/dedicated virtual private servers, as well as platform as a service (PaaS) options including Amazon Beanstalk, Google App Engine, Heroku, Cloud Foundry, and Jelastic. A comparison chart evaluates these platforms based on factors such as pricing, control, reliability, and scalability. The document emphasizes that competition and changes in the cloud space are rapid and recommends keeping applications loosely coupled and testing platforms using free trials.
Cloudera Impala - Las Vegas Big Data Meetup Nov 5th 2014cdmaxime
Maxime Dumas gives a presentation on Cloudera Impala, which provides fast SQL query capability for Apache Hadoop. Impala allows for interactive queries on Hadoop data in seconds rather than minutes by using a native MPP query engine instead of MapReduce. It offers benefits like SQL support, improved performance of 3-4x up to 90x faster than MapReduce, and flexibility to query existing Hadoop data without needing to migrate or duplicate it. The latest release of Impala 2.0 includes new features like window functions, subqueries, and spilling joins and aggregations to disk when memory is exhausted.
MariaDB is a community developed fork of MySQL that is feature enhanced and backward compatible. It aims to be a 100% drop-in replacement for MySQL. Recent versions have included storage engines like Percona XtraDB and PrimeBase PBXT, as well as new features like pluggable authentication, virtual columns, and an improved query optimizer. The project is open source and community developed by the MariaDB Foundation and its partners. Future plans include a focus on InnoDB and replication improvements.
This document summarizes MariaDB 10.0 and what's new in the project. It provides an overview of MariaDB's history and goals of being compatible with MySQL. Key features of MariaDB 10.0 include backported features from MySQL 5.6, new features like multi-source replication, and engines for Cassandra and LevelDB. The roadmap is to have parity with MySQL 5.6 by MariaDB 10.1 while continuing to enhance and expand the feature set. Community involvement and the new MariaDB Foundation are discussed.
* Use cases of MySQL as well as edge cases of MySQL topologies using real-life examples and "war" stories
* How scalability and proxy wars make MySQL topologies more robust to serve webscale shops
* Open-source tools, utilities, and surrounding MySQL Ecosystem.
MySQL is commonly used as the default database in OpenStack. It provides high availability through options like Galera and MySQL Group Replication. Galera is a third party active/active cluster that provides synchronous replication, while Group Replication is a native MySQL plugin that also enables active/active clusters with built-in conflict detection. MySQL NDB Cluster is an alternative that provides in-memory data storage with automatic sharding and strong consistency across shards. Both Galera/Group Replication and NDB Cluster can be used to implement highly available MySQL services in OpenStack environments.
This was a short 25 minute talk, but we go into a bit of a history of MySQL, how the branches and forks appeared, what's sticking around today (branch? Percona Server. Fork? MariaDB Server). What should you use? Think about what you need today and what the roadmap holds.
OpenStack Days East -- MySQL Options in OpenStackMatt Lord
In most production OpenStack installations, you want the backing metadata store to be highly available. For this, the de facto standard has become MySQL+Galera. In order to help you meet this basic use case even better, I will introduce you to the brand new native MySQL HA solution called MySQL Group Replication. This allows you to easily go from a single instance of MySQL to a MySQL service that's natively distributed and highly available, while eliminating the need for any third party library and implementations.
If you have an extremely large OpenStack installation in production, then you are likely to eventually run into write scaling issues and the metadata store itself can become a bottleneck. For this use case, MySQL NDB Cluster can allow you to linearly scale the metadata store as your needs grow. I will introduce you to the core features of MySQL NDB Cluster--which include in-memory OLTP, transparent sharding, and support for active/active multi-datacenter clusters--that will allow you to meet even the most demanding of use cases with ease.
The MySQL ecosystem - understanding it, not running away from it! Colin Charles
You're a busy DBA thinking about having to maintain a mix of this. Or you're a CIO planning to choose one branch over another. How do you go about picking? Supporting multiple databases? Find out more in this talk. Also covered is a deep-dive into what feature differences exist between MySQL/Percona Server/MariaDB Server. Within 20 minutes, you'll leave informed and knowledgable on what to pick.
A base blog post to get started: https://www.percona.com/blog/2017/11/02/mysql-vs-mariadb-reality-check/
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdfAlkin Tezuysal
MySQL is still hot, with Percona XtraDB Cluster (PXC) and MariaDB Server. Welcome back post-pandemic to see what is on offer in the current ecosystem.
Did you know that Amazon RDS now uses semi-sync replication rather than DRBD for multi-AZ deployments? Did you know that Galera Cluster for MySQL 8 is much more efficient with CLONE SST rather than using the xtrabackup method for SST? Did you know that Percona Server continues to extend MyRocks? Did you know that MariaDB Server has more Oracle syntax compatibility? This and more will be covered in the session, while short and quick, should leave you wandering to discover new features for production.
Introducing Apache Kudu (Incubating) - Montreal HUG May 2016Mladen Kovacevic
The document introduces Apache Kudu (incubating), a new updatable columnar storage system for Apache Hadoop designed for fast analytics on fast and changing data. It was designed to simplify architectures that use HDFS and HBase together. Kudu aims to provide high throughput for scans, low latency for individual rows, and database-like ACID transactions. It uses a columnar format and is optimized for SSD and new storage technologies.
MariaDB - Fast, Easy & Strong - Get Started Tutorialphamhphuc
MariaDB - Fast, Easy & Strong - Get Started Guide. You can understand why you should use MariaDB and how easy to install it for your server. Let 's enjoy!!!
Differences between MariaDB 10.3 & MySQL 8.0Colin Charles
MySQL and MariaDB are becoming more divergent. Learn what is different from a high level. It is also a good idea to ensure that you use the correct database for the correct job.
MariaDB Server 10.3 is a culmination of features from MariaDB Server 10.2+10.1+10.0+5.5+5.3+5.2+5.1 as well as a base branch from MySQL 5.5 and backports from MySQL 5.6/5.7. It has many new features, like a GA-ready sharding engine (SPIDER), MyRocks, as well as some Oracle compatibility, system versioned tables and a whole lot more.
MySQL features missing in MariaDB ServerColin Charles
MySQL features missing in MariaDB Server. Here's an overview from the New York developer's Unconference in February 2018. This is primarily aimed at the developers, to decide what goes into MariaDB 10.4, as opposed to users.
High level comparisons are made between MySQL 5.6/5.7 with of course MySQL 8.0 as well. Here's to ensuring MariaDB Server 10/310.4 has more "Drop-in" compatibility.
Percona ServerをMySQL 5.6と5.7用に作るエンジニアリング(そしてMongoDBのヒント)Colin Charles
Engineering that goes into making Percona Server for MySQL 5.6 & 5.7 different (and a hint of MongoDB) for dbtechshowcase 2017 - the slides also have some Japanese in it. This should help a Japanese audience to read it. If there are questions due to poor translation, do not hesitate to drop me an email (byte@bytebot.net) or tweet: @bytebot
Databases require capacity planning (and to those coming from traditional RDBMS solutions, this can be thought of as a sizing guide). Capacity planning prevents resource exhaustion. Capacity planning can be hard. This talk has a heavier leaning on MySQL, but the concepts and addendum will help with any other data store.
Lessons from {distributed,remote,virtual} communities and companiesColin Charles
A last minute talk for the people at DevOps Amsterdam, happening around the same time as O'Reilly Velocity Amsterdam 2016. Here are lessons one can learn from distributed/remote/virtual communities and companies from someone that has spent a long time being remote and distributed.
Forking Successfully - or is a branch better?Colin Charles
Forking Successfully or do you think a branch will work better? Learn from history, see what's current, etc. Presented at OSCON London 2016. This is forking beyond the github generation. And if you're going to do it, some tips on how you could be successful.
MariaDB Server Compatibility with MySQLColin Charles
At the MariaDB Server Developer's meeting in Amsterdam, Oct 8 2016. This was the deck to talk about what MariaDB Server 10.1/10.2 might be missing from MySQL versions up to 5.7. The focus is on compatibility of MariaDB Server with MySQL.
Failure happens, and we can learn from it. We need to think about backups, but also verification of them. We should definitely make use of replication and think about automatic failover. And security is key, but don't forget that encryption is now available in MySQL, Percona Server and MariaDB Server.
Presented at the MySQL Chicago Meetup in August 2016. The focus of the talk is on backups and verification, replication and failover, as well as security and encryption.
7 Most Powerful Solar Storms in the History of Earth.pdfEnterprise Wired
Solar Storms (Geo Magnetic Storms) are the motion of accelerated charged particles in the solar environment with high velocities due to the coronal mass ejection (CME).
Best Practices for Effectively Running dbt in Airflow.pdfTatiana Al-Chueyr
As a popular open-source library for analytics engineering, dbt is often used in combination with Airflow. Orchestrating and executing dbt models as DAGs ensures an additional layer of control over tasks, observability, and provides a reliable, scalable environment to run dbt models.
This webinar will cover a step-by-step guide to Cosmos, an open source package from Astronomer that helps you easily run your dbt Core projects as Airflow DAGs and Task Groups, all with just a few lines of code. We’ll walk through:
- Standard ways of running dbt (and when to utilize other methods)
- How Cosmos can be used to run and visualize your dbt projects in Airflow
- Common challenges and how to address them, including performance, dependency conflicts, and more
- How running dbt projects in Airflow helps with cost optimization
Webinar given on 9 July 2024
Mitigating the Impact of State Management in Cloud Stream Processing SystemsScyllaDB
Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states.
In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing.
Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.
Kief Morris rethinks the infrastructure code delivery lifecycle, advocating for a shift towards composable infrastructure systems. We should shift to designing around deployable components rather than code modules, use more useful levels of abstraction, and drive design and deployment from applications rather than bottom-up, monolithic architecture and delivery.
How RPA Help in the Transportation and Logistics Industry.pptxSynapseIndia
Revolutionize your transportation processes with our cutting-edge RPA software. Automate repetitive tasks, reduce costs, and enhance efficiency in the logistics sector with our advanced solutions.
The DealBook is our annual overview of the Ukrainian tech investment industry. This edition comprehensively covers the full year 2023 and the first deals of 2024.
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsMydbops
This presentation, delivered at the Postgres Bangalore (PGBLR) Meetup-2 on June 29th, 2024, dives deep into connection pooling for PostgreSQL databases. Aakash M, a PostgreSQL Tech Lead at Mydbops, explores the challenges of managing numerous connections and explains how connection pooling optimizes performance and resource utilization.
Key Takeaways:
* Understand why connection pooling is essential for high-traffic applications
* Explore various connection poolers available for PostgreSQL, including pgbouncer
* Learn the configuration options and functionalities of pgbouncer
* Discover best practices for monitoring and troubleshooting connection pooling setups
* Gain insights into real-world use cases and considerations for production environments
This presentation is ideal for:
* Database administrators (DBAs)
* Developers working with PostgreSQL
* DevOps engineers
* Anyone interested in optimizing PostgreSQL performance
Contact info@mydbops.com for PostgreSQL Managed, Consulting and Remote DBA Services
An invited talk given by Mark Billinghurst on Research Directions for Cross Reality Interfaces. This was given on July 2nd 2024 as part of the 2024 Summer School on Cross Reality in Hagenberg, Austria (July 1st - 7th)
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Chris Swan
Have you noticed the OpenSSF Scorecard badges on the official Dart and Flutter repos? It's Google's way of showing that they care about security. Practices such as pinning dependencies, branch protection, required reviews, continuous integration tests etc. are measured to provide a score and accompanying badge.
You can do the same for your projects, and this presentation will show you how, with an emphasis on the unique challenges that come up when working with Dart and Flutter.
The session will provide a walkthrough of the steps involved in securing a first repository, and then what it takes to repeat that process across an organization with multiple repos. It will also look at the ongoing maintenance involved once scorecards have been implemented, and how aspects of that maintenance can be better automated to minimize toil.
Coordinate Systems in FME 101 - Webinar SlidesSafe Software
If you’ve ever had to analyze a map or GPS data, chances are you’ve encountered and even worked with coordinate systems. As historical data continually updates through GPS, understanding coordinate systems is increasingly crucial. However, not everyone knows why they exist or how to effectively use them for data-driven insights.
During this webinar, you’ll learn exactly what coordinate systems are and how you can use FME to maintain and transform your data’s coordinate systems in an easy-to-digest way, accurately representing the geographical space that it exists within. During this webinar, you will have the chance to:
- Enhance Your Understanding: Gain a clear overview of what coordinate systems are and their value
- Learn Practical Applications: Why we need datams and projections, plus units between coordinate systems
- Maximize with FME: Understand how FME handles coordinate systems, including a brief summary of the 3 main reprojectors
- Custom Coordinate Systems: Learn how to work with FME and coordinate systems beyond what is natively supported
- Look Ahead: Gain insights into where FME is headed with coordinate systems in the future
Don’t miss the opportunity to improve the value you receive from your coordinate system data, ultimately allowing you to streamline your data analysis and maximize your time. See you there!
1. Databases in the
Hosted Cloud
Colin Charles, MariaDB Corporation Ab
colin@mariadb.org | byte@bytebot.net
http://mariadb.com/ | http://mariadb.org/
http://bytebot.net/blog/ | @bytebot on Twitter
Percona Live Europe,Amsterdam, Netherlands
22 September 2015
1
2. whoami
• Work on MariaDB at MariaDB Corporation
(SkySQL Ab)
• Merged with Monty Program Ab, makers of
MariaDB
• Formerly MySQL AB (exit: Sun Microsystems)
• Past lives include Fedora Project (FESCO),
OpenOffice.org
• Recipient of the MySQL Community Contributor of
theYear Award 2014
2
3. Agenda
• MySQL as a service offering (DBaaS)
• Choices
• Considerations
• MySQL versions & access
• Costs
• Deeper into RDS
• Should you run this on EC2 or an equivalent?
• Conclusion
3
4. MySQL as a service
• Database as a Service (DBaaS)
• MySQL available on-demand, without any
installation/configuration of hardware/
software
• Pay-per-usage based
• Provider maintains MySQL, you don’t
maintain, upgrade, or administer the database
4
5. New way of
deployment
• Enter a credit card
number
• call API (or use the GUI)
ec2-run-instances ami-
xxx -k ${EC2_KEYPAIR} -t
m1.large
nova boot --image
centos6-x86_64 --flavor
m1.large db1
5
credit: http://www.flickr.com/photos/68751915@N05/6280507539/
6. Why DBaaS?
• “Couldn’t we just have a few more servers
to handle the traffic spike during the
elections?”
• Don’t have a lot of DBAs, optimise for
operational ease
• Rapid deployment & scale-out
6
7. Your choices today
• Amazon Web Services Relational Database
Service (RDS)
• Rackspace Cloud Databases
• Google Cloud SQL
7
9. There are more
• Jelastic - PaaS offering MySQL, MariaDB
• ClearDB - MySQL partnered with heroku,
Azure clouds
• Joyent - Image offers Percona MySQL and a
Percona SmartMachine
9
10. The new entrants
• Google Compute Engine offers Percona
XtraDB Cluster as a “click-to-deploy” app
• comes with Galera 3, Percona Toolkit,
XtraBackup as well
• Pivotal CloudFoundry - “MySQL” PaaS which
is MariaDB Galera Cluster 10
• Red Hat OpenShift - MySQL 5.1/5.5,
MariaDB 5.5
10
11. Beware
• GenieDB - globally distributed MySQL as a
service, master-master replication, works
on EC2, Rackspace, Google Compute
Engine, HP Cloud
• Xeround - 2 weeks notice...
11
13. What else do you get?
• AWS RDS has PostgreSQL — notice the
price
• Most MongoDB users run within the cloud —
but not hosted, it tends to be “roll your own”
• Compose.io (now IBM) & the like
(MongoLab) — hosted MongoDB, Redis,
Enhanced PostgreSQL, ElasticSearch,
RethinkDB
13
14. Regions & Availability
Zones
• Region: a data centre
location, containing
multiple Availability
Zones
• Availability Zone (AZ):
isolated from failures
from other AZs + low-
latency network
connectivity to other
zones in same region
14
15. Location, location,
location
• AWS RDS: US East (N.Virginia), US West
(Oregon), US West (California), EU (Ireland,
Frankfurt),APAC (Singapore,Tokyo, Sydney), South
America (São Paulo), GovCloud
• Rackspace: USA (Dallas DFW, Chicago ORD, N.
Virginia IAD),APAC (Sydney, Hong Kong), EU
(London)*
• Google Cloud SQL: US, EU,Asia
• HP Cloud: US-East (Virginia), US-West
15
16. Service Level
Agreements (SLA)
• AWS - 99.95% in a calendar month
• Rackspace - 99.9% in a calendar month
• Google - 99.95% in a calendar month
• HP Cloud - no specific DB SLA, 99.95% in a
calendar month
• SLAs exclude “scheduled maintenance” which may
storage I/O + elevate latency
• AWS is 30 minutes/week, so really 99.65%
16
17. Support
• AWS - forums; $49/mo gets email; $100+
phone #
• Rackspace - live chat, phone #, forums
• Google - forums; $150/mo gets support
portal; $400+ for phone #
• HP Cloud - phone #, chat, customer forum
17
18. Who manages this?
• AWS: self-management, Enterprise ($15k+)
• Rackspace: $100 + 0.04 cents/hr over
regular pricing
• Google: self-management
• HP Cloud: self-management
18
19. MySQL versions
• AWS: MySQL Community 5.1, 5.5, 5.6
• Rackspace: MariaDB 10, MySQL 5.6/5.1,
Percona Server 5.6
• Google: MySQL Community 5.5, 5.6
(preview)
• HP Cloud: Percona Server 5.5.28
19
20. Access methods
• AWS - within Amazon, externally via mysql client,
API access.
• Rackspace - private hostname within Rackspace
network, API access.
• Google - within AppEngine, a command line Java
tool (gcutil), standard mysql client
• HP Cloud - within HP Cloud, externally via client
(trove-cli, reddwarf),API access, mysql client
20
21. Can you configure
MySQL?
• You don’t access
my.cnf naturally
• In AWS you
have parameter
groups which
allow
configuration of
MySQL
21
source: http://www.mysqlperformanceblog.com/2013/08/21/amazon-rds-with-mysql-5-6-configuration-variables/
22. Cost
• Subscribe to relevant newsletters of your
services
• Cost changes rapidly, plus you get new
instance types and new features (IOPS)
• Don’t forget network access costs
• Monitor your costs daily, hourly if possible
(EC2 instances can have spot pricing)
22
24. Costs:AWS II
• Medium instances (3.75GB) useful for
testing ($1,577/yr [2014] vs $2,411/yr
[2013])
• Large instance (7.5GB) production ready
($3,241/yr vs $4,777/yr [2013])
• m3.2XL (30GB, 8vCPUs) ($12,964/yr)
• XL instance (15GB, 8ECUs) ($9,555/yr)
24
25. Costs: Rackspace
• Option to have regular Cloud Database or
Managed Instances
• 4GB instance (testing) is $2,102/yr (vs.
$3,504/yr in 2013)
• 8GB instance (production) is $4,205/yr (vs
$6,658/yr in 2013)
• Consider looking at I/O priority, and the
actual TPS you get
25
26. Costs: Google
• You must enable billing before you create Cloud
SQL instances
• https://developers.google.com/cloud-sql/docs/billing
• Testing (D8 - 4GB RAM) - ($4,274.15)
• XL equivalent for production (D16 - 8GB RAM) -
($8,548.30)
• Packages billing plans are cheaper than per-use
billing plans
26
27. Costs: HP Cloud
• 50% off pricing while in public beta
• 4GB RAM, 60GB storage - $1,752/yr (usual:
$3,504/yr)
• 8GB RAM, 120GB storage - $3,504/yr
(usual: $7,008/yr)
27
28. Where do you host
your application?
• Typically within the compute clusters of the
service you’re running the DBaaS in
• This also means your language choices are
limited based on what the platform offers
(eg.AppEngine only offers Java, Python, PHP,
Go)
28
29. RDS: Multi-AZ
• Provides enhanced durability (synchronous
data replication)
• Increased availability (automatic failover)
• Warning: can be slow (1-10 mins+)
• Easy GUI administration
• Doesn’t give you another usable “read-
replica” though
29
30. External replication
• MySQL 5.6 you can do RDS -> Non-RDS
• enable backup retention, you now have
binlog access
• target: exporting data out of RDS
• Replicate into RDS with 5.5.33 or later
• AWS provides stored procedures like
mysql.rds_set_external_master nowadays
30
31. MySQL 5.6, MariaDB
10
• MySQL 5.6 in RDS provides crash-safe slaves, the
InnoDB memcached interface, online schema
changes, full-text InnoDB indexes, optimizer
improvements, INFORMATION_SCHEMA
enhancements, scalability/replication
improvements, PERFORMANCE_SCHEMA
enhancements
• MariaDB 10 has much of that that, plus multi-
source replication, GTIDs that don’t require full
restarts, threadpool, audit plugin and more
31
32. Getting started
• Importing data into the cloud?
• mysqldump is a good choice today
• Upgrading from RDS 5.5 to RDS 5.6?
• mysqldump before, but nowadays you can
do this via Read Replicas
32
33. Handling backups
• You don’t get to use xtrabackup!
• Google Cloud SQL automates backups (has
a backup window - 4h)
• Amazon has automated backups (with point-
in-time recovery), with full daily snapshots
(has a backup window).
• Rackspace + HPCloud allow instance
backups too
33
34. Monitoring
• Options are limited,AWS has the best
options currently available
• Today you have CloudWatch
• Google has basic read/write graphs
• Rackspace has started with basic graphs,
visuals for MySQL coming soon, have a beta
Cloud Intelligence product
34
35. Storage Engines
• MySQL (/MariaDB) have many
• cool ones include TokuDB, SPIDER,
CONNECT
• You basically use InnoDB and MyISAM with
cloud solutions
• MyISAM on RDS won’t guarantee point-
in-time recovery, snapshot restore
35
36. High Availability
• Plan for node failures
• Don’t assume node provisioning is quick
• Backup, backup, backup!
• “Bad” nodes exist
• HA is not equal across options - RDS wins
so far
36
37. Unsupported features
• AWS: GTIDs, InnoDB Cache Warming,
InnoDB transportable tablespaces,
authentication plugins, semi-sync replication
• Google: UDFs, replication, LOAD DATA
INFILE, INSTALL PLUGIN, SELECT ... INTO
OUTFILE
37
38. Provisioned IOPS
• Only available on Amazon
• Faster, predictable, consistent I/O
performance with low latencies
• Good throughput, RAID on backed
• EBS is more reliable
38
39. More on RDS
• log access via API
• no SUPER access to skip replication errors
easily
• sync_binlog=0 not available
• no OS access (sar, ps, tcpdump)
• https://github.com/boto/boto
39
40. Warning: automatic
upgrades
• Regressions happen even with a minor
version upgrade in the MySQL world
• InnoDB update that modifies rows PK
triggers recursive behaviour until all disk
space is exceeded? 5.5.24->5.5.25 (fixed:
5.5.25a)
• Using query cache for partitioned tables?
Disabled since 5.5.22->5.5.23!
40
41. Benchmarking for use
• sysbench
• OLTP test, use tables
with 20M rows and 20M
transactions, check
1-128 threads/run (run
this on RDS, Rackspace)
• June 2013, tps,
performance per dollar,
Rackspace delivers more
performance across all
flavours except 512MB
instance
• Yahoo! Cloud Serving
Benchmark
• https://github.com/
brianfrankcooper/YCSB
• Google’s PerfKit
Benchmarker
• https://github.com/
GoogleCloudPlatform/
PerfKitBenchmarker
41
42. Roadmaps?
• There don’t seem to be public roadmaps.
You find out when there’s a change!
• Presumably HPCloud will get 5.6… and
maybe Google will get some MariaDB?
42
44. PostgreSQL in RDS
• loading data?
backup_retntion=0
• disable multi-AZ when
loading
• disable autovacuum
• dump compressed, restore
in parallel
• don’t disable fsync (really!)
• sync replication using
multi-AZ
• you can control the
upgrade time though —
this is a bonus
• Use PIOPS
• SSL should be on
44
45. Running MySQL in EC2
• Can do multiple geographic
regions via replication
• Run just one Percona Server/
MariaDB server/instance
• Use additional EBS volumes for
InnoDB tablespaces
• RAID EBS volumes (RAID1)
• Warm up data partitions, mount
partitions with noatime, nodirtime
• Vertical scaling with SSD-backed
storage
• Monitoring with nagios
• Snapshot backups and save to S3
• Can use Elastic Load Balancer
• Can use spot instances
• Can use tools like MHA to
provide automatic failover
• Can use MariaDB Galera Cluster/
Percona XtraDB Cluster
45
46. AWS Aurora
• Bigger instances work better
• Zero-downtime migration from RDS
• Metrics via CloudWatch
• Connectors via MariaDB
• 99.99% uptime
46
47. Some closing thoughts
• Hardware varies per region
• Sometimes, software manageability varies
per region
• Beware cost on your credit card!
47