The document provides an overview of the Complete MariaDB Server Tutorial presentation. It introduces MariaDB and discusses what it is, its goals of being compatible with MySQL and having stable releases. It also covers MariaDB architecture, installation, utilities, and storage engines.
An introduction to MongoDB from an experienced MySQL user and developer. There are differences and we go thru the What/Why/Who/Where of MongoDB, the "similarities" to the MySQL world like storage engines, how replication is a little more interesting with built-in sharding and automatic failover, backups, monitoring, DBaaS, going to production and finding out more resources.
MySQL features missing in MariaDB Server. Here's an overview from the New York developer's Unconference in February 2018. This is primarily aimed at the developers, to decide what goes into MariaDB 10.4, as opposed to users.
High level comparisons are made between MySQL 5.6/5.7 with of course MySQL 8.0 as well. Here's to ensuring MariaDB Server 10/310.4 has more "Drop-in" compatibility.
This document provides an overview of MariaDB 10.0 and what's new compared to previous versions. Some of the key highlights include backporting features from MySQL 5.6 such as InnoDB, Performance Schema, and online ALTER TABLE. MariaDB 10.0 also includes new features like multi-source replication, persistent statistics, and integration with NoSQL databases. The goals are to have feature parity with MySQL 5.6 and provide an open source alternative to Oracle's MySQL with more active development.
Presented at Percona Live Amsterdam 2016, this is an in-depth look at MariaDB Server right up to MariaDB Server 10.1. Learn the differences. See what's already in MySQL. And so on.
MySQL is a unique adult (now 21 years old) in many ways. It supports plugins. It supports storage engines. It is also owned by Oracle, thus birthing two branches of the popular opensource database: Percona Server and MariaDB Server. It also once spawned a fork: Drizzle. Lately a consortium of web scale users (think a chunk of the top 10 sites out there) have spawned WebScaleSQL.
You're a busy DBA having to maintain a mix of this. Or you're a CIO planning to choose one branch. How do you go about picking? Supporting multiple databases? Find out more in this talk. Also covered is a deep-dive into what feature differences exist between MySQL/Percona Server/MariaDB/WebScaleSQL, how distributions package the various databases differently. Within the hour, you'll be informed about the past, the present, and hopefully be knowledgeable enough to know what to pick in the future.
Note, there will also be coverage of the various trees around WebScaleSQL, like the Facebook tree, the Alibaba tree as well as the Twitter tree.
Some best practices about tuning Linux for your database workloads. The focus is not just on MySQL or MariaDB Server but also on understanding the OS from hardware/cloud, I/O, filesystems, memory, CPU, network, and resources.
MariaDB is a community-developed fork of MySQL that aims to be a drop-in replacement. It focuses on being compatible, stable with no regressions, and feature-enhanced compared to MySQL. The presentation covered MariaDB's architecture including connections, query caching, storage engines, and tools for administration and development like mysql, mysqldump, and EXPLAIN.
This is my third iteration of the talk presented in Tokyo, Japan - first was at a keynote at rootconf.in in April 2016, then at the MySQL meetup in New York, and now for dbtechshowcase. The focus is on database failures of the past, and how modern MySQL / MariaDB Server technologies could have helped them avoid such failure. The focus is on backups and verification, replication and failover, and security and encryption.
Meet MariaDB Server 10.1 London MySQL meetup December 2015
Meet MariaDB Server 10.1, the server that got released recently. Presented at the London MySQL Meetup in December 2015. Learn about the new features in MariaDB Server, especially around the focus of what we did to improve security.
A presentation about how to make MySQL highly available, presented at the San Francisco MySQL Meetup (http://www.sfmysql.org/events/15760472/) on January 26th, 2011.
A video recording of this presentation is available from Ustream: http://ustre.am/fyLk
MariaDB 10: A MySQL Replacement. Current up to 10.0.9, right before the 10.0.10 GA release presented the weekend before the release in Hong Kong, at the Hong Kong Open Source Conference.
Having spent more than the last decade being the main point of contact for distributions shipping MySQL, then MariaDB Server, it's clear that working with distributions have many challenges. Licensing changes (when MySQL moved the client libraries from LGPL to GPL with a FOSS Exception), ABI changes, speed (or lack thereof) of distribution releases/freezes, supporting the software throughout the lifespan of the distribution, specific bugs due to platforms, and a lot more will be discussed in this talk. Let's not forget the politics. How do we decide "tiers" of importance for distributions? As a bonus, there will be a focus on how much effort it took to "replace" MySQL with MariaDB.
Benefits: if you're making a distribution, this is the point of view of the upstream package makers. Why are distribution statistics important to us? Do we monitor your bugs system or do you have a better escalation to us? How do we test to make sure things are going well before release. This and more will be spoken about.
As an upstream project (package), we love nothing more than being available everywhere. But time and energy goes into making this is so as there are quirks in every distribution.
Better encryption & security with MariaDB 10.1 & MySQL 5.7
Talking about the improvements in MariaDB on MySQL security and encryption features that are so important in today's data landscape. Presented http://www.meetup.com/EffectiveMySQL/events/224828891/
Choosing a MySQL High Availability solution - Percona Live UK 2011
Henrik Ingo presented on choosing a MySQL replication and high availability solution. He discussed various options like MySQL replication, Tungsten, Galera, DRBD, and NDB. Galera provides synchronous multi-master replication with no single point of failure and supports InnoDB with good performance based on benchmarks. It offers simplicity over more complex clustering frameworks. The presentation evaluated solutions based on usability, performance, and support for features like asynchronous replication and global transaction IDs.
MariaDB is a community developed branch of MySQL that is feature enhanced and backward compatible. It aims to be a 100% drop-in replacement for MySQL that is stable, bug-free, and released under the GPLv2 license. Major releases of MariaDB include new storage engines like XtraDB and Aria, as well as new features for performance, scalability, and compatibility. MariaDB is developed as an open source project and supported by Monty Program and other community contributors and service providers.
The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScale
This document discusses MySQL proxy technologies including MySQL Router, ProxySQL, and MariaDB MaxScale. It provides an overview of each technology, including when they were released, key features, and comparisons between them. ProxySQL is highlighted as a popular option currently with integration with Percona tools, while MySQL Router may become more widely used due to its support for MySQL InnoDB Cluster. MariaDB MaxScale is noted for its binlog routing capabilities. Overall the document aims to help people understand and choose between the different MySQL proxy options.
MariaDB - a MySQL replacement at South East Linux Fest 2014 - SELF2014. Learn about features that are not in MySQL 5.6, some that are only just coming in MySQL 5.7, and some that just don't exist.
(DAT209) NEW LAUNCH! Introducing MariaDB on Amazon RDS
We are excited to announce the immediate availability of MariaDB on Amazon RDS. You can now run your MariaDB database on AWS while taking advantage of RDS management features like automated backups, point-in-time recovery, cross-region replication, and multi-AZ deployments for high availability. In this session, you learn about how to leverage RDS to get the most out of your MariaDB database. Steven Grandchamp, Vice President and GM at MariaDB, is a participant in this session.
The document provides configuration instructions and guidelines for setting up streaming replication between a PostgreSQL master and standby server, including setting parameter values for wal_level, max_wal_senders, wal_keep_segments, creating a dedicated replication role, using pg_basebackup to initialize the standby, and various recovery target options to control the standby's behavior. It also discusses synchronous replication using replication slots and monitoring the replication process on both the master and standby servers.
I gave that talk at https://fosdem.org/2016/ on January, 31st 2015 in the PostgreSQL developer room.
The talk covert PostgreSQL JSON features and included new functions and operators introduced in 9.5.
The SQL statements are available on GiHub at https://github.com/sjstoelting/talks/tree/master/json-by-example
Webinar slides: Replication Topology Changes for MySQL and MariaDB
This document discusses replication topology changes for MySQL and MariaDB databases. It covers making changes using GTID or regular replication, the failover process, and tools like MaxScale and ProxySQL that can help automate query rerouting during a failover. Specific topics covered include reslaving nodes, setting up master-master replication, and performing both offline and online failovers.
This presentation reviews the top ten new features that will appear in the Postgres 9.5 release.
Postgres 9.5 adds many features designed to enhance the productivity of developers: UPSERT, CUBE, ROLLUP, JSONB functions, and PostGIS improvements. For administrators, it has row-level security, a new index type, and performance enhancements for large servers.
Performance improvements in PostgreSQL 9.5 and beyond
This document discusses several performance improvements made in PostgreSQL versions 9.5 and beyond. Some key improvements discussed include:
- Faster sorting through allowing sorting by inlined functions, abbreviated keys for VARCHAR/TEXT/NUMERIC, and Sort Support benefits.
- Improved hash joins through reduced palloc overhead, smaller NTUP_PER_BUCKET, and dynamically resizing the hash table.
- Index improvements like avoiding index tuple copying, GiST and bitmap index scan optimizations, and block range tracking in BRIN indexes.
- Aggregate functions see speedups through using 128-bit integers for internal state instead of NUMERIC in some cases.
- Other optimizations affect PL/pgSQL performance,
Deep Dive Into How To Monitor MySQL or MariaDB Galera Cluster / Percona XtraD...
MySQL provides hundreds of status counters, but how do you make sense of all that monitoring data?
If you’re in Operations and your job is to monitor the health of MySQL/MariaDB Galera Cluster or Percona XtraDB Cluster, then this webinar is for you. Setting up a Galera Cluster is fairly straightforward, but keeping it in a good shape and knowing what to look for when it’s having production issues can be a challenge.
Status counters can be tricky to read …
Which of them are more important than others?
How do you find your way in a labyrinth of different variables?
Which of them can make a significant difference?
How might a host’s health impact MySQL performance?
How to identify problematic nodes in your cluster?
To find out more, read these webinar slides (or watch the replay).
Our colleague Krzysztof Książek provided a deep-dive session on what to monitor in Galera Cluster for MySQL & MariaDB. Krzysztof is a MySQL DBA with experience in managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
Amongst other things, Krzysztof discussed why having a good monitoring system is a must, covering the following topics:
Galera monitoring
• cluster status
• flow control
Host metrics and their impact on MySQL
• CPU
• memory
• I/O
InnoDB metrics
• CPU-related
• I/O-related
MySQL Cluster powers the subscriber databases of major communication services providers as well as next generation web, cloud, social and mobile applications. It is designed to deliver:
- Real-time, in-memory performance for both OLTP and analytics workloads
- Linear scale-out for both reads and writes
99.999% High Availability
- Transparent, cross-shard transactions and joins
- Update-Anywhere Geographic replication
- SQL or native NoSQL APIs
All that while still providing full ACID transactions.
This document summarizes MariaDB 10.0 and what's new in the project. It provides an overview of MariaDB's history and goals of being compatible with MySQL. Key features of MariaDB 10.0 include backported features from MySQL 5.6, new features like multi-source replication, and engines for Cassandra and LevelDB. The roadmap is to have parity with MySQL 5.6 by MariaDB 10.1 while continuing to enhance and expand the feature set. Community involvement and the new MariaDB Foundation are discussed.
This document provides an overview of MariaDB 10 and the MariaDB Foundation. It discusses the history and development of MariaDB, including key features added in versions 5.1 through 10.0 such as new storage engines, performance improvements, and features backported from MySQL. It outlines the goals of MariaDB to be compatible with MySQL while adding new features, and describes the community-led development model. The roadmap aims to have MariaDB be a drop-in replacement for MySQL 5.6 by releasing version 10.1.
MariaDB - Fast, Easy & Strong - Get Started Tutorial
MariaDB - Fast, Easy & Strong - Get Started Guide. You can understand why you should use MariaDB and how easy to install it for your server. Let 's enjoy!!!
The document summarizes the history and current state of the MySQL database server ecosystem. It discusses the origins and development of MySQL, MariaDB, Percona Server, and other related projects. It also describes some of the key features and innovations in recent versions of these database servers. The ecosystem is very active with contributions from many organizations and the future remains promising with ongoing work.
The MySQL ecosystem - understanding it, not running away from it!
You're a busy DBA thinking about having to maintain a mix of this. Or you're a CIO planning to choose one branch over another. How do you go about picking? Supporting multiple databases? Find out more in this talk. Also covered is a deep-dive into what feature differences exist between MySQL/Percona Server/MariaDB Server. Within 20 minutes, you'll leave informed and knowledgable on what to pick.
A base blog post to get started: https://www.percona.com/blog/2017/11/02/mysql-vs-mariadb-reality-check/
MariaDB is a community developed fork of MySQL that is feature enhanced and backward compatible. It aims to be a 100% drop-in replacement for MySQL. Recent versions have included storage engines like Percona XtraDB and PrimeBase PBXT, as well as new features like pluggable authentication, virtual columns, and an improved query optimizer. The project is open source and community developed by the MariaDB Foundation and its partners. Future plans include a focus on InnoDB and replication improvements.
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)
MariaDB is like the "new" MySQL, and its available everywhere. This talk was given at LinuxCon Europe in Dublin in October 2015. Learn about all the new features, considering the release was just around the corner. Changes in replication are also very interesting
MySQL and MariaDB are becoming more divergent. Learn what is different from a high level. It is also a good idea to ensure that you use the correct database for the correct job.
At the MariaDB Server Developer's meeting in Amsterdam, Oct 8 2016. This was the deck to talk about what MariaDB Server 10.1/10.2 might be missing from MySQL versions up to 5.7. The focus is on compatibility of MariaDB Server with MySQL.
- MariaDB is a community-developed fork of MySQL that is fully compatible and intended as a drop-in replacement. It aims to be stable, high-performance, and feature-enhanced compared to MySQL.
- Major new features in MariaDB 5.1 include storage engines like XtraDB and PBXT, extended slow query log statistics, and bug fixes. MariaDB 5.2 includes virtual columns, pluggable authentication, and optimizations.
- The MariaDB community is open source and welcomes contributions beyond just coding, such as writing documentation, testing, and evangelism.
OSDC 2016 - Tuning Linux for your Database by Colin Charles
Many operations folk know that performance varies depending on using one of the many Linux filesystems like EXT4 or XFS. They also know of the schedulers available, they see the OOM killer coming and more. However, appropriate configuration is necessary when you're running your databases at scale.
Learn best practices for Linux performance tuning for MariaDB/MySQL (where MyISAM uses the operating system cache, and InnoDB maintains its own aggressive buffer pool), as well as PostgreSQL and MongoDB (more dependent on the operating system). Topics that will be covered include: filesystems, swap and memory management, I/O scheduler settings, using and understanding the tools available (like iostat/vmstat/etc), practical kernel configuration, profiling your database, and using RAID and LVM.
There is a focus on bare metal as well as configuring your cloud instances in.
Learn from practical examples from the trenches.
The document provides an overview of the evolution of open source databases from the past to present and future. It discusses the early days of navigational and hierarchical databases. It then covers the development of relational databases and SQL. It outlines the rise of open source databases like MySQL, PostgreSQL, and SQLite. It also summarizes the emergence of NoSQL databases and NewSQL systems to handle big data and cloud computing. The document predicts continued development and blending of features between SQL, NoSQL, and NewSQL databases.
OSDC 2018 | Scaling & High Availability MySQL learnings from the past decade+...
The MySQL world is full of tradeoffs and choosing a High Availability (HA) solution is no exception. This session aims to look at all of the alternatives in an unbiased nature. While the landscape will be covered, including but not limited to MySQL replication, MHA, DRBD, Galera Cluster, etc. the focus of the talk will be what is recommended for today, and what to look out for. Thus, this will include extensive deep-dive coverage of ProxySQL, semi-sync replication, Orchestrator, MySQL Router, and Galera Cluster variants like Percona XtraDB Cluster and MariaDB Galera Cluster. I will also touch on group replication.
Learn how we do this for our nearly 4000+ customers!
MariaDB is an open-source relational database management system that was created to be more open and community-focused than its predecessor, MySQL. It was founded in 2009 by the original developers of MySQL after Oracle acquired Sun Microsystems. MariaDB aims to preserve the open nature of MySQL by using an open governance model and keeping its code open source under GPL. It has become the default database in several major Linux distributions and is available on major cloud platforms. MariaDB provides an enterprise-grade database with high availability, performance, scalability and security features.
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdf
MySQL is still hot, with Percona XtraDB Cluster (PXC) and MariaDB Server. Welcome back post-pandemic to see what is on offer in the current ecosystem.
Did you know that Amazon RDS now uses semi-sync replication rather than DRBD for multi-AZ deployments? Did you know that Galera Cluster for MySQL 8 is much more efficient with CLONE SST rather than using the xtrabackup method for SST? Did you know that Percona Server continues to extend MyRocks? Did you know that MariaDB Server has more Oracle syntax compatibility? This and more will be covered in the session, while short and quick, should leave you wandering to discover new features for production.
Colin Charles presented on running MySQL in the hosted cloud. He discussed various database as a service (DBaaS) options like Amazon RDS, Rackspace, and Google Cloud SQL. Key considerations for DBaaS include location, service level agreements, support options, available MySQL/MariaDB versions, access methods, configuration options, costs, and features like high availability and backups. Running MySQL on EC2 is also an option but requires more management of hardware, software, networking, storage and backups. Benchmarking and monitoring tools were recommended to evaluate performance and usage.
This document summarizes MySQL Fabric, an open-source tool from Oracle for sharding and managing high availability of MySQL databases at scale. It discusses how MySQL Fabric uses server groups, a fabric node, and fabric-aware connectors to partition data across shards and handle failures while providing a global view of the database schema and tables. Key features of MySQL Fabric mentioned include automated shard splitting and movement, minimized downtime during failures, and use of XML-RPC to avoid proxy hops between applications and shards.
You want to use MySQL in Amazon RDS, Rackspace Cloud, Google Cloud SQL or HP Helion Public Cloud? Check this out, from Percona Live London 2014. (Note that pricing of Google Cloud SQL changed prices on the same day after the presentation)
MariaDB Server 10.3 is a culmination of features from MariaDB Server 10.2+10.1+10.0+5.5+5.3+5.2+5.1 as well as a base branch from MySQL 5.5 and backports from MySQL 5.6/5.7. It has many new features, like a GA-ready sharding engine (SPIDER), MyRocks, as well as some Oracle compatibility, system versioned tables and a whole lot more.
Presented at OSCON 2018. A review of what is available from MySQL, MariaDB Server, MongoDB, PostgreSQL, and more. Covering your choices, considerations, versions, access methods, cost, a deeper look at RDS and if you should run your own instances or not.
With a focus on Amazon AWS RDS MySQL and PostgreSQL, Rackspace cloud, Google Cloud SQL, Microsoft Azure for MySQL and PostgreSQL as well as a hint of the other clouds
Engineering that goes into making Percona Server for MySQL 5.6 & 5.7 different (and a hint of MongoDB) for dbtechshowcase 2017 - the slides also have some Japanese in it. This should help a Japanese audience to read it. If there are questions due to poor translation, do not hesitate to drop me an email (byte@bytebot.net) or tweet: @bytebot
Databases require capacity planning (and to those coming from traditional RDBMS solutions, this can be thought of as a sizing guide). Capacity planning prevents resource exhaustion. Capacity planning can be hard. This talk has a heavier leaning on MySQL, but the concepts and addendum will help with any other data store.
Lessons from {distributed,remote,virtual} communities and companies
A last minute talk for the people at DevOps Amsterdam, happening around the same time as O'Reilly Velocity Amsterdam 2016. Here are lessons one can learn from distributed/remote/virtual communities and companies from someone that has spent a long time being remote and distributed.
Forking Successfully or do you think a branch will work better? Learn from history, see what's current, etc. Presented at OSCON London 2016. This is forking beyond the github generation. And if you're going to do it, some tips on how you could be successful.
Co-presented alongside Ronald Bradford, this covers MySQL, Percona Server, and MariaDB Server (since the latter occasionally can be different enough). Go thru insecure practices, focus on communication security, connection security, data security, user accounts and server access security.
This was a short 25 minute talk, but we go into a bit of a history of MySQL, how the branches and forks appeared, what's sticking around today (branch? Percona Server. Fork? MariaDB Server). What should you use? Think about what you need today and what the roadmap holds.
Failure happens, and we can learn from it. We need to think about backups, but also verification of them. We should definitely make use of replication and think about automatic failover. And security is key, but don't forget that encryption is now available in MySQL, Percona Server and MariaDB Server.
Presented at the MySQL Chicago Meetup in August 2016. The focus of the talk is on backups and verification, replication and failover, as well as security and encryption.
This document discusses MariaDB plugins and provides examples of several useful plugins, including authentication plugins, password validation plugins, SQL error logging, audit logging, query analysis, and more. It encourages contributing plugins to help extend MariaDB's functionality.
If you’ve ever had to analyze a map or GPS data, chances are you’ve encountered and even worked with coordinate systems. As historical data continually updates through GPS, understanding coordinate systems is increasingly crucial. However, not everyone knows why they exist or how to effectively use them for data-driven insights.
During this webinar, you’ll learn exactly what coordinate systems are and how you can use FME to maintain and transform your data’s coordinate systems in an easy-to-digest way, accurately representing the geographical space that it exists within. During this webinar, you will have the chance to:
- Enhance Your Understanding: Gain a clear overview of what coordinate systems are and their value
- Learn Practical Applications: Why we need datams and projections, plus units between coordinate systems
- Maximize with FME: Understand how FME handles coordinate systems, including a brief summary of the 3 main reprojectors
- Custom Coordinate Systems: Learn how to work with FME and coordinate systems beyond what is natively supported
- Look Ahead: Gain insights into where FME is headed with coordinate systems in the future
Don’t miss the opportunity to improve the value you receive from your coordinate system data, ultimately allowing you to streamline your data analysis and maximize your time. See you there!
Choose our Linux Web Hosting for a seamless and successful online presence
Our Linux Web Hosting plans offer unbeatable performance, security, and scalability, ensuring your website runs smoothly and efficiently.
Visit- https://onliveserver.com/linux-web-hosting/
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
Your comprehensive guide to RPA in healthcare for 2024. Explore the benefits, use cases, and emerging trends of robotic process automation. Understand the challenges and prepare for the future of healthcare automation
These fighter aircraft have uses outside of traditional combat situations. They are essential in defending India's territorial integrity, averting dangers, and delivering aid to those in need during natural calamities. Additionally, the IAF improves its interoperability and fortifies international military alliances by working together and conducting joint exercises with other air forces.
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
This presentation, delivered at the Postgres Bangalore (PGBLR) Meetup-2 on June 29th, 2024, dives deep into connection pooling for PostgreSQL databases. Aakash M, a PostgreSQL Tech Lead at Mydbops, explores the challenges of managing numerous connections and explains how connection pooling optimizes performance and resource utilization.
Key Takeaways:
* Understand why connection pooling is essential for high-traffic applications
* Explore various connection poolers available for PostgreSQL, including pgbouncer
* Learn the configuration options and functionalities of pgbouncer
* Discover best practices for monitoring and troubleshooting connection pooling setups
* Gain insights into real-world use cases and considerations for production environments
This presentation is ideal for:
* Database administrators (DBAs)
* Developers working with PostgreSQL
* DevOps engineers
* Anyone interested in optimizing PostgreSQL performance
Contact info@mydbops.com for PostgreSQL Managed, Consulting and Remote DBA Services
Implementations of Fused Deposition Modeling in real world
The presentation showcases the diverse real-world applications of Fused Deposition Modeling (FDM) across multiple industries:
1. **Manufacturing**: FDM is utilized in manufacturing for rapid prototyping, creating custom tools and fixtures, and producing functional end-use parts. Companies leverage its cost-effectiveness and flexibility to streamline production processes.
2. **Medical**: In the medical field, FDM is used to create patient-specific anatomical models, surgical guides, and prosthetics. Its ability to produce precise and biocompatible parts supports advancements in personalized healthcare solutions.
3. **Education**: FDM plays a crucial role in education by enabling students to learn about design and engineering through hands-on 3D printing projects. It promotes innovation and practical skill development in STEM disciplines.
4. **Science**: Researchers use FDM to prototype equipment for scientific experiments, build custom laboratory tools, and create models for visualization and testing purposes. It facilitates rapid iteration and customization in scientific endeavors.
5. **Automotive**: Automotive manufacturers employ FDM for prototyping vehicle components, tooling for assembly lines, and customized parts. It speeds up the design validation process and enhances efficiency in automotive engineering.
6. **Consumer Electronics**: FDM is utilized in consumer electronics for designing and prototyping product enclosures, casings, and internal components. It enables rapid iteration and customization to meet evolving consumer demands.
7. **Robotics**: Robotics engineers leverage FDM to prototype robot parts, create lightweight and durable components, and customize robot designs for specific applications. It supports innovation and optimization in robotic systems.
8. **Aerospace**: In aerospace, FDM is used to manufacture lightweight parts, complex geometries, and prototypes of aircraft components. It contributes to cost reduction, faster production cycles, and weight savings in aerospace engineering.
9. **Architecture**: Architects utilize FDM for creating detailed architectural models, prototypes of building components, and intricate designs. It aids in visualizing concepts, testing structural integrity, and communicating design ideas effectively.
Each industry example demonstrates how FDM enhances innovation, accelerates product development, and addresses specific challenges through advanced manufacturing capabilities.
Transcript: Details of description part II: Describing images in practice - T...
This presentation explores the practical application of image description techniques. Familiar guidelines will be demonstrated in practice, and descriptions will be developed “live”! If you have learned a lot about the theory of image description techniques but want to feel more confident putting them into practice, this is the presentation for you. There will be useful, actionable information for everyone, whether you are working with authors, colleagues, alone, or leveraging AI as a collaborator.
Link to presentation recording and slides: https://bnctechforum.ca/sessions/details-of-description-part-ii-describing-images-in-practice/
Presented by BookNet Canada on June 25, 2024, with support from the Department of Canadian Heritage.
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Best Practices for Effectively Running dbt in Airflow.pdf
As a popular open-source library for analytics engineering, dbt is often used in combination with Airflow. Orchestrating and executing dbt models as DAGs ensures an additional layer of control over tasks, observability, and provides a reliable, scalable environment to run dbt models.
This webinar will cover a step-by-step guide to Cosmos, an open source package from Astronomer that helps you easily run your dbt Core projects as Airflow DAGs and Task Groups, all with just a few lines of code. We’ll walk through:
- Standard ways of running dbt (and when to utilize other methods)
- How Cosmos can be used to run and visualize your dbt projects in Airflow
- Common challenges and how to address them, including performance, dependency conflicts, and more
- How running dbt projects in Airflow helps with cost optimization
Webinar given on 9 July 2024
The integration of programming into civil engineering is transforming the industry. We can design complex infrastructure projects and analyse large datasets. Imagine revolutionizing the way we build our cities and infrastructure, all by the power of coding. Programming skills are no longer just a bonus—they’re a game changer in this era.
Technology is revolutionizing civil engineering by integrating advanced tools and techniques. Programming allows for the automation of repetitive tasks, enhancing the accuracy of designs, simulations, and analyses. With the advent of artificial intelligence and machine learning, engineers can now predict structural behaviors under various conditions, optimize material usage, and improve project planning.
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
Six months into 2024, and it is clear the privacy ecosystem takes no days off!! Regulators continue to implement and enforce new regulations, businesses strive to meet requirements, and technology advances like AI have privacy professionals scratching their heads about managing risk.
What can we learn about the first six months of data privacy trends and events in 2024? How should this inform your privacy program management for the rest of the year?
Join TrustArc, Goodwin, and Snyk privacy experts as they discuss the changes we’ve seen in the first half of 2024 and gain insight into the concrete, actionable steps you can take to up-level your privacy program in the second half of the year.
This webinar will review:
- Key changes to privacy regulations in 2024
- Key themes in privacy and data governance in 2024
- How to maximize your privacy program in the second half of 2024
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...
Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, Philip Torr "A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models" arXiv2023
https://arxiv.org/abs/2307.12980
How Social Media Hackers Help You to See Your Wife's Message.pdf
In the modern digital era, social media platforms have become integral to our daily lives. These platforms, including Facebook, Instagram, WhatsApp, and Snapchat, offer countless ways to connect, share, and communicate.
Details of description part II: Describing images in practice - Tech Forum 2024
This presentation explores the practical application of image description techniques. Familiar guidelines will be demonstrated in practice, and descriptions will be developed “live”! If you have learned a lot about the theory of image description techniques but want to feel more confident putting them into practice, this is the presentation for you. There will be useful, actionable information for everyone, whether you are working with authors, colleagues, alone, or leveraging AI as a collaborator.
Link to presentation recording and transcript: https://bnctechforum.ca/sessions/details-of-description-part-ii-describing-images-in-practice/
Presented by BookNet Canada on June 25, 2024, with support from the Department of Canadian Heritage.
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
Presented at Gartner Data & Analytics, London Maty 2024. BT Group has used the Neo4j Graph Database to enable impressive digital transformation programs over the last 6 years. By re-imagining their operational support systems to adopt self-serve and data lead principles they have substantially reduced the number of applications and complexity of their operations. The result has been a substantial reduction in risk and costs while improving time to value, innovation, and process automation. Join this session to hear their story, the lessons they learned along the way and how their future innovation plans include the exploration of uses of EKG + Generative AI.
Measuring the Impact of Network Latency at Twitter
Widya Salim and Victor Ma will outline the causal impact analysis, framework, and key learnings used to quantify the impact of reducing Twitter's network latency.
Quantum Communications Q&A with Gemini LLM. These are based on Shannon's Noisy channel Theorem and offers how the classical theory applies to the quantum world.
MariaDB 10 Tutorial - 13.11.11 - Percona Live LondonIvan Zoratti
This document provides an overview and summary of MariaDB 10 features presented by Ivan Zoratti. It discusses new features in MariaDB 10 like storage engines, administration improvements, and replication capabilities. The document also summarizes optimization enhancements in MariaDB 10 like the new optimizer, improved indexing techniques, and subquery optimizations. Various agenda topics are outlined for the MariaDB 10 tutorial.
MariaDB started life as a database to host the Maria storage engine in 2009. Not long after its inception, the MySQL community went through yet another change in ownership, and it was deemed that MariaDB will be a complete database branch developed to extend MySQL, but with constant merging of upstream changes.
The goal of the MariaDB project is to ensure that everyone is part of the community, including employees of the major steering companies. MariaDB also features enhanced features, some of which are common with the Percona Performance Server. Most importantly, MariaDB is a drop-in replacement and is completely backward compatible with MySQL. In 2010, MariaDB released 5.1 in February, and 5.2 in November – two major releases in a span of one calendar year is a feat that was achieved!
DBAs and developers alike will gain an introduction to MariaDB, what is different with MySQL, how to make use of the feature enhancements, and more.
An introduction to MongoDB from an experienced MySQL user and developer. There are differences and we go thru the What/Why/Who/Where of MongoDB, the "similarities" to the MySQL world like storage engines, how replication is a little more interesting with built-in sharding and automatic failover, backups, monitoring, DBaaS, going to production and finding out more resources.
MySQL features missing in MariaDB ServerColin Charles
MySQL features missing in MariaDB Server. Here's an overview from the New York developer's Unconference in February 2018. This is primarily aimed at the developers, to decide what goes into MariaDB 10.4, as opposed to users.
High level comparisons are made between MySQL 5.6/5.7 with of course MySQL 8.0 as well. Here's to ensuring MariaDB Server 10/310.4 has more "Drop-in" compatibility.
MariaDB 10 and what's new with the projectColin Charles
This document provides an overview of MariaDB 10.0 and what's new compared to previous versions. Some of the key highlights include backporting features from MySQL 5.6 such as InnoDB, Performance Schema, and online ALTER TABLE. MariaDB 10.0 also includes new features like multi-source replication, persistent statistics, and integration with NoSQL databases. The goals are to have feature parity with MySQL 5.6 and provide an open source alternative to Oracle's MySQL with more active development.
Presented at Percona Live Amsterdam 2016, this is an in-depth look at MariaDB Server right up to MariaDB Server 10.1. Learn the differences. See what's already in MySQL. And so on.
MySQL is a unique adult (now 21 years old) in many ways. It supports plugins. It supports storage engines. It is also owned by Oracle, thus birthing two branches of the popular opensource database: Percona Server and MariaDB Server. It also once spawned a fork: Drizzle. Lately a consortium of web scale users (think a chunk of the top 10 sites out there) have spawned WebScaleSQL.
You're a busy DBA having to maintain a mix of this. Or you're a CIO planning to choose one branch. How do you go about picking? Supporting multiple databases? Find out more in this talk. Also covered is a deep-dive into what feature differences exist between MySQL/Percona Server/MariaDB/WebScaleSQL, how distributions package the various databases differently. Within the hour, you'll be informed about the past, the present, and hopefully be knowledgeable enough to know what to pick in the future.
Note, there will also be coverage of the various trees around WebScaleSQL, like the Facebook tree, the Alibaba tree as well as the Twitter tree.
Tuning Linux for your database FLOSSUK 2016Colin Charles
Some best practices about tuning Linux for your database workloads. The focus is not just on MySQL or MariaDB Server but also on understanding the OS from hardware/cloud, I/O, filesystems, memory, CPU, network, and resources.
MariaDB: in-depth (hands on training in Seoul)Colin Charles
MariaDB is a community-developed fork of MySQL that aims to be a drop-in replacement. It focuses on being compatible, stable with no regressions, and feature-enhanced compared to MySQL. The presentation covered MariaDB's architecture including connections, query caching, storage engines, and tools for administration and development like mysql, mysqldump, and EXPLAIN.
This is my third iteration of the talk presented in Tokyo, Japan - first was at a keynote at rootconf.in in April 2016, then at the MySQL meetup in New York, and now for dbtechshowcase. The focus is on database failures of the past, and how modern MySQL / MariaDB Server technologies could have helped them avoid such failure. The focus is on backups and verification, replication and failover, and security and encryption.
Meet MariaDB Server 10.1 London MySQL meetup December 2015Colin Charles
Meet MariaDB Server 10.1, the server that got released recently. Presented at the London MySQL Meetup in December 2015. Learn about the new features in MariaDB Server, especially around the focus of what we did to improve security.
A presentation about how to make MySQL highly available, presented at the San Francisco MySQL Meetup (http://www.sfmysql.org/events/15760472/) on January 26th, 2011.
A video recording of this presentation is available from Ustream: http://ustre.am/fyLk
MariaDB 10: A MySQL Replacement - HKOSC Colin Charles
MariaDB 10: A MySQL Replacement. Current up to 10.0.9, right before the 10.0.10 GA release presented the weekend before the release in Hong Kong, at the Hong Kong Open Source Conference.
Having spent more than the last decade being the main point of contact for distributions shipping MySQL, then MariaDB Server, it's clear that working with distributions have many challenges. Licensing changes (when MySQL moved the client libraries from LGPL to GPL with a FOSS Exception), ABI changes, speed (or lack thereof) of distribution releases/freezes, supporting the software throughout the lifespan of the distribution, specific bugs due to platforms, and a lot more will be discussed in this talk. Let's not forget the politics. How do we decide "tiers" of importance for distributions? As a bonus, there will be a focus on how much effort it took to "replace" MySQL with MariaDB.
Benefits: if you're making a distribution, this is the point of view of the upstream package makers. Why are distribution statistics important to us? Do we monitor your bugs system or do you have a better escalation to us? How do we test to make sure things are going well before release. This and more will be spoken about.
As an upstream project (package), we love nothing more than being available everywhere. But time and energy goes into making this is so as there are quirks in every distribution.
Better encryption & security with MariaDB 10.1 & MySQL 5.7Colin Charles
Talking about the improvements in MariaDB on MySQL security and encryption features that are so important in today's data landscape. Presented http://www.meetup.com/EffectiveMySQL/events/224828891/
Choosing a MySQL High Availability solution - Percona Live UK 2011Henrik Ingo
Henrik Ingo presented on choosing a MySQL replication and high availability solution. He discussed various options like MySQL replication, Tungsten, Galera, DRBD, and NDB. Galera provides synchronous multi-master replication with no single point of failure and supports InnoDB with good performance based on benchmarks. It offers simplicity over more complex clustering frameworks. The presentation evaluated solutions based on usability, performance, and support for features like asynchronous replication and global transaction IDs.
MariaDB is a community developed branch of MySQL that is feature enhanced and backward compatible. It aims to be a 100% drop-in replacement for MySQL that is stable, bug-free, and released under the GPLv2 license. Major releases of MariaDB include new storage engines like XtraDB and Aria, as well as new features for performance, scalability, and compatibility. MariaDB is developed as an open source project and supported by Monty Program and other community contributors and service providers.
The Proxy Wars - MySQL Router, ProxySQL, MariaDB MaxScaleColin Charles
This document discusses MySQL proxy technologies including MySQL Router, ProxySQL, and MariaDB MaxScale. It provides an overview of each technology, including when they were released, key features, and comparisons between them. ProxySQL is highlighted as a popular option currently with integration with Percona tools, while MySQL Router may become more widely used due to its support for MySQL InnoDB Cluster. MariaDB MaxScale is noted for its binlog routing capabilities. Overall the document aims to help people understand and choose between the different MySQL proxy options.
MariaDB - a MySQL Replacement #SELF2014Colin Charles
MariaDB - a MySQL replacement at South East Linux Fest 2014 - SELF2014. Learn about features that are not in MySQL 5.6, some that are only just coming in MySQL 5.7, and some that just don't exist.
We are excited to announce the immediate availability of MariaDB on Amazon RDS. You can now run your MariaDB database on AWS while taking advantage of RDS management features like automated backups, point-in-time recovery, cross-region replication, and multi-AZ deployments for high availability. In this session, you learn about how to leverage RDS to get the most out of your MariaDB database. Steven Grandchamp, Vice President and GM at MariaDB, is a participant in this session.
The document provides configuration instructions and guidelines for setting up streaming replication between a PostgreSQL master and standby server, including setting parameter values for wal_level, max_wal_senders, wal_keep_segments, creating a dedicated replication role, using pg_basebackup to initialize the standby, and various recovery target options to control the standby's behavior. It also discusses synchronous replication using replication slots and monitoring the replication process on both the master and standby servers.
I gave that talk at https://fosdem.org/2016/ on January, 31st 2015 in the PostgreSQL developer room.
The talk covert PostgreSQL JSON features and included new functions and operators introduced in 9.5.
The SQL statements are available on GiHub at https://github.com/sjstoelting/talks/tree/master/json-by-example
Webinar slides: Replication Topology Changes for MySQL and MariaDBSeveralnines
This document discusses replication topology changes for MySQL and MariaDB databases. It covers making changes using GTID or regular replication, the failover process, and tools like MaxScale and ProxySQL that can help automate query rerouting during a failover. Specific topics covered include reslaving nodes, setting up master-master replication, and performing both offline and online failovers.
This presentation reviews the top ten new features that will appear in the Postgres 9.5 release.
Postgres 9.5 adds many features designed to enhance the productivity of developers: UPSERT, CUBE, ROLLUP, JSONB functions, and PostGIS improvements. For administrators, it has row-level security, a new index type, and performance enhancements for large servers.
Performance improvements in PostgreSQL 9.5 and beyondTomas Vondra
This document discusses several performance improvements made in PostgreSQL versions 9.5 and beyond. Some key improvements discussed include:
- Faster sorting through allowing sorting by inlined functions, abbreviated keys for VARCHAR/TEXT/NUMERIC, and Sort Support benefits.
- Improved hash joins through reduced palloc overhead, smaller NTUP_PER_BUCKET, and dynamically resizing the hash table.
- Index improvements like avoiding index tuple copying, GiST and bitmap index scan optimizations, and block range tracking in BRIN indexes.
- Aggregate functions see speedups through using 128-bit integers for internal state instead of NUMERIC in some cases.
- Other optimizations affect PL/pgSQL performance,
Deep Dive Into How To Monitor MySQL or MariaDB Galera Cluster / Percona XtraD...Severalnines
MySQL provides hundreds of status counters, but how do you make sense of all that monitoring data?
If you’re in Operations and your job is to monitor the health of MySQL/MariaDB Galera Cluster or Percona XtraDB Cluster, then this webinar is for you. Setting up a Galera Cluster is fairly straightforward, but keeping it in a good shape and knowing what to look for when it’s having production issues can be a challenge.
Status counters can be tricky to read …
Which of them are more important than others?
How do you find your way in a labyrinth of different variables?
Which of them can make a significant difference?
How might a host’s health impact MySQL performance?
How to identify problematic nodes in your cluster?
To find out more, read these webinar slides (or watch the replay).
Our colleague Krzysztof Książek provided a deep-dive session on what to monitor in Galera Cluster for MySQL & MariaDB. Krzysztof is a MySQL DBA with experience in managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
Amongst other things, Krzysztof discussed why having a good monitoring system is a must, covering the following topics:
Galera monitoring
• cluster status
• flow control
Host metrics and their impact on MySQL
• CPU
• memory
• I/O
InnoDB metrics
• CPU-related
• I/O-related
What's new in MySQL Cluster 7.4 webinar chartsAndrew Morgan
MySQL Cluster powers the subscriber databases of major communication services providers as well as next generation web, cloud, social and mobile applications. It is designed to deliver:
- Real-time, in-memory performance for both OLTP and analytics workloads
- Linear scale-out for both reads and writes
99.999% High Availability
- Transparent, cross-shard transactions and joins
- Update-Anywhere Geographic replication
- SQL or native NoSQL APIs
All that while still providing full ACID transactions.
This document summarizes MariaDB 10.0 and what's new in the project. It provides an overview of MariaDB's history and goals of being compatible with MySQL. Key features of MariaDB 10.0 include backported features from MySQL 5.6, new features like multi-source replication, and engines for Cassandra and LevelDB. The roadmap is to have parity with MySQL 5.6 by MariaDB 10.1 while continuing to enhance and expand the feature set. Community involvement and the new MariaDB Foundation are discussed.
Maria db 10 and the mariadb foundation(colin)kayokogoto
This document provides an overview of MariaDB 10 and the MariaDB Foundation. It discusses the history and development of MariaDB, including key features added in versions 5.1 through 10.0 such as new storage engines, performance improvements, and features backported from MySQL. It outlines the goals of MariaDB to be compatible with MySQL while adding new features, and describes the community-led development model. The roadmap aims to have MariaDB be a drop-in replacement for MySQL 5.6 by releasing version 10.1.
MariaDB - Fast, Easy & Strong - Get Started Tutorialphamhphuc
MariaDB - Fast, Easy & Strong - Get Started Guide. You can understand why you should use MariaDB and how easy to install it for your server. Let 's enjoy!!!
The document summarizes the history and current state of the MySQL database server ecosystem. It discusses the origins and development of MySQL, MariaDB, Percona Server, and other related projects. It also describes some of the key features and innovations in recent versions of these database servers. The ecosystem is very active with contributions from many organizations and the future remains promising with ongoing work.
The MySQL ecosystem - understanding it, not running away from it! Colin Charles
You're a busy DBA thinking about having to maintain a mix of this. Or you're a CIO planning to choose one branch over another. How do you go about picking? Supporting multiple databases? Find out more in this talk. Also covered is a deep-dive into what feature differences exist between MySQL/Percona Server/MariaDB Server. Within 20 minutes, you'll leave informed and knowledgable on what to pick.
A base blog post to get started: https://www.percona.com/blog/2017/11/02/mysql-vs-mariadb-reality-check/
MariaDB is a community developed fork of MySQL that is feature enhanced and backward compatible. It aims to be a 100% drop-in replacement for MySQL. Recent versions have included storage engines like Percona XtraDB and PrimeBase PBXT, as well as new features like pluggable authentication, virtual columns, and an improved query optimizer. The project is open source and community developed by the MariaDB Foundation and its partners. Future plans include a focus on InnoDB and replication improvements.
MariaDB - the "new" MySQL is 5 years old and everywhere (LinuxCon Europe 2015)Colin Charles
MariaDB is like the "new" MySQL, and its available everywhere. This talk was given at LinuxCon Europe in Dublin in October 2015. Learn about all the new features, considering the release was just around the corner. Changes in replication are also very interesting
Differences between MariaDB 10.3 & MySQL 8.0Colin Charles
MySQL and MariaDB are becoming more divergent. Learn what is different from a high level. It is also a good idea to ensure that you use the correct database for the correct job.
MariaDB Server Compatibility with MySQLColin Charles
At the MariaDB Server Developer's meeting in Amsterdam, Oct 8 2016. This was the deck to talk about what MariaDB Server 10.1/10.2 might be missing from MySQL versions up to 5.7. The focus is on compatibility of MariaDB Server with MySQL.
- MariaDB is a community-developed fork of MySQL that is fully compatible and intended as a drop-in replacement. It aims to be stable, high-performance, and feature-enhanced compared to MySQL.
- Major new features in MariaDB 5.1 include storage engines like XtraDB and PBXT, extended slow query log statistics, and bug fixes. MariaDB 5.2 includes virtual columns, pluggable authentication, and optimizations.
- The MariaDB community is open source and welcomes contributions beyond just coding, such as writing documentation, testing, and evangelism.
OSDC 2016 - Tuning Linux for your Database by Colin CharlesNETWAYS
Many operations folk know that performance varies depending on using one of the many Linux filesystems like EXT4 or XFS. They also know of the schedulers available, they see the OOM killer coming and more. However, appropriate configuration is necessary when you're running your databases at scale.
Learn best practices for Linux performance tuning for MariaDB/MySQL (where MyISAM uses the operating system cache, and InnoDB maintains its own aggressive buffer pool), as well as PostgreSQL and MongoDB (more dependent on the operating system). Topics that will be covered include: filesystems, swap and memory management, I/O scheduler settings, using and understanding the tools available (like iostat/vmstat/etc), practical kernel configuration, profiling your database, and using RAID and LVM.
There is a focus on bare metal as well as configuring your cloud instances in.
Learn from practical examples from the trenches.
The Evolution of Open Source DatabasesIvan Zoratti
The document provides an overview of the evolution of open source databases from the past to present and future. It discusses the early days of navigational and hierarchical databases. It then covers the development of relational databases and SQL. It outlines the rise of open source databases like MySQL, PostgreSQL, and SQLite. It also summarizes the emergence of NoSQL databases and NewSQL systems to handle big data and cloud computing. The document predicts continued development and blending of features between SQL, NoSQL, and NewSQL databases.
OSDC 2018 | Scaling & High Availability MySQL learnings from the past decade+...NETWAYS
The MySQL world is full of tradeoffs and choosing a High Availability (HA) solution is no exception. This session aims to look at all of the alternatives in an unbiased nature. While the landscape will be covered, including but not limited to MySQL replication, MHA, DRBD, Galera Cluster, etc. the focus of the talk will be what is recommended for today, and what to look out for. Thus, this will include extensive deep-dive coverage of ProxySQL, semi-sync replication, Orchestrator, MySQL Router, and Galera Cluster variants like Percona XtraDB Cluster and MariaDB Galera Cluster. I will also touch on group replication.
Learn how we do this for our nearly 4000+ customers!
MariaDB is an open-source relational database management system that was created to be more open and community-focused than its predecessor, MySQL. It was founded in 2009 by the original developers of MySQL after Oracle acquired Sun Microsystems. MariaDB aims to preserve the open nature of MySQL by using an open governance model and keeping its code open source under GPL. It has become the default database in several major Linux distributions and is available on major cloud platforms. MariaDB provides an enterprise-grade database with high availability, performance, scalability and security features.
MySQL Ecosystem in 2023 - FOSSASIA'23 - Alkin.pptx.pdfAlkin Tezuysal
MySQL is still hot, with Percona XtraDB Cluster (PXC) and MariaDB Server. Welcome back post-pandemic to see what is on offer in the current ecosystem.
Did you know that Amazon RDS now uses semi-sync replication rather than DRBD for multi-AZ deployments? Did you know that Galera Cluster for MySQL 8 is much more efficient with CLONE SST rather than using the xtrabackup method for SST? Did you know that Percona Server continues to extend MyRocks? Did you know that MariaDB Server has more Oracle syntax compatibility? This and more will be covered in the session, while short and quick, should leave you wandering to discover new features for production.
MySQL in the Hosted Cloud - Percona Live 2015Colin Charles
Colin Charles presented on running MySQL in the hosted cloud. He discussed various database as a service (DBaaS) options like Amazon RDS, Rackspace, and Google Cloud SQL. Key considerations for DBaaS include location, service level agreements, support options, available MySQL/MariaDB versions, access methods, configuration options, costs, and features like high availability and backups. Running MySQL on EC2 is also an option but requires more management of hardware, software, networking, storage and backups. Benchmarking and monitoring tools were recommended to evaluate performance and usage.
This document summarizes MySQL Fabric, an open-source tool from Oracle for sharding and managing high availability of MySQL databases at scale. It discusses how MySQL Fabric uses server groups, a fabric node, and fabric-aware connectors to partition data across shards and handle failures while providing a global view of the database schema and tables. Key features of MySQL Fabric mentioned include automated shard splitting and movement, minimized downtime during failures, and use of XML-RPC to avoid proxy hops between applications and shards.
You want to use MySQL in Amazon RDS, Rackspace Cloud, Google Cloud SQL or HP Helion Public Cloud? Check this out, from Percona Live London 2014. (Note that pricing of Google Cloud SQL changed prices on the same day after the presentation)
Similar to The Complete MariaDB Server Tutorial - Percona Live 2015 (20)
MariaDB Server 10.3 is a culmination of features from MariaDB Server 10.2+10.1+10.0+5.5+5.3+5.2+5.1 as well as a base branch from MySQL 5.5 and backports from MySQL 5.6/5.7. It has many new features, like a GA-ready sharding engine (SPIDER), MyRocks, as well as some Oracle compatibility, system versioned tables and a whole lot more.
Presented at OSCON 2018. A review of what is available from MySQL, MariaDB Server, MongoDB, PostgreSQL, and more. Covering your choices, considerations, versions, access methods, cost, a deeper look at RDS and if you should run your own instances or not.
With a focus on Amazon AWS RDS MySQL and PostgreSQL, Rackspace cloud, Google Cloud SQL, Microsoft Azure for MySQL and PostgreSQL as well as a hint of the other clouds
Percona ServerをMySQL 5.6と5.7用に作るエンジニアリング(そしてMongoDBのヒント)Colin Charles
Engineering that goes into making Percona Server for MySQL 5.6 & 5.7 different (and a hint of MongoDB) for dbtechshowcase 2017 - the slides also have some Japanese in it. This should help a Japanese audience to read it. If there are questions due to poor translation, do not hesitate to drop me an email (byte@bytebot.net) or tweet: @bytebot
Databases require capacity planning (and to those coming from traditional RDBMS solutions, this can be thought of as a sizing guide). Capacity planning prevents resource exhaustion. Capacity planning can be hard. This talk has a heavier leaning on MySQL, but the concepts and addendum will help with any other data store.
Lessons from {distributed,remote,virtual} communities and companiesColin Charles
A last minute talk for the people at DevOps Amsterdam, happening around the same time as O'Reilly Velocity Amsterdam 2016. Here are lessons one can learn from distributed/remote/virtual communities and companies from someone that has spent a long time being remote and distributed.
Forking Successfully - or is a branch better?Colin Charles
Forking Successfully or do you think a branch will work better? Learn from history, see what's current, etc. Presented at OSCON London 2016. This is forking beyond the github generation. And if you're going to do it, some tips on how you could be successful.
Securing your MySQL / MariaDB Server dataColin Charles
Co-presented alongside Ronald Bradford, this covers MySQL, Percona Server, and MariaDB Server (since the latter occasionally can be different enough). Go thru insecure practices, focus on communication security, connection security, data security, user accounts and server access security.
This was a short 25 minute talk, but we go into a bit of a history of MySQL, how the branches and forks appeared, what's sticking around today (branch? Percona Server. Fork? MariaDB Server). What should you use? Think about what you need today and what the roadmap holds.
Failure happens, and we can learn from it. We need to think about backups, but also verification of them. We should definitely make use of replication and think about automatic failover. And security is key, but don't forget that encryption is now available in MySQL, Percona Server and MariaDB Server.
Presented at the MySQL Chicago Meetup in August 2016. The focus of the talk is on backups and verification, replication and failover, as well as security and encryption.
This document discusses MariaDB plugins and provides examples of several useful plugins, including authentication plugins, password validation plugins, SQL error logging, audit logging, query analysis, and more. It encourages contributing plugins to help extend MariaDB's functionality.
Coordinate Systems in FME 101 - Webinar SlidesSafe Software
If you’ve ever had to analyze a map or GPS data, chances are you’ve encountered and even worked with coordinate systems. As historical data continually updates through GPS, understanding coordinate systems is increasingly crucial. However, not everyone knows why they exist or how to effectively use them for data-driven insights.
During this webinar, you’ll learn exactly what coordinate systems are and how you can use FME to maintain and transform your data’s coordinate systems in an easy-to-digest way, accurately representing the geographical space that it exists within. During this webinar, you will have the chance to:
- Enhance Your Understanding: Gain a clear overview of what coordinate systems are and their value
- Learn Practical Applications: Why we need datams and projections, plus units between coordinate systems
- Maximize with FME: Understand how FME handles coordinate systems, including a brief summary of the 3 main reprojectors
- Custom Coordinate Systems: Learn how to work with FME and coordinate systems beyond what is natively supported
- Look Ahead: Gain insights into where FME is headed with coordinate systems in the future
Don’t miss the opportunity to improve the value you receive from your coordinate system data, ultimately allowing you to streamline your data analysis and maximize your time. See you there!
Choose our Linux Web Hosting for a seamless and successful online presencerajancomputerfbd
Our Linux Web Hosting plans offer unbeatable performance, security, and scalability, ensuring your website runs smoothly and efficiently.
Visit- https://onliveserver.com/linux-web-hosting/
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxSynapseIndia
Your comprehensive guide to RPA in healthcare for 2024. Explore the benefits, use cases, and emerging trends of robotic process automation. Understand the challenges and prepare for the future of healthcare automation
INDIAN AIR FORCE FIGHTER PLANES LIST.pdfjackson110191
These fighter aircraft have uses outside of traditional combat situations. They are essential in defending India's territorial integrity, averting dangers, and delivering aid to those in need during natural calamities. Additionally, the IAF improves its interoperability and fortifies international military alliances by working together and conducting joint exercises with other air forces.
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsMydbops
This presentation, delivered at the Postgres Bangalore (PGBLR) Meetup-2 on June 29th, 2024, dives deep into connection pooling for PostgreSQL databases. Aakash M, a PostgreSQL Tech Lead at Mydbops, explores the challenges of managing numerous connections and explains how connection pooling optimizes performance and resource utilization.
Key Takeaways:
* Understand why connection pooling is essential for high-traffic applications
* Explore various connection poolers available for PostgreSQL, including pgbouncer
* Learn the configuration options and functionalities of pgbouncer
* Discover best practices for monitoring and troubleshooting connection pooling setups
* Gain insights into real-world use cases and considerations for production environments
This presentation is ideal for:
* Database administrators (DBAs)
* Developers working with PostgreSQL
* DevOps engineers
* Anyone interested in optimizing PostgreSQL performance
Contact info@mydbops.com for PostgreSQL Managed, Consulting and Remote DBA Services
Implementations of Fused Deposition Modeling in real worldEmerging Tech
The presentation showcases the diverse real-world applications of Fused Deposition Modeling (FDM) across multiple industries:
1. **Manufacturing**: FDM is utilized in manufacturing for rapid prototyping, creating custom tools and fixtures, and producing functional end-use parts. Companies leverage its cost-effectiveness and flexibility to streamline production processes.
2. **Medical**: In the medical field, FDM is used to create patient-specific anatomical models, surgical guides, and prosthetics. Its ability to produce precise and biocompatible parts supports advancements in personalized healthcare solutions.
3. **Education**: FDM plays a crucial role in education by enabling students to learn about design and engineering through hands-on 3D printing projects. It promotes innovation and practical skill development in STEM disciplines.
4. **Science**: Researchers use FDM to prototype equipment for scientific experiments, build custom laboratory tools, and create models for visualization and testing purposes. It facilitates rapid iteration and customization in scientific endeavors.
5. **Automotive**: Automotive manufacturers employ FDM for prototyping vehicle components, tooling for assembly lines, and customized parts. It speeds up the design validation process and enhances efficiency in automotive engineering.
6. **Consumer Electronics**: FDM is utilized in consumer electronics for designing and prototyping product enclosures, casings, and internal components. It enables rapid iteration and customization to meet evolving consumer demands.
7. **Robotics**: Robotics engineers leverage FDM to prototype robot parts, create lightweight and durable components, and customize robot designs for specific applications. It supports innovation and optimization in robotic systems.
8. **Aerospace**: In aerospace, FDM is used to manufacture lightweight parts, complex geometries, and prototypes of aircraft components. It contributes to cost reduction, faster production cycles, and weight savings in aerospace engineering.
9. **Architecture**: Architects utilize FDM for creating detailed architectural models, prototypes of building components, and intricate designs. It aids in visualizing concepts, testing structural integrity, and communicating design ideas effectively.
Each industry example demonstrates how FDM enhances innovation, accelerates product development, and addresses specific challenges through advanced manufacturing capabilities.
Transcript: Details of description part II: Describing images in practice - T...BookNet Canada
This presentation explores the practical application of image description techniques. Familiar guidelines will be demonstrated in practice, and descriptions will be developed “live”! If you have learned a lot about the theory of image description techniques but want to feel more confident putting them into practice, this is the presentation for you. There will be useful, actionable information for everyone, whether you are working with authors, colleagues, alone, or leveraging AI as a collaborator.
Link to presentation recording and slides: https://bnctechforum.ca/sessions/details-of-description-part-ii-describing-images-in-practice/
Presented by BookNet Canada on June 25, 2024, with support from the Department of Canadian Heritage.
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Best Practices for Effectively Running dbt in Airflow.pdfTatiana Al-Chueyr
As a popular open-source library for analytics engineering, dbt is often used in combination with Airflow. Orchestrating and executing dbt models as DAGs ensures an additional layer of control over tasks, observability, and provides a reliable, scalable environment to run dbt models.
This webinar will cover a step-by-step guide to Cosmos, an open source package from Astronomer that helps you easily run your dbt Core projects as Airflow DAGs and Task Groups, all with just a few lines of code. We’ll walk through:
- Standard ways of running dbt (and when to utilize other methods)
- How Cosmos can be used to run and visualize your dbt projects in Airflow
- Common challenges and how to address them, including performance, dependency conflicts, and more
- How running dbt projects in Airflow helps with cost optimization
Webinar given on 9 July 2024
Best Programming Language for Civil EngineersAwais Yaseen
The integration of programming into civil engineering is transforming the industry. We can design complex infrastructure projects and analyse large datasets. Imagine revolutionizing the way we build our cities and infrastructure, all by the power of coding. Programming skills are no longer just a bonus—they’re a game changer in this era.
Technology is revolutionizing civil engineering by integrating advanced tools and techniques. Programming allows for the automation of repetitive tasks, enhancing the accuracy of designs, simulations, and analyses. With the advent of artificial intelligence and machine learning, engineers can now predict structural behaviors under various conditions, optimize material usage, and improve project planning.
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc
Six months into 2024, and it is clear the privacy ecosystem takes no days off!! Regulators continue to implement and enforce new regulations, businesses strive to meet requirements, and technology advances like AI have privacy professionals scratching their heads about managing risk.
What can we learn about the first six months of data privacy trends and events in 2024? How should this inform your privacy program management for the rest of the year?
Join TrustArc, Goodwin, and Snyk privacy experts as they discuss the changes we’ve seen in the first half of 2024 and gain insight into the concrete, actionable steps you can take to up-level your privacy program in the second half of the year.
This webinar will review:
- Key changes to privacy regulations in 2024
- Key themes in privacy and data governance in 2024
- How to maximize your privacy program in the second half of 2024
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...Toru Tamaki
Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, Philip Torr "A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models" arXiv2023
https://arxiv.org/abs/2307.12980
How Social Media Hackers Help You to See Your Wife's Message.pdfHackersList
In the modern digital era, social media platforms have become integral to our daily lives. These platforms, including Facebook, Instagram, WhatsApp, and Snapchat, offer countless ways to connect, share, and communicate.
Details of description part II: Describing images in practice - Tech Forum 2024BookNet Canada
This presentation explores the practical application of image description techniques. Familiar guidelines will be demonstrated in practice, and descriptions will be developed “live”! If you have learned a lot about the theory of image description techniques but want to feel more confident putting them into practice, this is the presentation for you. There will be useful, actionable information for everyone, whether you are working with authors, colleagues, alone, or leveraging AI as a collaborator.
Link to presentation recording and transcript: https://bnctechforum.ca/sessions/details-of-description-part-ii-describing-images-in-practice/
Presented by BookNet Canada on June 25, 2024, with support from the Department of Canadian Heritage.
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfNeo4j
Presented at Gartner Data & Analytics, London Maty 2024. BT Group has used the Neo4j Graph Database to enable impressive digital transformation programs over the last 6 years. By re-imagining their operational support systems to adopt self-serve and data lead principles they have substantially reduced the number of applications and complexity of their operations. The result has been a substantial reduction in risk and costs while improving time to value, innovation, and process automation. Join this session to hear their story, the lessons they learned along the way and how their future innovation plans include the exploration of uses of EKG + Generative AI.
Measuring the Impact of Network Latency at TwitterScyllaDB
Widya Salim and Victor Ma will outline the causal impact analysis, framework, and key learnings used to quantify the impact of reducing Twitter's network latency.
Quantum Communications Q&A with Gemini LLM. These are based on Shannon's Noisy channel Theorem and offers how the classical theory applies to the quantum world.
The Complete MariaDB Server Tutorial - Percona Live 2015
1. The Complete
MariaDB Server
Tutorial
Colin Charles
colin@mariadb.org
http://bytebot.net/blog/ | @bytebot on twitter
http://mariadb.org/ | http://mariadb.com/
Percona Live Santa Clara, California, USA
13 April 2015
1
3. whoami
• Work on MariaDB at MariaDB Corporation
(SkySQL Ab)
• Merged with Monty Program Ab, makers of
MariaDB
• Formerly MySQL AB (exit: Sun Microsystems)
• Past lives include Fedora Project (FESCO),
OpenOffice.org
• Been a MySQL user since 2000
3
4. MariaDB is very social
• facebook: fb.com/MariaDB.dbms
• twitter: @MariaDB
• g+: plus.google.com/+mariadb/
• Tweet things you learn with #mariadb
4
9. What is MariaDB?
9
• Community developed
• maria-captains: 42% Team
MariaDB, 58% community
including Sphinxsearch,
Twitter, SkySQL,Taobao,
Facebook, Percona,
Codership & more
• Feature enhanced
• MariaDB doesn’t depend on
MySQL for development -
many features are
developed independently of
MySQL
• Backwards compatible with
MySQL
• feature complete
• replication supported for
easy migration
10. Aims of MariaDB
• Compatible, drop-in replacement to MySQL
• your application shouldn’t care that its
running MariaDB, easy upgrade (uninstall
mysql, install mariadb, continue ops!)
• Stable (bug-free) releases with no
regressions
• GPLv2
10
11. 5 years, many server
releases
• MariaDB 5.1, GA February 2010
• MariaDB 5.2, GA November 2010
• MariaDB 5.3, GA February 2012
• MariaDB 5.5, GA April 2012
• MariaDB Galera Cluster, GA March 2013
• MariaDB 10.0.10 (March 2014)
11
13. MariaDB 5.6?
• There will never be a MariaDB 5.6 - the
numbers have changed
• Many companies will still continue to
support both MySQL and MariaDB releases
13
16. Importance of understanding
MariaDB (and MySQL)
• Generics are inefficient
• Since you have chosen MariaDB:
• maximise its strengths
• minimise its weaknesses
16
17. Sample databases to
play with
• http://dev.mysql.com/doc/index-other.html
• sakila sample database, world
database (used in MySQL training),
menagerie database (used in book:
Beginning MySQL), employees
database (large dataset, comes with data -
best to play with)
17
18. Picking hardware
• Just use 64-bit hardware
• VM’s are improving to use 64-bit OSes &
software
• Physical > virtual
• Disk: battery backed storage, plan for RAID
usage
• MariaDB 10.1 has optimisations for flash/SSD/
FusionIO
18
19. Testing MySQL
• Use MySQL Sandbox
• http://mysqlsandbox.net/
• Express one-click MySQL installs
• make_sandbox foo.tar.gz
• Does not require root privileges
19
20. Installation
• Binaries (tarballs) are available at http://mariadb.org/ (source too)
• Built, tested by MariaDB
• Graphical installer & configuration for Microsoft Windows, with
HeidiSQL GUI
• Up-to-date predictable release schedule
• RPM, DEB packages are provided, includingYUM & APT
repositories
• Use the repository configuration tool
• Inside Linux/*BSD distributions
• Easy to install, basic defaults, may be older than upstream
20
21. Upgrades
• Review changelogs carefully, even for minor
versions
• Make backups (using xtrabackup)
• Don’t forget to run mysql_upgrade
• by default with a distribution package
• Replication can reduce downtime by upgrading
the slave, promoting it, then upgrading the master
21
22. Upgrading from MySQL
5.0
• MariaDB 5.1 fixes upgrades better than MySQL 5.1
handles this
• InnoDB + Archive tables upgraded properly
• mysql_upgrade, mysqlcheck have more options to see
what’s going on
• mysqlcheck wrong warnings removed
• MySQL 5.1 -> MariaDB 5.1 is “drop-in”
• https://mariadb.com/kb/en/upgrading-to-mariadb-from-
mysql/
22
23. Packages (RPM
example)
• MariaDB-common
• character sets
• MariaDB-shared
• latest libmysqlclient, some plugins,
/etc/my.cnf.d/
• MariaDB-compat
• compatible client libraries that
replace mysql-libs
• MariaDB-server
• The actual server!
• MariaDB-client
• mysql client utilities
• MariaDB-test
• test suite
• MariaDB-devel
• development headers enabling
you to build your own package
• MariaDB-CassandraSE
• plugin for CassandraSE
23
24. What is libmysqlclient?
• Client libraries
• Many applications are compiled against
libmysqlclient
• It is an Application Binary Interface (ABI)
• When you write in Java & use Connector/J,
Connector/J is compiled against libmysqlclient
• Similarly with PHP & Connector/PHP
24
25. MySQL utilities
• Why does MariaDB use all the same client utilities?
• Because the aim is to be a compatible drop-in
replacement
• There are very few non-MySQL utilities shipped:
aria_chk, aria_dump_log, aria_ftdump,
aria_pack, aria_read_log
• xtstat (PBXT) is deprecated in MariaDB 5.5
onwards
• mytop as it adds features not-present upstream
25
26. Finding help
• How do you know what CLI options to use?
• Use the man(ual) pages!
•man mysql
• man <command_name> usually works for
any CLI command
• HELP SELECT in mysql works too (help
tables) - HELP <operator>
26
27. Error messages
• perror is a great tool
• MariaDB-specific errors are 1900 and
above
• https://mariadb.com/kb/en/mariadb-error-
codes/#mariadb-specific-error-codes
27
28. mysql_secure_insta
llation
• Now that you’ve tested MySQL works fine (by
default, no root password), it’s time to ensure you
secure the installation
• Set root password
• Remove anonymous users
• Disallow remote root login
• Remove test database
• Reload privileges
28
29. mysqladmin
• Manage user accounts, passwords,
permissions
• Display mysqld settings & status counters
• Shutdown, create, drop databases
shell> mysqladmin processlist
shell> mysqladmin extended-status
shell> mysqladmin -uroot -p create wordpress
29
SHOW PROCESSLIST
SHOW STATUS
30. mysqldump
• Dump schema and/or data to .sql file, tab
• Useful for backups, transferring data
• Locks for consistency, so troublesome
shell> mysqldump --all-databases > backup.sql
shell> mysqldump --all-databases --single-transaction >
backup.sql
30
31. Other tools
• There are many other command line
tools...
• Front-ends
• HeidiSQL
• Webyog’s SQLyog
• Sequel Pro (OSX)
31
33. Making client
connections
• TCP/IP is available on all platforms
• disable using --skip-networking
• Socket files available on Unix
• fastest communication path
• MySQL connections are generally cheap
• If you have many short running queries (typical web
app), you benefit from MariaDB threadpool
• Set connection limit max_connections=n in my.cnf
33
34. Connection Pool
• mysqld’s main thread listens for connections
• Each connection has a thread assigned to it
• Threads can be:
• created newly
• reused from thread cache
• User authentication processed based on hostname,
username & password
• Client specific buffers for session variables are allocated
34
36. Query Cache
• Stores SELECT queries and their result sets
• Subquery cache exists only in MariaDB
• Frequently changed table data will cause
queries to naturally be missed
• Regularly replaced these days with
memcached, et al.
36
37. SQL Parser
• Lexical scanner & grammar rules
• Parse SQL into tokens
• Apply grammar rules to check statement
validity
• Construct a parse tree for Optimizer to
use
37
38. Optimizer
• Reads the parse tree and calculates the best query
execution plan (QEP) to handle query
• find indexes
• determine JOIN order
• eliminate unnecessary tables
• etc.
• The optimizer is smart, don’t try to force query
plans
38
39. Pluggable Storage
Engines
• MariaDB can use many storage engines
with different features
• Installed/removed on fly with INSTALL/
UNINSTALL PLUGIN
• Mixing & matching on same server, even on
same query
39
40. Value proposition
• Unmatched flexibility + customisation potential
• MEMORY engine for performance/routine
lookup data
• Right storage engine can improve performance
in many applications
• ARCHIVE compresses data, up to 80%
• Partners & community benefit from this
40
41. What makes engines
different?
• Storage: how the data is stored on disk
• Or in NDB (memory+disk), CassandraSE (access a Cassandra Cluster), SphinxSE
(access the Sphinx daemon)
• Indexes: improves search operations
• Memory usage: improves data access for speed
• Transactions: protects the integrity of your data (Atomic-Consistent-Isolated-
Durable - ACID)
• Locking level: MyISAM (table locks), InnoDB (row locks), old BDB (page locks)
• Data types: Data types may be converted, MEMORY doesn’t support TEXT, etc.
• Caching: InnoDB caches data & indexes, MyISAM caches indexes only (relying on OS
disk cache for data)
• Full-text search capability: MyISAM has this, InnoDB 5.6 got this
• GIS: MyISAM & Aria work (R-tree indexes exist), InnoDB 5.7 has this too
41
42. INFORMATION_SCHE
MA
• Holds metadata (data about the data) on all
other databases & tables, exposed as
regular tables
• Generated on the fly
• Has extensions in MariaDB as we expose
more data than native MySQL
42
43. PERFORMANCE_SCH
EMA
• Allows for monitoring execution at a low
level
• This is a storage engine, monitoring server
events (anything that takes time & can be
instrumented)
• Tables are views or temporary tables that
use no on-disk storage
43
44. Transactional vs. non-
transactional
• Transaction-safe tables
(InnoDB) have advantages
over non-transaction safe
tables (MyISAM):
• server crash? Automatic
recovery, or a backup
+transaction log
• ROLLBACK can be
executed to ignore
changes
• Update fails? Changes
reverted
• Concurrency - tables w/
many update +
concurrent reads
• Disadvantages in today’s
environments (transaction
overhead = slower), more
disk space requirements,
more memory to perform
updates don’t seem like they
apply any longer
44
45. Indexes
• Tree Indexes
• B-Trees
• B+Trees (InnoDB)
• T-Trees (NDB)
• Red-black binary trees
(MEMORY)
• R-Trees (MyISAM for spatial
indexes)
• Hash Indexes (MEMORY, NDB,
InnoDB)
• If table fits entirely in
memory, fastest way to
perform queries is a hash
index
• InnoDB has an internal
adaptive hash index. InnoDB
monitors index searches,
and if it notices that it will
benefit from a hash index,
InnoDB automatically builds
one. (5.1.24 and greater)
45
46. MyISAM
• Pros?
• excellent INSERT
performance
• small footprint
• supports full-text
search (FTS)
• Cons?
• no transactions
• no foreign key
support
• Typical uses
• logging
• auditing
• data warehousing
46
47. MyISAM II
• In my.cnf, remember to
set the key_buffer_size.
This is memory*0.40, as
MyISAM uses the OS
cache for tables
• myisam_use_mmap
enables MyISAM to use
memory mapping
(7-40% speed
improvement)
• key_cache_segments =
1 enables segmented key
caches in MariaDB -
~250% improvements, as
it mitigates thread
contention for key cache
lock
47
48. MyISAM segmented key
caches
• Mitigates thread contention for key cache lock, with
notable performance improvements
• Key caches divided into different segments, allowing
for better key cache concurrency
• 1-64 segments
48
49. InnoDB
• Maintains its own buffer pool (does
aggressive memory caching)
• Uses tablespaces (several files on disk, raw
disk support)
• Typically used for OLTP operations
49
50. ARCHIVE
• Store large amounts of data without
indexes, in small disk footprint
• SELECT and INSERT operations only
• Good for data audit use
• Uses AZIO (zlib) compression
50
51. FederatedX
• Create logical pointers to tables that exist on other MySQL servers;
these can then be linked together to form one logical database
• A federated table pointing to an InnoDB table on another server, will
have transaction support (in 5.1)
• Capabilities limited to underlying engine on remote server
• CREATE TABLE t1 (...) ENGINE=FEDERATED
CONNECTION='mysql://username:pwd@myhost:3306/db_name/
tbl_name
• Can also be used for synchronous replication
• Federated table on master server pointing to slave; triggers on
master table to write all changes to remote table once applied to
the master
51
52. Memory
• Previously known as HEAP tables
• In-memory engine
• Hash index used by default (changes in 5.2,
enable much better INSERT performance),
B-Tree available too
• https://mariadb.com/kb/en/performance-of-
memory-tables/
52
53. Aria
• Based off the 5.1 code
• 1.0 – crash-safe MyISAM, with cacheable row format
• 1.5 – concurrent INSERT/SELECT
• Soon to be merged into 6.0, then...
• 2.0: transactional + ACID compliance
• 3.0: high concurrency, online backup
• Goal:ACID compliant, MVCC transactional storage engine, based on MyISAM
• Target? Data warehousing
• Uses big log files (1GB by default)
• 8K pages used by default (MyISAM uses 1K pages)
• Has group commit (MariaDB 5.2) to speed up inserts
53
54. PBXT (deprecated)
• MVCC, transactional,ACID compliant, foreign key
support
• row-level locking for updates, so maximum concurrency
• immediate notification if client processes are
deadlocked
• write-once, as it uses a log-based architecture (write
data to DB without first writing to transaction log)
• support for BLOB streaming with Blob Streaming
engine
54
55. Storage Engine API
• http://dev.mysql.com/tech-resources/
articles/creating-new-storage-engine.html
• SHOW PLUGINS;
• https://kb.askmonty.org/v/extending-create-
table
• storage/example/ha_example.cc and
storage/example/ha_example.h
55
56. Writing your own
• Find the plugin path - show variables like “%plugin%”;
+-----------------+-----------------------------------------------+!
| Variable_name | Value |!
+-----------------+-----------------------------------------------+!
| plugin_dir | /usr/local/Cellar/mariadb/10.0.15/lib/plugin/ |!
| plugin_maturity | unknown |!
+-----------------+-----------------------------------------------+!
• note that this is also where you store UDFs
• Copy the relevant engine (eg. myengine.so)
• INSTALL PLUGIN myengine SONAME 'myengine.so';
• Server registers plugin to mysql.plugin table, and now ENGINE=myengine will work
56
57. Things to think about
• Backup is not engine-independent
• MyISAM, InnoDB,TokuDB
• LVM/ZFS snapshots mitigate this
• Different engines have different monitoring
options
• Mix and match; use summary tables
57
58. Survey of popular OSS
tools - what they use
• Wordpress (blog): uses default engine, MyISAM is fine
• MediaWiki (wiki): prefers InnoDB, except for
“searchindex” table, which is MyISAM
• http://svn.wikimedia.org/viewvc/mediawiki/trunk/
phase3/maintenance/tables.sql?view=markup
• vBulletin (forum): MyISAM
• SugarCRM (CRM): MyISAM (with conversion script to
InnoDB provided)
• Zimbra Collaboration Suite: InnoDB
58
60. We start with...
• What came in 5.1, 5.2, 5.3, 5.5 (jump
around appropriately)
• What comes in 10.0 series
• I won’t talk about deprecated features like
PBXT in-depth, or even the old MariaDB
5.1 pool of threads (5.5 threadpool is
better + improvements in 10.1)
60
61. XtraDB
• A more performant
InnoDB designed to
scale on modern
hardware
• Less checkpointing
(smoother), less flushing
to disk
61
62. Switching between
XtraDB & InnoDB
mysqld --ignore-builtin-innodb --plugin-
load=innodb=ha_innodb.so --plugin_dir=/usr/local/
mysql/lib/mysql/plugin
!
Or in my.cnf
[mysqld]
ignore-builtin-innodb
plugin-load=innodb=ha_innodb.so
plugin_dir=/usr/local/mysql/lib/mysql/plugin
62
63. Usernames
• Usernames in MariaDB > 5.5.31? 80 character limit (which
you have to reload manually)
create user
'long12345678901234567890'@'localhost'
identified by 'pass';
Query OK, 0 rows affected (0.01 sec)
vs
ERROR 1470 (HY000): String
'long12345678901234567890' is too long for
user name (should be no longer than 16)
63
64. MariaDB 5.5: an
opensource threadpool
• Modified from 5.1 (libevent
based), great for CPU
bound loads and short
running queries
• No minimization of
concurrent transactions
with dynamic pool size
• thread_handling=po
ol-of-threads
• https://mariadb.com/kb/en/
mariadb/thread-pool-in-
mariadb/
64
65. Improved threadpool
• You can limit resources used by threads:
thread_pool_max_threads
• SHOW GLOBALVARIABLES LIKE
'%thread_%';
65
66. Better for DBAs: async
client library
• start operation, do work
in thread, operation
processed, result travels
back
• use cases: multiple
queries against single
server (utilize more
CPUs); queries against
multiple servers
(SHOW STATUS on
many machines)
• https://
kb.askmonty.org/en/
about-non-blocking-
operation-in-the-
client-library/
• fast node.js driver
available: mariasql
• https://
kb.askmonty.org/en/
mariasql-for-nodejs/
66
67. LIMIT ROWS
EXAMINED
• The purpose of this
optimization is to
provide the means to
terminate the execution
of SELECT statements
which examine too
many rows, and thus use
too many resources.
•SELECT * from
t1, t2 LIMIT 10
ROWS EXAMINED
1000;
• https://kb.askmonty.org/
en/limit-rows-examined/
67
68. SHOW STATUS
• SHOW STATUS provides server status
information. It is like mysqladmin extended-status.
•SHOW STATUS LIKE ‘Key%’;
• https://kb.askmonty.org/en/show-status/
• https://kb.askmonty.org/en/server-status-variables/
• MariaDB has opened_views, executed_triggers,
executed_events, feature_* as new options
68
69. SQL Error Logging
Plugin
• Log errors sent to clients in a log file that
can be analysed later. Log file can be rotated
(recommended)
• a MYSQL_AUDIT_PLUGIN
install plugin SQL_ERROR_LOG
soname 'sql_errlog.so';
69
70. Audit Plugin
• Log server activity - who connects to the
server, what queries run, what tables touched -
rotating log file or syslogd
• MariaDB has extended the audit API, so user
filtering is possible
• a MYSQL_AUDIT_PLUGIN
INSTALL PLUGIN server_audit SONAME
‘server_audit.so’;
70
71. Replication: selective
skipping
• All changes that are logged as events in the
binlog are replicated to all slaves
• However, sometimes you want all to be logged
to binlog but skipped replication to slaves
• @@skip_replication (session only)
• replicate_events_marked_for_skip
= replicate|filter_on_slave|
filter_on_master (dynamic)
71
72. Replication: dynamic
variables
• The variables replicate_do_*,
replicate_ignore_*, and replicate_wild_*
have been made dynamic, so they can be
changed without requiring a server restart.
• https://kb.askmonty.org/en/dynamic-
replication-variables/
72
73. Replication:Annotation
of RBR events
• MariaDB supports statement & row based
replication (RBR)
• In RBR, the binlog has no SQL statements,
only events are logged (INSERT, DELETE, etc)
• Option to include original SQL statement
(default OFF)
• https://kb.askmonty.org/en/
annotate_rows_log_event/
73
74. Replication: binlog
event checksums
• Backport from MySQL 5.6 (in MariaDB
5.3+)
• binlog_checksum option
• Slaves perform checksums on events
received & will stop if there is corruption
• https://kb.askmonty.org/en/binlog-event-
checksums/
74
75. Replication: group
commit in the binary log
•sync_binlog=1,
innodb_flush_log_at_trx_commit
=1
• https://www.facebook.com/note.php?
note_id=10150261692455933
• http://kb.askmonty.org/en/group-commit-for-
the-binary-log
• SHOW STATUS LIKE 'binlog_%commits';
75
77. Group commit in
MariaDB 5.3 onwards
• Do slow part of prepare() in parallel in
InnoDB (first fsync(), InnoDB group
commit)
• Put transaction in queue, decide commit
order
77
78. • First in queue runs serial part for all, rest
wait
• Wait for access to the binlog
• Write transactions into binlog, in order,
then sync (second fsync())
• Run the fast part of commit() for all
transactions in order
78
79. • Finally, run the slow part of commit() in
parallel (third fsync(), InnoDB group
commit)
• Only 2 context switches per thread (one
sleep, one wakeup)
• Note: MySQL 5.6, MariaDB 10 only does 2
fsyncs/group commit
79
80. Group commit in
MariaDB 10
• Remove commit in slow part of InnoDB
commit (stage 4)
• Reduce cost of crash-safe binlog
• A binlog checkpoint is a point in the
binlog where no crash recovery is
needed before it. In InnoDB you wait for
flush + fsync its redo log for commit
80
81. crash-safe binlog
• MariaDB 5.5 checkpoints after every
commit —> quite expensive!
• 5.5/5.6 stalls commits around binlog rotate,
waiting for all prepared transactions to
commit (since crash recovery can only scan
latest binlog file)
81
82. crash-safe binlog 10.0
• 10.0 makes binlog checkpoints
asynchronous
• A binlog can have no checkpoints at all
• Ability to scan multiple binlogs during
crash recovery
• Remove stalls around binlog rotates
82
86. Extensions to the SE
API
• prepare() - write prepared trx in
parallel w/group commit
• prepare_ordered() - called serially, in
commit order
• commit_ordered() - called serially, in
commit order; fast commit to memory
• commit() - commit to disk in parallel,
86
87. group commit in 10.1
• Tricky locking issues hard to change without getting deadlocks
sometimes
• mysql#68251, mysql#68569
• New code? Binlog rotate in background thread (further reducing
stalls). Split transactions across binlogs, so big transactions do not
lead to big binlog files
• Enhanced semi-sync replication (wait for slave before commit on
the master rather than after commit)
87
88. Replication: START TRANSACTION
WITH CONSISTENT SNAPSHOT
• Works with the binlog, possible to obtain the binlog position
corresponding to a transactional snapshot of the database without
blocking any other queries.
• by-product of group commit in the binlog to view commit
ordering
• Used by the command mysqldump--single-transaction
--master-data to do a fully non-blocking backup which can be
used to provision a new slave
• Works consistently between transactions involving more than one
storage engine
• https://kb.askmonty.org/en/enhancements-for-start-transaction-with-
consistent/
88
89. GIS support!
• MySQL has OpenGIS SFS (Simple feature access,
SQL access method)
• Now, SQL with full geometry types
• ST_ prefix (incl. ST_RELATE,
ST_BOUNDARY, etc.)
• MyISAM,Aria for SPATIAL & non-spatial indexes
• When 5.7-InnoDB is merged, it will get
support too
89
90. GIS II
• Spatial reference systems support
(REF_SYSTEM_ID) can be specified as a column
attribute
• INFORMATION_SCHEMA.GEOMETRY_COLUMN
S for queries of references
• Use Osmosis, you can load all OpenStreetMap data
into MariaDB now
• https://mariadb.com/kb/en/gis-features-in-533/
• https://mariadb.com/kb/en/openstreetmap-dataset/
90
91. Progress reporting
• ALTER TABLE & LOAD DATA INFILE
MariaDB [mail]> alter table mail engine = maria;
Stage: 1 of 2 'copy to tmp table' 17.55% of stage done
MariaDB [mail]> select id, user, db, command, state,
-> time_ms, progress from information_schema.processlist;
+---------+-------------------+-----------+----------+
| command | state | time_ms | progress |
+---------+-------------------+-----------+----------+
| Query | copy to tmp table | 23407.131 | 17.551 |
+---------+-------------------+-----------+----------+
1 row in set (0.47 sec)
91
92. TIME_MS in
I_S.PROCESSLIST
• Extra column 'TIME_MS' has been added to
the
INFORMATION_SCHEMA.PROCESSLIST
table
• Units of milliseconds with microsecond
precision (the unit and precision of the
'TIME' column is one second).
92
93. New KILL syntax
• HARD | SOFT & USER USERNAME are MariaDB-specific
(5.3.2)
• KILL QUERY ID query_id (10.0.5) - kill by query id, rather
than thread id
• SOFT ensures things that may leave a table in an inconsistent
state aren’t interrupted (like REPAIR or INDEX creation for
MyISAM or Aria)
KILL [HARD | SOFT] [CONNECTION | QUERY]
[thread_id | USER user_name]
93
95. The old days
• Download MySQL, including sources
• Download SphinxSE for compiling
• Download Sphinx to compile with MySQL
support
• Documented: http://www.howtoforge.com/
sphinx-as-mysql-storage-engine-sphinxse
95
96. Today
• Install sphinx from your distribution
• Install MariaDB 5.5 from your distribution
or from http://mariadb.org/
• Get started!
96
99. What is SphinxSE?
• SphinxSE is just the storage engine that still
depends on the Sphinx daemon
• It doesn’t store any data itself
• Its just a built-in client to allow MariaDB to
talk to Sphinx searchd, run queries, obtain
results
• Indexing, searching is performed on Sphinx
99
100. Configure sphinx!
• /usr/local/sphinx/sphinx.conf
• Source (multiple, include mysql, with
connection info)
• Setup indexer (esp. if its on localhost) -
mem_limit, max_iops, max_iosize
• Setup searchd (where to listen to, query
log, etc.)
100
101. Use case scenarios
• Already have an existing application that
makes use of full-text-search in MyISAM?
Porting should be easier
• Have a programming language without a
native API for Sphinx? Surely there’s a
connector for MariaDB ;-)
101
102. Use case scenarios
• Results from Sphinx itself almost always
require additional work involving MariaDB
• Say to pull out text column that Sphinx
index doesn’t store
• JOIN with another table (using a different
engine)
102
103. An example
CREATE TABLE t1!
(!
id INTEGER UNSIGNED NOT NULL,!
weight INTEGER NOT NULL,!
query VARCHAR(3072) NOT NULL,!
group_id INTEGER,!
INDEX(query)!
) ENGINE=SPHINX CONNECTION="sphinx://localhost:9312/test";!
!
SELECT * FROM t1 WHERE query='test it;mode=any';!
103
104. Sphinx search tables
• 1st column: INTEGER UNSIGNED or
BIGINT (document ID)
• 2nd column: match weight
• 3rd column: VARCHAR or TEXT (your
query)
• Query column needs indexing, no other
column needs to be
104
105. What actually happens
• SELECT passes a Sphinx query as the query
column in the WHERE clause
• searchd returns the results
• SphinxSE translates and returns the results
to MariaDB
105
106. SHOW ENGINE
SPHINX STATUS
• Per-query & per-word statistics that searchd returns are accessible via SHOW STATUS
!
mysql> SHOW ENGINE SPHINX STATUS;!
+--------+-------+-------------------------------------------------+!
| Type | Name | Status |!
+--------+-------+-------------------------------------------------+!
| SPHINX | stats | total: 25, total found: 25, time: 126, words: 2 | !
| SPHINX | words | sphinx:591:1256 soft:11076:15945 | !
+--------+-------+-------------------------------------------------+!
2 rows in set (0.00 sec)!
106
107. What queries are
supported?
• Most of the Sphinx API is exposed to SphinxSE
• query, mode, sort, offset, limit, index, minid,
maxid, weights, filter, !filter, range, !range,
maxmatches, groupby, groupsort, indexweights,
comment, select
• Sphinx search modes can also be supported via
_sph attributes
• obtain value of @groupby? use ‘_sph_groupby’
107
108. Efficiency
• Allow Sphinx to perform sorting, filtering, and slicing
of result set
• ... as opposed to using WHERE, ORDER BY, LIMIT
clauses on MariaDB
• Why?
• Sphinx optimises and performs better on these
tasks
• Less data packed by searchd, and transferred and
unpacked by SphinxSE
108
109. JOINs
• Perform JOINs on a SphinxSE search table using tables from other engines
SELECT content, date_added FROM test.documents docs!
-> JOIN t1 ON (docs.id=t1.id) !
-> WHERE query="one document;mode=any";!
+-------------------------------------+---------------------+!
| content | docdate |!
+-------------------------------------+---------------------+!
| this is my test document number two | 2006-06-17 14:04:28 | !
| this is my test document number one | 2006-06-17 14:04:28 | !
+-------------------------------------+---------------------+!
2 rows in set (0.00 sec)!
109
110. Statistics
• Understand server activity better to understand database loads
•SET GLOBAL userstat=1;
•SHOW CLIENT_STATISTICS; SHOW USER_STATISTICS;
• # of connections, CPU usage, bytes received/sent, row statistics
•SHOW INDEX_STATISTICS; SHOW TABLE_STATISTICS;
• # rows read, changed, indexes
• INFORMATION_SCHEMA.PROCESSLIST has MEMORY_USAGE,
EXAMINED_ROWS (similar with SHOW STATUS output)
110
MariaDB 10.0+
111. Table Elimination
• Resolve a query without accessing some
tables query refers to
• Great for querying highly normalised data
• Basis of “anchor modelling”
• http://www.anchormodeling.com/
• SQL Server 2005/2008, Oracle 11g have it
111
112. Virtual columns
• A column in a table that has its value
automatically calculated either with a pre-
calculated/deterministic expression or values
of other fields in the table
• PERSISTENT (computed when data is inserted
or stored in a table) orVIRTUAL (like aVIEW)
• Similar to MS SQL or Oracle
• https://kb.askmonty.org/en/virtual-columns/
112
118. Extended keys
• Default is extended_keys=on
• Extended Keys, introduced in MariaDB 5.5, is an
optimization which makes use of existing
components of InnoDB/XtraDB keys to generate
more efficient execution plans. Using these
components in many cases allows the server to
generate execution plans which employ index-
only look-ups.
• https://mariadb.com/kb/en/extended-keys/
118
119. NoSQL: HandlerSocket
• Comes with
HandlerSocket
• direct
access to
XtraDB/
InnoDB for
CRUD
operations
• INSTALL
PLUGIN
handlersoc
ket
SONAME
'handlersoc
ket.so';
• SQL:
105,000
qps (60%
usr, 28%
sys)
• memcached
: 420,000
qps (8% usr,
88% sys)
• HandlerSoc
ket:
750,000
qps (45%
usr, 53%
sys)
119
120. NoSQL: dynamic
columns
• Allows you to create virtual columns with dynamic content for
each row in table
• Basically a BLOB with handling functions
• Store different attributes for each item (like a web store). Hard to
do relationally
• In MariaDB 10.0: name support (instead of referring to columns by
numbers, name it), convert all dynamic column content to JSON
array, interface with Cassandra
• INSERT INTO tbl SET
dyncol_blob=COLUMN_CREATE("column_name", "value");
• https://kb.askmonty.org/en/dynamic-columns/
120
122. auth_socket
• Authenticates against the Unix socket file
• Uses so_peercred socket option to
obtain information about user running
client
•CREATE USER
‘monty’@‘localhost’
IDENTIFIED with auth_socket;
122
124. Let’s get somethings
out of the way
• PAM = Pluggable Authentication Module
• Use pam_ldap to to authenticate
credentials against LDAP server —
configure /etc/pam_ldap.conf (you also
obviously need /etc/ldap.conf)
• Simplest way is of course /etc/shadow auth
124
125. MariaDB
INSTALL SONAME ‘auth_pam’;
CREATE USER byte IDENTIFIED via pam
USING ‘mariadb’;
Edit /etc/pam.d/mariadb:
auth required
pam_unix.so
account required
pam_unix.so
125
126. For MySQL
compatibility
• Just use —pam-use-cleartext-
plugin for MySQL to use
mysql_cleartext_password instead of dialog
plugin
126
127. Possible errors
• Connectors don’t support it:
• Client does not support authentication
protocol requested by server; consider
upgrading MySQL client.
• You really have to re-compile connector
using libmysqlclient to have said support
127
129. Why MariaDB 10.0?
• The 5.5 merge took about a year (!)
• We (MariaDB-5.5) have over 1.5 million
lines of extra code with a ~61MB diff
• We didn’t want to repeat this for 5.6
• Also, MySQL 5.6 has a lot of re-factoring,
thus loosing commit history
129
130. In a nutshell
• Built on MariaDB 5.5
• Backported features from MySQL 5.6
• New features
130
131. What about tools?
• SELECTVERSION() will
return 10.0.1-MariaDB
• Oops, we found a bug in
MySQL: https://
mariadb.atlassian.net/
browse/MDEV-4088 &
http://bugs.mysql.com/
bug.php?id=68187
• Still deciding:
• Use 9.0 for a name
• Lie to clients (no)
• Disallow replication
(no)
• Use handshake packet
5.5.30-mysql-10.0.2-
MariaDB without
affectingVERSION() /
@@global.version
131
132. What about tools II?
• Tools really should recognise MariaDB
version as there are already many new
features that MySQL doesn’t have
• eg. HeidiSQL supports virtual columns
(http://www.heidisql.com/forum.php?
t=8671)
132
134. InnoDB & XtraDB
• MariaDB 10.0 ships InnoDB from MySQL 5.6
• MariaDB 10 ships Percona XtraDB as default
• minimal performance improvements expected,
just functionality & features
• bitmap changed page tracking so xtrabackup
can do incremental backups without scanning
all InnoDB files
• SHOW GLOBALVARIABLES LIKE 'innodb_ver%';
134
135. More from MySQL 5.6
• PERFORMANCE_SCHEMA
• InnoDB read-only transactions (TRANSACTION READ
ONLY)
• Optimizer:
• EXISTS-TO-IN optimization
• ORDER BY...LIMIT optimization (show only few rows
of a result set)
• CURRENT_TIMESTAMP as DEFAULT for DATETIME
columns (this is a re-implementation in MariaDB)
135
137. MariaDB 10 replication
• Global Transaction ID
• have complex replication topologies; simple failover & slave promotion
• doesn’t require restarts!
• new slave provisioning: SET GLOBAL GTID_SLAVE_POS =
BINLOG_GTID_POS("masterbin.00045", 600); CHANGE MASTER TO
master_host="192.168.2.4", master_use_gtid=slave_pos; START
SLAVE;
• turning on GTID for slaves: STOP SLAVE
CHANGE MASTER TO master_use_gtid=current_pos; START SLAVE;
• change masters: STOP SLAVE
CHANGE MASTER TO master_host="10.2.3.5"; START SLAVE;
• Crash-safe slaves - GTID position stored in InnoDB table
137
138. Automatic binlog position for master failover
• On Server2: CHANGE MASTER TO master_host=’server2’, master_use_gtid=1;
139. Why different GTID
compared to 5.6?
• MySQL 5.6 GTID does not support multi-
source replication
• Supports —log-slave-updates=0 for
efficiency
• Enabled by default, with self-healing
capabilities
139
140. Binlog (size matters!)
• Example query: INSERT INTO t1VALUES
(10,“foo”);
• MySQL 5.6… 265 bytes
• MariaDB 10.0… 161 bytes
• Do you want a 60% larger binlog size?
140
141. Crash-safe slave (w/
InnoDB DML)
• Replace non-transactional file relay_log.info
with transactional
mysql.rpl_slave_state
• Changes to rpl_slave_state are
transactionally recovered after crash along
with user data.
141
142. Replication domains
• Keep central concept that replication is just
applying events in-order from a serial binlog
stream.
• Allow multi-source replication with
multiple active masters
• Let’s the DBA configure multiple
independent binlog streams (one per active
master: mysqld --git-domain-
142
144. Parallel replication
• Multi-source replication from different masters executed
in parallel
• Queries from different domains are executed in parallel
• Queries that are run in parallel on the master are run in
parallel on the slave (based on group commit).
• Transactions modifying the same table can be updated
in parallel on the slave!
• Supports both statement based and row based replication.
144
145. Multi-source replication
• Work from Taobao
• Many users partition data across many masters... now you
can replicate many masters to a single slave
• Great for analytical queries, complete backups, etc.
• @@default_master_connection contains current
connection name (used if connection name is not given)
• All master/slave commands take a connection name now
(like CHANGE MASTER “connection_name”, SHOW SLAVE
“connection_name” STATUS, etc.)
• https://kb.askmonty.org/en/multi-source-replication/
145
146. Only in 10.0
• SHOW EXPLAIN for
<thread_id> (https://
mariadb.com/kb/en/show-
explain/) gets the query plan
of a running statement
• EXPLAIN ANALYZE
equivalent
• Faster ALTER TABLE with
unique keys for Aria &
MyISAM
• Segmented MyISAM
keycaches (up to 64) since
MariaDB 5.2 exist too
• Per-thread memory usage
(Taobao)
• I_S.PROCESSLIST has
MEMORY_USAGE &
EXAMINED_ROWS
• SHOW STATUS has
memory usage too
146
147. SHUTDOWN
• shuts down the server; requires GRANTs
similar to mysqladmin shutdown command
• you can create an event that does a
SHUTDOWN of the server as an
example…
• https://mariadb.com/kb/en/shutdown/
147
149. CassandraSE
• Integration with NoSQL/Big Data DB,Apache Cassandra cluster,
seen as a storage engine to MariaDB
• Combine (join) data between Cassandra & MariaDB
• Write to Cassandra from SQL (SELECT, INSERT, UPDATE,
DELETE)
• CQL is great, but the goal is for you to just work with SQL, not
switch between CQL & SQL
• Data is mapped: rowkey, static columns, dynamic columns
• super columns aren’t supported
• No 1-1 direct map for data types (ref: https://kb.askmonty.org/en/
cassandra-storage-engine/)
149
150. TokuDB
• Opensource - separate MariaDB 5.5+TokuDB/
integrated in 10.0.5
• Improved insert (10-20x faster) & query speed,
compression (up to 90% space reduction),
replication performance and online schema
flexibility
• Uses Fractal Tree Indexes instead of B-Tree
• Tests & builds of TokuDB on multiple platforms
(think greater distribution)
150
151. CONNECT
• CONNECT will speak XML or even grab
data over an ODBC connection
• You can CONNECT to Oracle (via
ODBC), join results from Cassandra (via
CassandraSE) and have all your results sit in
InnoDB
• Turn on engine condition pushdown
151
152. SPIDER
• Spider has built-in sharding features
• Partitioning & XA transaction capable
• Different MariaDB instance tables handled
like it is the same instance
152
153. Engine-independent
persistent statistics
• InnoDB has persistent statistics in MySQL
5.6; we have an engine-independent version
• These statistics aren’t limited by the SE API,
and are used by query optimizer to choose
best execution plan for each statement
• Statistics collected for non-indexed
columns too (unlike InnoDB’s)
153
154. MariaDB 10.0.2
• Support for atomic writes on FusionIO
DirectFS
• Optimizer collects & can use histogram-based
statistics for non-indexed columns
• Better table discovery, so FederatedX has
assisted discovery, Sequence engine (creates
ascending/descending sequences, useful in joins)
• SHOW PLUGINS SONAME;
154
155. MariaDB 10.0.4
• SPIDER storage engine for database sharding
merged
• Audit plugin
• complete PERFORMANCE_SCHEMA
• INFORMATION_SCHEMA with upstream
defaults too
• Online ALTER for InnoDB and thread
information for in-place operations
155
156. MariaDB 10.0.5
• Parallel replication - https://mariadb.com/
kb/en/parallel-replication/
• automatically detect independent
transactions, parallel within same table,
adapts to master load, and preserves
commit ordering
• EXPLAIN in the slow query log
156
157. MariaDB 10.0.6
• Serious incompatibility and data corruption
of DATETIME and DATE types due to
get_innobase_type_from_mysql_type
refactor combined with InnoDB Online DDL
• https://mariadb.atlassian.net/browse/
MDEV-5248
• Fixed upgrades from MySQL 5.1 -> MariaDB
• Parallel replication improvements
157
158. MariaDB 10.0.7
• Mostly bug fixes, to stabilise the code
• XtraDB 5.6 merged (InnoDB still default)
• OQGraph v3 - stores data on disk,
persistent, larger graph support
• INFORMATION_SCHEMA.METADATA_L
OCK_INFO plugin to see active metadata
locks
158
159. PCRE Regular
Expressions
• Powerful REGEXP/RLIKE operator
• New operators:
• REGEXP_REPLACE(sub,pattern,replace)
• REGEXP_INSTR(sub,pattern)
• REGEXP_SUBSTR(sub,pattern)
• Works with multi-byte character sets that
MariaDB supports, including East-Asian sets
159
160. Roles
• Bundles users together, with similar
privileges - follows the SQL standard
CREATE ROLE audit_bean_counters;
GRANT SELECT ON accounts.* to
audit_bean_counters;
GRANT audit_bean_counters to
ceo;
160
161. MariaDB 10.0.9 (RC)
• InnoDB 5.6.15 (XtraDB
default; InnoDB plugin)
• Extended keys
optimization on by
default
• MASTER_GTID_WAIT(
) + @@last_gtid
• TIME casted to
DATETIME, date is
CURRENT_DATE not
0000-00-00 - SQL
standards compliant
• @@old_mode=ZER
O_DATE_TIME_CAS
T
161
162. MariaDB 10.0.10 (GA)
• audit plugin now ships
• XtraDB performance fixed incorrect
calculation of flushed pages
• TokuDB compression is now
TOKUDB_ZLIB
• Engine independent table statistics
improved
162
163. What about MySQL
5.6?
• We love the fact that many features we’ve worked on for
a long time are now in 5.6
• Optimizer enhancements
• Microseconds
• Binary log annotations
• Binary log group commit (10.0 has a newer faster version
now)
• Precise GIS
• Threadpool
163
164. What are we missing
from 5.6 currently?
• EXPLAIN output in JSON
• InnoDB memcached interface
164
165. today what do we
have…
• 30 Jun 2014 - MariaDB 10.1.0
• 17 Oct 2014 - MariaDB 10.1.1
• 7 Dec 2014 - MariaDB 10.1.2
• 2 March 2015 - MariaDB 10.1.3
• 13 April 2015 - MariaDB 10.1.4 — late ;-)
165
166. Galera Cluster
integrated
• Full integration of Galera Cluster 4 into MariaDB 10.1 — it
won’t be a separate download!
• no lost transactions
• optimisations for WAN replication
• non-blocking DDL
• no limits on transaction size
•Server version: 10.1.3-MariaDB-wsrep MariaDB
Server, wsrep_25.10.r4144
• Granular monitoring in INFORMATION_SCHEMA —
WSREP_MEMBERSHIP, WSREP_STATUS
166
167. Encryption
• Encryption: tablespace and table level encryption with support for
rolling keys using the AES algorithm
• table encryption — PAGE_ENCRYPTION=1
• tablespace encryption — encrypts everything including log files
(not the binlog)
• Overhead of ~10%
• XtraDB/InnoDB only;Aria for temporary tables
• New file_key_management now
• Pushed & documented — https://mariadb.com/kb/en/mariadb/
documentation/managing-mariadb/securing-mariadb/encryption/
table-and-tablespace-encryption/
167
168. Encryption II
• Table level encryption (Eperi)
• must choose an encryption_algorithm=aes_ctr
(or equivalents)
• Have to use the key management plugin
• loading from filesystem? Insecure.You need a key
management server (Eperi has one commercially)
• don’t forget to create keys!
• eg. openssl enc -aes-256-ctr -k mypass -P
-md sha1
168
170. Encryption IV
• Tablespace encryption (Google)
• again, you need to pick an encryption algorithm
• specify what to encrypt: innodb-encrypt-tables,
aria, aria-encrypt-tables, encrypt-
tmp-disk-tables, innodb-encrypt-log
• don’t forget key rotation:
•innodb-encryption-threads=4
•innodb-encryption-rotate-key-
age=1800
170
171. EncryptionV
• we also have tablespace scrubbing
• background process that regularly scans
through the tables and upgrades the
encryption keys
• specify in seconds when to scrub data —
https://mariadb.com/kb/en/mariadb/
xtradb-innodb-data-scrubbing/
171
172. EncryptionVI
• 10.1.3 vs 10.1.4 have changes (incompatible)
• The distinction between “tablespace encryption” and “page
encryption” was removed, now there is only one single
encryption feature.
• Per table PAGE_ENCRYPTION_KEY was renamed to
ENCRYPTION_KEY_ID.
• Global variable innodb_default_page_encryption_key become a
session innodb_default_encryption_key_id.
• Eperi code is mostly torn out. Per-table encryption implemented
via Google’s patches
• https://mariadb.com/kb/en/mariadb/table-encryption/
172
173. Optimistic parallel
replication
• Before, transactions committed in parallel on the
master could be run in parallel
• Now, more than one transaction will be
considered to be run in parallel giving another
performance boost in master-to-slave replication
• We have to check if this only will work with a 10.1
master
• Isn’t fully pushed (or documented) yet — see:
https://mariadb.atlassian.net/browse/MDEV-6676
173
175. InnoDB improvements
• Multi-threaded flush (also in 5.7, different
implementation + we’re first)
• Page compression (optimised for Flash, SSD, FusionIO)
• 64KB pages in InnoDB (old limit = 16KB).
• Defragementation (FB, backported by DaumKakao)
• Forced primary key
• If option is true, create table without primary key or
unique key where all keyparts are NOT NULL is not
accepted. Instead an error message is printed.
175
176. InnoDB WebScaleSQL
• MDEV-6936: Buffer pool list scan optimisation
• MDEV-6929: Port Facebook Prefix Index Queries Optimization
• MDEV-6932: Enable Lazy Flushing
• MDEV-6931: Page cleaner should do LRU flushing regardless of server
activity
• fixes mysql#71988, mysql#70500
• DB-746 merge clustering key is covering key for mariadb 10 (TokuDB)
• MDEV-6933: Spurious lock_wait_timeout_thread wakeup in
lock_wait_suspend_thread()
• fixes mysql#72123
176
177. Per query variables
• Long history (http://www.bytebot.net/blog/
archives/2014/05/04/per-query-variable-
settings-in-mysqlpercona-
serverwebscalesql)
•SET STATEMENT
max_statement_time=1000 FOR
SELECT name FROM name ORDER
BY name;
177
178. Statement timeouts
• from Twitter patch; re-written by monty
• MAX_STATEMENT_TIME to abort long
running queries
• We call it “query timeouts” + have a
different syntax
• https://mariadb.atlassian.net/browse/
MDEV-4427
178
179. Optimiser
enhancements
• UNION ALL without temporary tables (5.7)
• Improve ORDER BY in optimiser
• Mostly there is EXPLAIN JSON (like 5.6)*
• EXPLAIN ANALYZE with FORMAT=JSON
• includes data from the query execution itself —
this is MariaDB only
• https://mariadb.com/kb/en/mariadb/analyze-
formatjson-examples/
179
181. WebScaleSQL
• WebScaleSQL improvements
• https://mariadb.atlassian.net/browse/
MDEV-6039
• Lots of running thru AddressSanitizer
(ASan)
• Many of these also get backported to
10.0.13
181
182. Passwords
• Password validation plugin exists now
• https://mariadb.com/kb/en/mariadb/development/mariadb-
internals-documentation/password-validation/
• simple_password_check password validation plugin
• can enforce a minimum password length and guarantee that a
password contains at least a specified number of uppercase
and lowercase letters, digits, and punctuation characters.
• cracklib_password_check password validation plugin
• Allows passwords that are strong enough to pass CrackLib
test.This is the same test that pam_cracklib.so does
182
183. Audit plugin
improvements
• Monitor access, locate errors, etc.
• Connection — connect/disconnect/failed;
Query — DDL/DML+TCL/DCL; Object —
Database/Tables
• Passwords in 1.2 replaced by a placeholder
(filtered, i.e. not in audit log)
183
184. CONNECT
• CONNECT having full JSON/BSON support
• Can read filename.json files with ease
• Writing — INSERT, UPDATE, DELETE is
supported — however, if you have added/
modified values for objects or arrays, there
can be complications (similar like the XML
type object issue)
• Works with Sveta’s JSON UDFs as well
184
185. Other bits
• Slaves can execute triggers now
• Dump thread enhancements (remove binlog
lock LOCK_log) from 5.7 included (Google)
• CREATE or REPLACE for most database
objects minus indexes
• SET DEFAULT ROLE (there is a default role
now for current user)
185
186. Other bits
• FRM files are now not created for temporary
tables
• INFORMATION_SCHEMA.SYSTEM_VARIABLES
- information for system variables
• Compiled with security hardening options (fortify
source - https://mariadb.atlassian.net/browse/
MDEV-5730)
• @@sql_log_slow can now be controlled on a
session basis (not just globally)
186
187. GIS
• Full compliance for the OGC standards around GIS.
• yes, we are missing a few functions, but its likely to
improve
• MDEV-4045 Missing OGC Spatial functions.
• MDEV-60 Support for Spatial Reference systems for the
GIS data.
• MDEV-12 OpenGIS: create required tables:
GeometryColumns, related views.
• Speaking shortly, the MariaDB GIS part is now OpenGIS
compliant, and passes all the OpenGIS conformance tests
187
188. Likely
• Kerberos authentication plugin
• Audit plugin to track password changes
• IPv6/IPv4 datatype (pending review)
• Additional character sets (GB18030) for
Chinese govt mandate (pending review)
188
189. Compatibility
• Temporary tables are stored in Aria but now there
is a —default-tmp-storage-engine option
• engine_condition_pushdown flag removed (its
always on for engines that support it)
• --mysql56-temporal-format option to use the
MySQL-5.6 low level formats to store TIME,
DATETIME and TIMESTAMP types
• PERFORMANCE_SCHEMA disabled by default
like in 10.0
189
191. Support
• Five years from every release
• MariaDB 5.5 in Red Hat Enterprise Linux 7
• MariaDB 10 in SUSE Enterprise Linux 12
191
192. Benchmarks
• “Lies, damned lies, and statistics” - Mark Twain
• http://blog.mariadb.org/sysbench-oltp-
mysql-5-6-vs-mariadb-10-0/
• http://dimitrik.free.fr/blog/archives/2013/02/
mysql-performance-mysql-56-vs-mysql-55-
vs-mariadb-55.html
• Yes, we’ve gotten Oracle to notice
MariaDB :-)
192
193. Continued
commitments
• Security
• Since about two+ years now, we’re the go-to people for security
- good track record
• We don’t like regressions
• http://www.skysql.com/blogs/hartmut/nasty-innodb-regression-
mysql-5525
• http://www.skysql.com/blogs/kolbe/heads-no-more-query-cache-
partitioned-tables-mysql-5523
• We care about backward compatibility & introduce features carefully
• XtraDB innodb_adaptive_checkpoint=none|reflex|estimate|
keep_average (no more reflex...)
193
194. We really care about
quality
• Automated test suite run upon every push
• Better QA & code coverage
• MySQL test cases: 1,765
• Percona Server test cases: 1,837
• MariaDB test cases: 2,180
194
195. MariaDB deployed
“MariaDB had these same bugs that we ran into with
MySQL. However the big difference was that when we
reported these bugs, they were quickly resolved within 48
hours!” -- Dreas van Donselaar, Chief Technology Officer,
SpamExperts
B.V. after migrating over 300 servers from MySQL 5.0
to MariaDB 5.1.
“Migrating from MySQL 5.1 to MariaDB 5.2 was as simple
as removing MySQL RPMs and installing the MariaDB
packages, then running mysql_upgrade.” - Panayot Belchev,
proprietor, Host Bulgaria on providing
MariaDB to over 7,000 of their web hosting customers.
“We made the switch on Saturday --
and we’re seeing benefits already -- our
daily optimization time is down from
24 minutes to just 4 minutes” -- Ali
Watters, CEO, travelblog.org
happy users: pap.fr, Paybox Services, OLX, Jelastic,
Web of Trust,Wikipedia, Craigslist, etc.
“@nginxorg & @mariadb
have helped me save
$12000/year in
infrastructure cost. I love it!
Do more with less!” -
Ewdison Then, CEO,
Slashgear
We upgraded the support.mozilla.org
databases from Percona 5.1 to MariaDB 5.5.
One of the engineers and I had a
conversation where he mentioned that “one
of our worst performing views on SUMO is
doing waaaayyy better with the upgraded
databases”, that it “seems more stable” and
that “I stopped receiving ‘MySQL went away
or disconnected emails’ which came in once
in a while.” - Sheeri Cabral, Mozilla IT
195
197. Books!
1. MariaDB Crash Course, Ben Forta (September 2011)
2. Getting Started with MariaDB, Daniel Bartholomew (October 2013)
3. MariaDB Cookbook, Daniel Bartholomew (March 2014)
4. Real MariaDB, Matt Lee (April 2014)
5. Building a Web Application with PHP & MariaDB:A Reference Guide,
Sai Srinivas Sriparasa (June 2014)
6. MariaDB: Beginners Guide, Rodrigo Ribeiro (August 2014)
7. Mastering MariaDB, Federico Razzioli (September 2014)
8. MariaDB High Performance, Pierre Mavro (September 2014)
9. Learning MySQL & MariaDB, Russell Dyer (April 2015)
197
198. Q&A / Enjoy your
evening
colin@mariadb.org
slides: slideshare.net/bytebot
http://bytebot.net/blog/ | @bytebot on twitter
http://mariadb.org/ | http://mariadb.com/
198