State of MariaDB
- 1. MariaDB
State of MariaDB
Michael “Monty” Widenius
MariaDB hacker
monty@askmonty.org
http://mariadb.com/
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 2. The origin of My (SQL)
At start: Lots of traveling and meeting interesting people
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 3. The origin of My (SQL)
Combined with working from home
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 4. The origin of My (SQL)
Things were good
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 5. The origin of My (SQL)
Even if there were some growing pains
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 6. The origin of My (SQL)
Then we came into strange company
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 7. The origin of My (SQL)
Which scared some of us a bit...
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 8. The origin of My (SQL) and Maria (DB)
Fortunately there is someone else that can continue
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 9. The origin of My
While My continues to travel
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 10. The origin of My
Free as a bird
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 11. The origin of Maria (DB)
But is the next generation up to it?
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 12. The origin of Maria (DB)
It's a hard job taking over a success
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 13. The origin of Maria (DB)
Like pulling out a rabbit from a hat
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 14. The origin of Maria (DB)
Just charm is not enough
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 15. The origin of Maria (DB)
But we are confident we can pull it off
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 16. The origin of Maria (DB)
So lets be happy that...
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 17. The origin of Maria (DB)
We still can put our trust in the creators of MySQL
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 18. The team behind MariaDB at MP
The project name has changed, but the core team is the same
(Please ignore a couple of innocent bystanders)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 19. Mug shot of the guilty
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 20. The team behind MariaDB at MP
Igor Babaev Timour Katchaounov Sergey Petrunya
Since 2002 Since 2002 Since 2003
Optimizer team
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 21. The team behind MariaDB at MP
Sergei Goulibchik Igor Babaev Michael “Monty” Widenius
Since 2000 Since 2002 Since 1993
The Original Architects
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 22. The team behind MariaDB
Kristian Nielsen
Since 2005
Replication (and build) team
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 23. The team behind MariaDB at MP
Sanja Byelkin Jani Tolonen
Since 2001 Since 1998
General Team (Optimizer, runtime, engines etc)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 24. The team behind MariaDB at MP
Vladislav “Wlad” Vaintroub Alexey “Holyfoot” Botchkov
Since 2008 Since 2008
Windows, Performance & GIS
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 25. The team behind MariaDB at MP
Hakan Küçükyılmaz Philip Stoev
Since 2004 Since 2007
QA and performance team
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 26. The team behind MariaDB at MP
Colin Charles Kurt von Finck
Since 2005 “The new guy”
Photo by James Duncan Davidson
Community team
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 27. The team behind MariaDB at MP
Bryan Alsdorf Daniel Barholomew
Since 2004 Another new guy
Knowledgebase and Web
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 28. The team behind MariaDB at MP
Rasmus Johansson Camilla Zilliacus
COO, Since 2010 Admin, since 2002
And of course, someone has to take care of this merry group...
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 29. There are a lot of others involved
● Most features in MariaDB 5.2, released as stable in
November 2010, were contributed by the community!
● In the askmonty.org knowledge base (free MariaDB and
MySQL documentation) we have now 1200+ articles
Statistics from the past month:
● Added/Changed Articles: 58
● On Freenode #maria, 227 people wrote 6085 lines
● Launchpad Activity:
● 15 active branches
● 210 commits by 10 people
● Emails: 117 (discuss, dev)
● Hundreds of thousands of downloads of MariaDB 5.2 (since
release)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 30. Free MySQL / MariaDB documentation
The knowledgebase allows you to:
● Find answers to your problems
● Ask questions and get answers from others
● Add your own documentation.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 31. User & partners
● We have seen companies converting hundreds of machines
to MariaDB in a few days without any problems.
● We are working with them to produce success stories of these.
● Monty Program Ab has 30+ partners involved in MySQL and
MariaDB
● Several companies are now sponsoring features for MariaDB!
● Monty Program Ab sponsors MariaDB development with 50 % of our
developers time!
● Some companies with MySQL expertise internally have
signed direct support contracts with Monty Program Ab; All
other support are done through partners.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 32. SkySQL
● Monty Program Ab has now signed a strategic partnership
rd
with SkySQL to be their 3 level support and main
development department.
This is important for me personally because:
● Monty Program Ab can provide a home for MySQL
developers, but not for support, training and consulting.
● The MySQL ecosystem need support companies like
SkySQL, were the first person you talk to knows more than
you about MySQL / MariaDB.
When MySQL AB joined with Sun we were 400-450 persons.
I was just told from someone that there are only 54 left at
Oracle! Much fewer than I would expect.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 33. When it's time to change support
It's time to change support providers when you get answers
like (without a clear reason why this is likely to fix the problem):
● Restart your database server
● Restart your machine
● Dump and restore your data
● Install a new major version and see if it helps.
● We don't support older MySQL versions (or MariaDB)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 34. Is MariaDB a branch or a fork?
Branch or fork, that are the answers. What is the question?
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 35. MariaDB is now a forsch (or brock?)
We have forked the MySQL client library (C connector) because :
● Oracle removed the original FOSS exception from the C
connector files and replaced it with a link to a web page that
can change any time and the text is extremely hard to parse.
For the main server, we are still doing merges to ensure
MariaDB continues to be a drop in replacement of MySQL.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 36. Current state of C connector
● We re-released the 3.23 LGPL client
● Works for many applications except:
● Not link-level compatible with the newest one (you have to recompile)
● Does not support the new authentication protocol
● The MariaDB source code and releases contains a C
connector with still has the original FOSS exception (we don't
apply new code from Oracle to it)
● We have created a new LGPL C Connector that will be
released shortly (this was sponsored by several companies)
● Alpha in April
● Beta in May
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 37. Why a free C connector is important
MySQL Server (GPL)
MySQL Client-
Server Protocol
/C /J /C /C
Connector
(GPL) JDBC Perl
ODBC
DBI
Standard Interfaces
(not GPL)
Not bound by
GPL
The C connector is the only MySQL/MariaDB connector where
the GPL affects your application. All others are free to use with
any program! (No commercial licenses)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 38. MariaDB server is a branch of MySQL
● User level (data, API, replication..) compatible with MySQL
● Drop in replacement (MariaDB 5.1 is even more
compatible with MySQL 5.0 than MySQL 5.1).
● More plugins, more features, faster, better code quality.
● GPL-only license. C Client library with FOSS exception.
● More open development
● Source in public repository on launchpad
● Active external contributors
● All development plans public on askmonty.org
● Current state
● MariaDB 5.2 was released as stable in November 2010
● MariaDB 5.3 out in alpha. Beta during April.
● MariaDB 5.5 beta planned for May.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 39. Major new features in MariaDB 5.2
● SphinxSE: Text search within MariaDB
● Built-in Sphinx client which allows MariaDB to talk to searchd, run
search queries, and obtain search results.
● Virtual columns
● Columns that are an expression and are calculated on retrieval.
● Extended User Statistics
● Client, User, Index and Table statistics.
● Segmented MyISAM key cache (see separate slide)
● Pluggable Authentication
● Storage-engine-specific CREATE TABLE
● Very fast 'copying to temp table' phase.
● Group commit & better recovery for the Aria engine.
● Speeds up multi-user inserts.
Blue means developed by the community
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 40. MyISAM Segmented key cache
● Blue line is without segmented key cache.
● Solves one of the major read bottlenecks for MyISAM
● We see up to 250% performance gain depending on the
amount of concurrent users.
● Fix applies to all MyISAM usage with many readers!
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 41. MariaDB 5.3 & 5.5
● 5.3 estimated to be in beta (all new features pushed) this month
● 5.3 → 5.5 merge is in the works. Estimated to be finished in
April/May.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 42. What's new in MariaDB 5.3
This is the biggest redesign of the MariaDB optimizer in 10 years
and it will finally make all subqueries usable in MariaDB.
● Faster subqueries
● Back porting and extending subquery optimization from
MySQL 6.0
● No materialization for many kinds of subqueries or VIEW's in
the FROM clause. SELECT * from (SELECT ....)
● Caching of subquery results
In applicable cases, you can get 10x – 100x speedups.
● Faster joins (of big tables) thanks to
● Multi-Read-Range (MRR) access (better than in MySQL 5.6)
● Batch key access (BKA)
● Index condition pushdown
● Classic Hash joins
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 43. What's new in MariaDB 5.3
Some common sub queries that are now significantly faster:
● No materialization or materialization with keys:
● SELECT anything FROM (SELECT ....) AS a WHERE a=...
● Caching of common values (Good if outer_ref has a few values)
● SELECT (SELECT ... WHERE outer_ref=xxx) FROM ...
● Transformations
● SELECT * FROM big_table WHERE big_table.col IN
(SELECT anything FROM small_table) ->
Reorders SELECT:s to use sub query as driving table
● Materialization with keys in temporary table also for WHERE
● SELECT ... WHERE a [NOT] IN (SELECT not-a-key ...)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 44. New Batched Key Access Speedups
● Join benchmark with BKA
3000
2500
query time, sec (smaller is better)
2000
BKA,
join_buffer_size=32M
1500
10x Regular join
7x
1000
500
3x
0
query_size=1 query_size=2 query_size=3
select max(l_extendedprice) from orders, lineitem where
o_orderdate between $DATE1 and $DATE2 and
l_orderkey=o_orderkey
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 45. What's new in MariaDB 5.3
● Full microsecond support. This includes TIMESTAMP, TIME
DATETIME types, NOW() and all CAST and TIME related
functions, replication etc.
● Group commit between binary log and storage engines
● FASTER and safer replication
● Lots of small optimizations, code cleanups, better error
messages and bug fixes.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 46. Group commit scales well!
Commits per second vs. number of connections, RAID 1 HDD
•Yellow line shows group commit performance
•Now get scalability, only pay the cost of the 3 * fsync()
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 47. MariaDB 5.3 and NoSQL
The main reasons for using NoSQL are:
● Handling of unstructured data (not everything is table and
fixed number of columns)
● Faster replication (usually with 'unconventional' shortcuts)
● The same way MySQL with it's storage engine interface can
handle both transactional and datawarehousing , we are
extending MariaDB to be a bridge between SQL and NoSQL.
● MariaDB 5.3 has now even better “NoSQL” support:
● Faster HANDLER commands; HANDLER READ now also
work with prepared statements.
● HandlerSocket compiled in (Direct access to InnoDB)
● Dynamic columns (each row can have different set of
columns)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 48. HANDLER READ improvements
● Streamlined HANDLER READ interface
● Added support for prepared statements
● Added support for MEMORY tables.
Effect is:
● All HANDLER READ calls are now 7% faster
● 20-50% speedup when using prepared statements
and better concurrency.
● You can now get up to 530,000 queries/second
trough SQL with NO-SQL commands (60% of
HandlerSocket).
Stephane Varoqui's blog:
http://varokism.blogspot.com/2011/01/20-to-50-improvement-in-mariadb-53.html
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 49. SQL doesn't solve all common problems
The (web) store problem:
All items needs: ID, Type, Price, Country, Manufacturer)
A T-Shirt has the following additional properties:
Size, color...
A computer has the following additional properties:
CPU, MHz, memory, Watt...
There is no easy way to store many different types into a
relational database!
(It will not work by having one table/types as joins becomes
impossible to manage).
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 50. Dynamic columns in MariaDB 5.3
● With dynamic columns all extra columns are stored in one
or many packed blobs, maintained by the database.
● You can instantly add more columns, remove or query
them for a row.
● You can access columns in the server or retrieve the full
blob to the client and manipulate it there.
● You can use virtual columns to create indexes on some
values.
● True indexes for dynamic columns is planned for later.
● Implemented trough functions to enable use by ODBC etc.
● First implementation uses an integer to access columns.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 51. Dynamic columns in MariaDB 5.3
● Simple set of functions (available in server and client):
● COLUMN_CREATE(column_nr, value,[column_nr,....])
● COLUMN_ADD(blob,column_nr, value, [column_nr,...])
● COLUMN_DELETE(blob, column_nr, column_nr...);
● COLUMN_EXISTS(blob, column_nr);
● COLUMN_LIST(blob, column_nr);
● COLUMN_GET(blob, column_nr, type);
As a proof of concept we plan to create an experimental
storage engine for HBASE where we use dynamic columns
as a bridge.
More about Dynamic columns at today's talk at 3.05 PM.
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 52. What are planned for MariaDB 5.6 ?
At the Lisbon MariaDB meeting we created the following list of
proposed features. This list is still work in progress, contact us if
you want to get something added / ensure something is done!
● GIS
● OpenGIS compliance
● Deeper integration of GIS with optimizer
● More online operations
● Analyze table
● ALTER ONLINE TABLE
● Compatibility & usability
● IPv6
● Query logging and summary per query
● Audit for specific users
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 53. What is planned for MariaDB 5.6 ?
● Replication
● Extend group commit to have on sync per group
commit
● Global transaction id
● Parallel applying of binary log in slave
● Statistics and monitoring
● Phone home (code done for 5.3)
● Better EXPLAIN
● Persistent table statistics
● Log all SQL errors
● Progress indicator for LOAD DATA and SELECT
● Better monitoring for replication
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 54. What is planned for MariaDB 5.6 ?
● Optimizer
● Implement UNION ALL without usage of a temporary table
● Grace HASH join (Need sponsor)
● Sort merge join (Need sponsor)
● Performance
● Better multi CPU performance above 16 cores
● Better thread pool (we already have a prototype for this)
● More scalable query cache under higher concurrency
● Optionally with stale data
● Faster VIEW (don't open & parse view for every query)
● Easy of use
● VARCHAR and BLOB support for memory tables
● Table functions
For full list, see http://kb.askmonty.org/v/plans-for-56
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 55. New thread pool for 5.6
Thread pools solves a couple of problems:
● Allows you to limit the number of worker threads at
your machines peek performance.
● More fair scheduler; Less query time distribution
● If too many queries, machine can run at 1% of
peek performance
● New tread pool for 5.6: (Tested with 24 CPU's):
● Always better on Windows
● Better one Linux than thread-per-connection after
1024 connections
● Much less performance degradation when more
connections (60% performance instead of 1%)
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 56. Announcements
We are arranging a Plugin & Storage Engine Summit
on Friday, to be held at Facebook.
http://kb.askmonty.org/v/storage-engine-summit-2011
● Contact Sergei, Colin or me if you are interested!
We are launching an MariaDB ambassador program
to spread knowledge about MariaDB / MySQL.
http://montyprogram.com/ambassadors/
We have a T-Shirt + dinner auction to support Japan
at Monty Program Ab boot #411
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.
- 57. Come to the MariaDB boot & Bof
● You can find MariaDB developers at the MariaDB /
Monty Program Ab booth #411.
● There is a Bof about the 5.6 features at 8.30 pm
Please come by if you want to
● Know more about MariaDB
● Participate in its development
● Sponsor features
● Get support for MySQL or MariaDB!
There is also a lot of sessions today and tomorrow
about MariaDB!
Notice: MySQL is a registered trademark of Sun Microsystems, Inc.