Scylla Summit 2017: Planning Your Queries for Maximum Performance

ScyllaDB CEO and co-founder Dor Laor shares his vision for Scylla and announces Scylla 2.0, a big step towards the first autonomous NoSQL database—one that dynamically tunes itself to varying conditions while always maintaining a high level of performance.

This presentation discusses the "cold node problem" that occurs when a node restarts in a Cassandra cluster. When a node restarts, it loses its cached data and becomes a bottleneck. The presentation proposes a "heat weighted load balancing" solution where the cluster tracks each node's cache hit ratio and redistributes requests based on this ratio after a restart. Testing shows this solution significantly improves throughput after a node restart by distributing requests more evenly across nodes based on their "heat" or cache contents.

Duarte Nunes presented on distributed materialized views in ScyllaDB. He discussed the challenges of implementing materialized views in a distributed system without a single master, including propagating updates from base tables to views, handling consistency when tables can diverge, and managing concurrent updates safely. His proposed solution uses asynchronous replica-based propagation paired with repair mechanisms and locking or optimistic concurrency to address these issues. Materialized views provide powerful indexing capabilities but also introduce performance overhead that is difficult to avoid given Scylla's data model.

AdGear runs an ad tech gateway at more than one million queries per second to Scylla and recently transitioned from Apache Cassandra. In this talk, we will highlight the tools and languages that we use (Erlang), how we do bulk imports, and how performance compares between the two database engines.

Kubernetes is a declarative system for automatically deploying, managing, and scaling applications and their dependencies. In this short talk, I'll demonstrate a small Scylla cluster running in Google Compute Engine via Kubernetes and our publicly-published Docker images.

I will be giving a talk about performance characterization and tuning of Scylla on Samsung NVMe SSDs. We will characterize the performance of Scylla on Samsung high-performance NVMe SSDs and show how Z-SSD ─ the Samsung ultra-low-latency NVMe drive ─ can significantly shrink the performance gap between in-memory and in-storage with Scylla. We will further evaluate the throughput-vs-latency profile of Scylla with NVMe devices and present end-to-end latencies (from the client's viewpoint) as well as the latencies of the software/hardware stack. We will show that a Z-SSD-backed Scylla cluster can provide competitive performance to an in-memory deployment while sharply reducing costs.

Shlomi Livne, VP of R&D at ScyllaDB, presented on the performance benefits of using user-defined types (UDTs) in ScyllaDB. He explained that with traditional columns, each column has overhead and flexibility comes at a price. However, with frozen UDTs, the columns are treated as a single unit, sharing metadata and improving performance. Livne showed results of a test where UDTs with many fields outperformed traditional columns with the same number of fields. However, he noted that Scylla's row cache and Java driver performance need improvement for UDTs.

In my talk, I will present the different compaction strategies that Scylla provides, and demonstrate when it is appropriate and when it is inappropriate to use each one. I will then present a new compaction strategy that we designed as a lesson from the existing compaction strategies by picking the best features of the existing strategies while avoiding their problems.

ScyllaDB CTO Avi Kivity gave a keynote on how Scylla has evolved. He discussed new features in Scylla 2.0—including Materialized Views and Heat-Weighted Load Balancing, changes in monitoring—and shared our product roadmap. He also talked about our recent acquisition of Seastar.io and how it will enable us to deliver a database-as-a-service offering.

In this presentation, I'll speak of the benefits of running Scylla on our Big Data environment which stores over 500TB of data as well as using Scylla as the indexing engine to replace MongoDB and Cassandra for our log data analysis platform.

The document appears to be a presentation on optimizing inter-data center communication. It discusses key topics like what inter-data center communication involves, the costs associated with it, best practices for setting snitches, keyspaces, client drivers and consistency levels for queries to optimize performance between data centers. It recommends using network topology replication strategies over simple strategies for multi-region deployments, setting load balancing and consistency levels appropriately in clients, and enabling internode compression to reduce costs of communication between data centers. The presentation encourages reviewing client locations, data access patterns, who is reading/writing data, and having conversations between operations and development teams to determine the best use cases.

Scylla's monitoring capability has come a long way in the last year. We now have native support for Prometheus. Through scylla-grafana-monitoring, we have started providing default dashboards summarizing the most important aspects of Scylla for users. In this talk, I will cover what is currently available in our metrics, other non-standard metrics that are interesting but not available in our main dashboard, as well as our future plans for enhancement.

Apache Kafka is a high-throughput distributed streaming platform that is being adopted by hundreds of companies to manage their real-time data. KSQL is an open source streaming SQL engine that implements continuous, interactive queries against Apache Kafka™. KSQL makes it easy to read, write and process streaming data in real-time, at scale, using SQL-like semantics. In my talk, I will discuss streaming ETL from Kafka into stores like Apache Cassandra using KSQL.

In this talk, we will share useful tools and techniques that we are using in the field to understand Scylla clusters. Users will learn how to use those same tools to better understand their deployment. Some of the questions that will be answered are: - how to find out which queries are the slowest and why - how we go about understanding the impact of the data model in a node's performance - how to check which resources are the bottlenecks in the cluster

JanusGraph, a highly scalable graph database solution, supports historically Cassandra and HBase as database backends. We decided to put Scylla in the mix, certainly searching for the best performing backend. We ran test scenarios that cover high volume reads and writes. In this talk, we will show you the performance results of Scylla vs others and also share our lessons learned during the performance evaluation.

Scylla and Spotinst together provide a strong combination of extreme performance and cost reduction. In this talk, we will present how a Scylla cluster can be used on AWS’s EC2 Spot without losing consistency with the help of Spotinst prediction technology and advanced stateful features. We will show a live demo on how to run Scylla on the Spotinst platform.

Our CEO and co-founder Dor Laor and our chairman Benny Schnaider sharing their vision for Scylla. This was also our opportunity to announce Scylla 2.0. Our latest release is a big step toward the first autonomous NoSQL database—one that dynamically tunes itself to varying conditions while always maintaining a high level of performance.

The document discusses new features and improvements in the MySQL 8.0 optimizer. Key highlights include: - New SQL syntax like SELECT...FOR UPDATE SKIP LOCKED and NOWAIT to handle row locking contention. - Support for common table expressions to improve readability and allow referencing derived tables multiple times. - Enhancements to the cost model to produce more accurate estimates based on factors like data location. - Better support for data types like UUID and IPv6, including optimized storage formats and new functions.

Apache Impala is an exceptional, best-of-breed massively parallel processing SQL query engine that is a fundamental component of the big data software stack. Juan Yu demystifies the cost model Impala Planner uses and how Impala optimizes queries and explains how to identify performance bottleneck through query plan and profile and how to drive Impala to its full potential.

The document summarizes best practices for migrating legacy data warehouses to Amazon Redshift. It covers architectural concepts like columnar storage and compression, data distribution styles, sort keys to optimize query performance, and materializing dimension columns in fact tables. The presentation provides an overview of these topics and their impact on storage, I/O and querying. Real-world examples are also given to illustrate key points.

The document discusses query execution in database management systems. It begins with an example query on a City, Country database and represents it in relational algebra. It then discusses different query execution strategies like table scan, nested loop join, sort merge join, and hash join. The strategies are compared based on their memory and disk I/O requirements. The document emphasizes that query execution plans can be optimized for parallelism and pipelining to improve performance.

STALKER is a machine learning algorithm that learns to extract data from web pages using a small number of labeled examples provided by the user. It generates extraction rules in a hierarchical manner, exploiting the structure of the web source. The algorithm is efficient because most web pages have a fixed template with few variations. It also uses an active learning approach called co-testing to select the most informative examples for the user to label. The system verifies extracted data by comparing it to learned statistical patterns, and can automatically repair wrappers when sites change.

The document contains log output from a plugin making calls to load, write, and render PDF files from various URLs and local files. It initializes the plugin, opens streams for the PDF files, writes the stream data in chunks, and finally destroys the streams and plugin instance. This process is repeated for multiple PDF files loaded by the plugin.

The document provides an overview of various Oracle tips and tricks, including CASE statements, joins, timestamps, renaming tables/columns, merge statements, subqueries, window functions, hierarchical queries, XML, grouping sets, rollups and cubes, indexes, temporary tables and more. Key features introduced in Oracle 9i such as the CASE statement, full outer joins, timestamps and the WITH clause are highlighted.

After completing this lesson, you should be able to do the following: Describe a view Create a view Retrieve data through a view Alter the definition of a view Insert, update, and delete data through a view Drop a view

This document provides examples of SQL queries using aggregation functions such as SUM, AVG, MIN, MAX, and COUNT. It demonstrates how to use aggregation functions to calculate values across entire tables or groups of rows. It also shows how to use the GROUP BY clause to aggregate values for each unique value in a column, and the HAVING clause to filter groups based on aggregation results. Proper order of operations for aggregation queries is also discussed.

This document provides an overview of the MySQL query optimizer. It discusses the main phases of the optimizer including logical transformations, cost-based optimizations, analyzing access methods, join ordering, and plan refinements. Logical transformations prepare the query for cost-based optimization by simplifying conditions. Cost-based optimizations select the optimal join order and access methods to minimize resources used. Access methods analyzed include table scans, index scans, and ref access. The join optimizer searches for the best join order. Plan refinements include sort avoidance and index condition pushdown.