MongoDB and Azure Databricks

As AWS continues to expand, enterprise customers are looking to our partner ecosystem to assist in migrating their workloads to the cloud. This session describes the challenges, lessons learned and best practices for large scale application migrations. We will use real examples from our consulting partners and AWS Professional Services to illustrate how to move workloads to the cloud while modernizing the associated applications to take advantage of AWS’ unique benefits. We will also dive into how to use an array of AWS services and features to improve a customer’s security posture as they are migrating and once they are up and running in the cloud

Webinar presentation deck for Azure Synapse. 100 level Presentation to provide overview of Synapse, capabilities etc.

Modern DW Architecture - The document discusses modern data warehouse architectures using Azure cloud services like Azure Data Lake, Azure Databricks, and Azure Synapse. It covers storage options like ADLS Gen 1 and Gen 2 and data processing tools like Databricks and Synapse. It highlights how to optimize architectures for cost and performance using features like auto-scaling, shutdown, and lifecycle management policies. Finally, it provides a demo of a sample end-to-end data pipeline.

This document outlines modules for a lab on moving data to Azure using Azure Data Factory. The modules will deploy necessary Azure resources, lift and shift an existing SSIS package to Azure, rebuild ETL processes in ADF, enhance data with cloud services, transform and merge data with ADF and HDInsight, load data into a data warehouse with ADF, schedule ADF pipelines, monitor ADF, and verify loaded data. Technologies used include PowerShell, Azure SQL, Blob Storage, Data Factory, SQL DW, Logic Apps, HDInsight, and Office 365.

Analyze key aspects to be considered before embarking on your cloud journey. The presentation outlines the strategies, approach, and choices that need to be made, to ensure a smooth transition to the cloud.

Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.

The document discusses modernizing a healthcare organization's data platform from version 1.0 to 2.0 using Azure Databricks. Version 1.0 used Azure HDInsight (HDI) which was challenging to scale and maintain. It presented performance issues and lacked integrations. Version 2.0 with Databricks will provide improved scalability, cost optimization, governance, and ease of use through features like Delta Lake, Unity Catalog, and collaborative notebooks. This will help address challenges faced by consumers, data engineers, and the client.

Jim Boriotti presents an overview and demo of Azure Synapse Analytics, an integrated data platform for business intelligence, artificial intelligence, and continuous intelligence. Azure Synapse Analytics includes Synapse SQL for querying with T-SQL, Synapse Spark for notebooks in Python, Scala, and .NET, and Synapse Pipelines for data workflows. The demo shows how Azure Synapse Analytics provides a unified environment for all data tasks through the Synapse Studio interface.

Azure Databricks is a fast, easy to use, and collaborative Apache Spark-based analytics platform optimized for Azure. It allows for interactive collaboration through a unified workspace, enables sharing of insights through integration with Power BI, and provides native integration with other Azure services. It also offers enterprise-grade security through integration with Azure Active Directory and compliance features.

So you have been running on-prem SQL Server for a while now. Maybe you have taken the step to move it from bare metal to a VM, and have seen some nice benefits. Ready to see a TON more benefits? If you said “YES!”, then this is the session for you as I will go over the many benefits gained by moving your on-prem SQL Server to an Azure VM (IaaS). Then I will really blow your mind by showing you even more benefits by moving to Azure SQL Database (PaaS/DBaaS). And for those of you with a large data warehouse, I also got you covered with Azure SQL Data Warehouse. Along the way I will talk about the many hybrid approaches so you can take a gradual approve to moving to the cloud. If you are interested in cost savings, additional features, ease of use, quick scaling, improved reliability and ending the days of upgrading hardware, this is the session for you!

- Azure provides a unified platform for modern business with compute, data, storage, networking and application services across global Azure regions and a consistent hybrid cloud. - Azure focuses on security and privacy with an emphasis on detection, response, and protection across infrastructure, platforms and applications. - Security is a shared responsibility between Microsoft and customers, with Microsoft providing security controls and capabilities to help protect customer data and applications.

The document provides an overview of a 1-day AWS Partner course on data analytics solutions on AWS. The course objectives are to identify AWS analytics services, describe data analytics architectures, discuss the AWS Data Pipeline and Data Flywheel models, and describe five technical solutions: modernizing a data warehouse with Redshift, data lakes, streaming data, data governance, and machine learning. It also notes that the course will help APN Partners engage with customers by providing sufficient technical knowledge of AWS analytics services.

Serverless computing allows running applications without managing infrastructure. Google Cloud Platform offers serverless options like Cloud Functions, Cloud Run, and App Engine. Common serverless patterns include publish-subscribe using PubSub, triggering functions from events, and data pipelines with Dataflow. Serverless applications are built using containers, functions, and fully managed services to focus on code and reduce operational overhead.

Databricks CEO Ali Ghodsi introduces Databricks Delta, a new data management system that combines the scale and cost-efficiency of a data lake, the performance and reliability of a data warehouse, and the low latency of streaming.

During this Big Data Warehousing Meetup, Caserta Concepts and Databricks addressed the number one operational and analytic goal of nearly every organization today – to have complete view of every customer. Customer Data Integration (CDI) must be implemented to cleanse and match customer identities within and across various data systems. CDI has been a long-standing data engineering challenge, not just one of logic and complexity but also of performance and scalability. The speakers brought together best practice techniques with Apache Spark to achieve complete CDI. Speakers: Joe Caserta, President, Caserta Concepts Kevin Rasmussen, Big Data Engineer, Caserta Concepts Vida Ha, Lead Solutions Engineer, Databricks The sessions covered a series of problems that are adequately solved with Apache Spark, as well as those that are require additional technologies to implement correctly. Topics included: · Building an end-to-end CDI pipeline in Apache Spark · What works, what doesn’t, and how do we use Spark we evolve · Innovation with Spark including methods for customer matching from statistical patterns, geolocation, and behavior · Using Pyspark and Python’s rich module ecosystem for data cleansing and standardization matching · Using GraphX for matching and scalable clustering · Analyzing large data files with Spark · Using Spark for ETL on large datasets · Applying Machine Learning & Data Science to large datasets · Connecting BI/Visualization tools to Apache Spark to analyze large datasets internally The speakers also touched on data governance, on-boarding new data rapidly, how to balance rapid agility and time to market with critical decision support and customer interaction. They also shared examples of problems that Apache Spark is not optimized for. For more information on the services offered by Caserta Concepts, visit our website: http://casertaconcepts.com/

This document provides an overview of Mustafa Kara's background and expertise in datacenter transformation. It discusses his 10 years of experience in roles such as senior consultant, Azure MVP, technical manager, and technical trainer. It then outlines his work as a speaker and writer for Microsoft events, Virtual Academy, universities, and personal websites. The rest of the document discusses strategies for transforming the datacenter, including moving from on-premises physical servers and VMs to a hybrid cloud model using public cloud off-premises and cloud on-premises. It highlights tools like Azure Migrate and database migration services that can help analyze costs and migrate applications, VMs, and data.

This document provides an overview of Microsoft Azure cloud services and why businesses use the cloud. It discusses Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS) models. Key Azure services are mentioned, including Virtual Machines, SQL Database, storage, and web apps. The cloud allows businesses to rapidly setup environments, scale as needed, and increase efficiency at a lower cost compared to on-premises infrastructure.

Participants will get a deep dive into one of Azure’s newest offering: Azure Databricks, a fast, easy and collaborative Apache® Spark™ based analytics platform optimized for Azure. In this session, we start with a technical overview of Spark and quickly jump into Azure Databricks’ key collaboration features, cluster management, and tight data integration with Azure data sources. Concepts are made concrete via a detailed walk through of an advance analytics pipeline built using Spark and Azure Databricks. Full video of the presentation: https://www.youtube.com/watch?v=14D9VzI152o Presentation demo: https://github.com/devlace/azure-databricks-anomaly

Spark is an open-source framework for large-scale data processing. Azure Databricks provides Spark as a managed service on Microsoft Azure, allowing users to deploy production Spark jobs and workflows without having to manage infrastructure. It offers an optimized Databricks runtime, collaborative workspace, and integrations with other Azure services to enhance productivity and scale workloads without limits.

Presented at: Global Azure Bootcamp (Melbourne) Participants will get a deep dive into one of Azure’s newest offering: Azure Databricks, a fast, easy and collaborative Apache® Spark™ based analytics platform optimized for Azure. In this session, we will go through Azure Databricks key collaboration features, cluster management, and tight data integration with Azure data sources. We’ll also walk through an end-to-end Recommendation System Data Pipeline built using Spark on Azure Databricks.

Slides of the presentation by Heather Grandy at C# Corner Toronto chapter Feb 2019 meetup on Azure Databricks Spark