Azure Data Lake Intro (SQLBits 2016)

Editor's Notes

1. The opportunity – more data than ever before to use The challenge – how to find value in that data, structured/unstructured
2. The Data Warehouses leverages the top-down approach where there is a well-architected information store and enterprisewide BI solution. To build a data warehouse follows the top-down approach where the company’s corporate strategy is defined first. This is followed by gathering of business and technical requirements for the warehouse. The data warehouse is then implemented by dimension modelling and ETL design followed by the actual development of the warehouse. This is all done prior to any data being collected. It utilizes a rigorous and formalized methodology because a true enterprise data warehouse supports many users/applications within an organization to make better decisions.
3. A data lake is an enterprise wide repository of every type of data collected in a single place. Data of all types can be arbitrarily stored in the data lake prior to any formal definition of requirements or schema for the purposes of operational and exploratory analytics. Advanced analytics can be done using Hadoop, Machine Learning tools, or act as a lower cost data preparation location prior to moving curated data into a data warehouse. In these cases, customers would load data into the data lake prior to defining any transformation logic. This is bottom up because data is collected first and the data itself gives you the insight and helps derive conclusions or predictive models.
4. Other points to make here, but not called out above Built on Apache YARN Scales dynamically with the turn of a dial Supports Azure AD for access control, roles, and integration with on-prem identity systems U-SQL’s scalable runtime processes data across multiple Azure data sources
5. ADLA allows you to compute on data anywhere and a join data from multiple cloud sources.
6. Tell them what they need to know in order to create this sample (pre-reqs)