Big Data in the Microsoft Platform
•Download as PPTX, PDF•
1 like•385 views
This document provides an overview of Hadoop ecosystem components including HDFS for distributed storage, MapReduce for distributed processing, Pig and Hive for data analysis languages, Sqoop for data transfer, HBase for NoSQL storage, Zookeeper for coordination, and HDInsight as a cloud-based Hadoop service. It also mentions demos of HDFS, MapReduce, Hive, Pig, and Sqoop to showcase bulk data loading and processing capabilities.
Report
Share
Big Data in the Microsoft Platform
- 8. • • • • •
- 9. • • • • •
- 21. ETL Tools BI Reporting RDBMS Zookeepr (Coordination) Pig (Data Flow) Hive (SQL) Sqoop Avro (Serialization) MapReduce (Job Scheduling/Execution System) HBase (key-value store) (Streaming/Pipes APIs) HDFS (Hadoop Distributed File System)
- 26. Block Size = 64MB Replication Factor = 3 Cost/GB is a few ¢/month vs $/month
- 27. HDFS Demo
- 29. • • •
- 31. Map Reduce Demo
- 38.
- 39. Rocking Data Processing with Pig
- 43. Bulk Data Loading Using Sqoop
- 46. HDInsight Service Overview