Data Science & Big Data - Theory.pdf
•
0 likes•186 views
This document outlines the curriculum for the course "Elective Theory II - Data Science and Big Data" for the VI semester of the Diploma in Computer Engineering program. The course covers 5 units over 80 hours on data science fundamentals, data modeling, and big data concepts including storage and processing. The objectives are to understand data science techniques, apply data analysis in Python and Excel, learn about big data characteristics and technologies like Hadoop, and explore applications of big data. Topics include linear regression, classification models, MapReduce, and using big data in fields such as marketing, healthcare, and advertising.
Report
Share
More Related Content
Similar to Data Science & Big Data - Theory.pdf
Similar to Data Science & Big Data - Theory.pdf (20)
Recently uploaded
Recently uploaded (20)
Data Science & Big Data - Theory.pdf
- 1. Curriculum Development Centre, DOTE. Page 174 4052633 – Elective Theory II Data Science and Big Data DIPLOMA IN COMPUTER ENGINEERING SEMESTER PATTERN III YEAR N – SCHEME VI SEMESTER
- 2. Curriculum Development Centre, DOTE. Page 175 STATE BOARD OF TECHNICAL EDUCATION &TRAINING, TAMILNADU DIPLOMA IN ENGINEERING / TECHNOLOGY SYLLABUS N-SCHEME (To be Implemented for the students admitted from the year 2021 - 2022 onwards) Course Name : 1052:Diploma in Computer Engineering Subject Code : 4052633 Semester : VI Subject title : Elective Theory - II Data Science and Big Data TEACHING AND SCHEME OF EXAMINATION No. of weeks per Semester 16 Weeks Subject Instructions Examination Data Science and Big Data Hours/ Week Hours/ Semester Marks 5 80 Internal Assessment Board Examination Total Duration 25 100 * 100 3 Hrs * Examinations will be conducted for 100 marks and it will be reduced to 75 marks. Topics and Allocation of Hours Unit No. Topic No. of Hours I Introduction to Data Science 15 II Fundamentals of Data Modelling 15 III Fundamentals of Big Data 15 IV Big Data Storage 14 V Big Data Processing 14 Test and Revision 7 Total 80
- 3. Curriculum Development Centre, DOTE. Page 176 RATIONALE: This course provides a comprehensive understanding of data science and data modeling. The foundation on data science is laid to understand the core concepts and the techniques that underlie today's big data computing technologies. This course helps the students in identifying and applying appropriate techniques and tools to solve problems in managing huge quantities of data. OBJECTIVES: This subject has two major divisions. The objectives of these topics are given below. Data Science After studying the first two units of this syllabus, students will be able ● To understand the fundamentals of data science, various data types, theirsources, problems and issues, various formats of data . ● To apply the Python libraries and Microsoft Excel for Data analysis. ● To work with Microsoft Excel for data analysis and applying various ● functions for data analysis. ● To familiarise with the basic data representation methods. ● To understand the concepts of samples, attributes and their relationships. ● To develop and implement simple linear regression models. ● To understand the concept of model equation and of fit. ● To understand and differentiate the concepts of predictive models and the classification models. ● To familiarize with the concepts of Neural Networks, Decision Trees and Nearest neighbors techniques. Big Data After studying the lessons from Units III to V, the students will be able to ● Get conceptual understanding of Big Data, Web data, classification of data, Big Data characteristics, types, classification and handling techniques. ● Get the conceptual understanding of the impact of ICT developments on Big Data Adoption.
- 4. Curriculum Development Centre, DOTE. Page 177 ● Understand the Big Data Analytics Life Cycle. ● Get the conceptual understandings of Big Data Storage systems and technologies. ● Understand the concepts of NoSQL databases, their types and characteristics. ● Understand the concepts of Hadoop and its Ecosystem. ● Understand the steps involved in Big data processing like parallel processing, distributed processing and Batch processing. ● Get understanding of MapReduce, map and reduce tasks, MapReduce algorithm. ● Understand the various techniques for Big Data analysis. ● Get introduced to the concepts and types of machine learning techniques. ● Explore the applications of Big Data in different fields. Detailed Syllabus Contents : Theory Unit Name of the Topics Hours I Introduction to Data Science 15 1.1.Data Science - Subfields of Data Science- Data Types-Data 6 Science Road Map- Programming languages for Data Science- Problems with Data- Formatting issues- Python features- Python Technical libraries- Python Arrays and Data Frames. 1.2.Data sources- Data Quality- Consistency and accuracy 4 (Integrity), Noise: Outliers, Missing and Duplicate values- Data Preprocessing using Cleaning, Enrichment, Editing, Reduction, Wrangling- Data Formats: TXT, CSV, XML, JSON, TLV- Loading and Saving files 1.3 Working with Excel: Loading data- Statistical functions- Text 5 Functions- Lookup Functions- Sorting- Filtering- Data Analysis: Correlation, covariance, Descriptive statistics, Regression. II Fundamentals of Data Modelling 15
- 5. Curriculum Development Centre, DOTE. Page 178 2.1.Linear Algebra: Data representation - Data as a Matrix - Samples and Attributes- Classification of attributes- Concept of Rank-Identify the relationship among attributes 5 2.2.Predictive models: Regression Models - Linear regression - Simple and Multiple Regression-Correlation-Mean squared Error- Testing goodness of fit-Model Equation 5 2.3.Classification models: Two class- Multi class classification- Separability- Performance measures- Terminology- Confusion Matrix-Types (Concepts only): Neural Network- Decision Trees- Nearest Neighbors. 5 III Fundamentals of Big Data 15 3.1Data - Web Data- Classification of Data- Big Data- Characteristics- Volume, Velocity, Variety, Veracity, Value- Need for Big Data- Big Data Types and classifications- Sources of Big Data- Big Data handling techniques-Challenges. 6 3.2 Impact of ICT developments on Big data Adoption: data analytics and data science, digitization, affordable technology and commodity hardware, social media, hyper connected communities and devices, cloud computing and IoT. 4 3.3.Big Data Analytics Life Cycle: Business Case Evaluation, Data Identification, Data Acquisition & Filtering, Data Extraction, Data Validation & Cleansing, Data Aggregation & Representation, Data Analysis, Data Visualization, Utilization of Analysis Results. 5 IV Big Data Storage 14 4.1.Storage Concepts: Clusters, File Systems, Distributed File System, NoSQL, Sharding, Replication, Master Slave, Peer to Peer, CAP Theorem 4
- 6. Curriculum Development Centre, DOTE. Page 179 4.2. Big Data Storage Technologies: On-Disk Storage Devices- Distributed File system-RDBMS- NoSQL Databases- Characteristics of NoSQL- Types of NoSQL Storage devices. In-Memory storage devices-Data Grids-Databases 5 4.3.Hadoop: Introduction- Hadoop and its Ecosystem: Hadoop core components - Features of Hadoop- Hadoop Ecosystem components- Hadoop streaming- Hadoop pipes- Hadoop distributed File system- HDFS data storage -Hadoop Ecosystem tools. 5 V Big Data Processing 14 5.1.Parallel data processing- Distributed data processing- Hadoop Framework- Processing workloads- cluster for processing- Batch processing with MapReduce- Map and Reduce Tasks- MapReduce algorithms- Processing in Realtime mode- Real time processing and MapReduce. 5 5.2.Big Data Analysis Techniques: Quantitative analysis, Qualitative analysis, Data mining, Statistical analysis: Correlation, regression, Machine Learning: Classification, clustering, outlier detection, filtering. Semantic analysis: Natural language processing, Text Analytics, Sentiment analysis, Visual Analysis 5 5.3.Big Data Analytics Applications and case studies: Big data in Marketing and sales- Big data and Healthcare- Big data in Medicine- Big Data in Advertising. 4 Reference books 1. Field Cady, “The Data Science Handbook”, Wiley, 2017. 2. Jake VanderPlas, “Python Data Science Handbook- Essential tools for working with data”, O’REILLY, 2017 3. Davy Cielen, Arno D. B. Meysman, Mohamed Ali, “Introducing Data Science”, manning publications, 2016 4. Thomas Erl, Wajid Khattak - Big Data Fundamentals Concepts, Drivers & Techniques-Prentice Hall (2016). 5. Raj kamal, Preeti Saxena, “Big Data Analytics-Introduction to Hadoop, Spark and Machine Learning”, McGraw Hill Education(India) Pvt Ltd., 2019.
- 7. Curriculum Development Centre, DOTE. Page 180 6. Michael Minelli, Michelle Chambers, and Ambiga Dhiraj, "Big Data, Big Analytics: Emerging Business Intelligence and Analytic Trends for Today's Businesses", Wiley, 2013. 7. Tom White, "Hadoop: The Definitive Guide", Third Edition, O'Reilley, 2012. 8. NPTEL MOOC courses on “Data Science” and “Big Data”.