SlideShare a Scribd company logo

DMDW Lesson 03 - Data Warehouse Theory

•Download as PPTX, PDF•
•
Johannes Hoppe
Johannes Hoppe

I'm planning to give you a detailed introduction to the concepts of the data warehouse world. We will also see why data mining and data warehouses are closely connected to each other.

Related slideshows

Data mining & data warehousing (ppt)
Data mining & data warehousing (ppt)Data mining & data warehousing (ppt)
Data mining & data warehousing (ppt)
 
Introduction to Data Warehousing
Introduction to Data WarehousingIntroduction to Data Warehousing
Introduction to Data Warehousing
 
DATA WAREHOUSING AND DATA MINING
DATA WAREHOUSING AND DATA MININGDATA WAREHOUSING AND DATA MINING
DATA WAREHOUSING AND DATA MINING
 
1 of 25
STUDIERENUND DURCHSTARTEN.Author I: Dip.-Inf. (FH) Johannes HoppeAuthor II: M.Sc. Johannes HofmeisterAuthor III: Prof. Dr. Dieter HomeisterDate: 18.03.2011
Data WarehouseAuthor I: Dip.-Inf. (FH) Johannes HoppeAuthor II: M.Sc. Johannes HofmeisterAuthor III: Prof. Dr. Dieter Homeister Date: 18.03.2011
01Data Warehouse3
Definition DW“A data warehouse is a single source for key, corporate information needed to enable business decisions .”Dieter Homeister (his DM Script)4
Data WarehouseOLTP and DSS DefinedAn application that updates is called an on-line transaction processing (OLTP) application An application that issues queries to the readonly database is called a decision support system (DSS) 5
Data WarehouseStovepipe vs. IntegrationWhen systems stand by themselves they are often referred to as stovepipes Systems that easily share data are called well integrated systems 6
Data WarehouseProblems with Stovepipe Architecture (1/2)ProblemsUsers who wish to access data must query several different DSS to find it Data may have fundamental conflicts between DSS a department code table in one DSS may differ in another DSS a measurement may be stored in meters in one DSS and yards in another7
Data WarehouseProblems with Stovepipe Architecture (2/2)Solution:Use a data warehouse, where data is integrated from the several different stovepipe systems Data warehouse is really sharing-lite -- you don’t have toco-ordinate as much when applications are built and you still reap the benefits of data sharing 8
Data WarehouseData Warehouse SolutionA data warehouse is an attempt to integrate separate DSS so that users can query one place to find the answers to their questions A data warehouse has the key, corporate data in the organization A data warehouse tracks historical data 9
02Selling the Data Warehouse10
Data WarehouseSelling the Data Warehouse(1/2)A data warehouse project will fail without corporate sponsorship Preferably, the project should be sponsored by the CEO The CEO must be sold on the value to the business to improve competitive advantage by deploying a data warehouse 11
Data WarehouseSelling the Data Warehouse (2/2)If an active, corporate sponsor does not exist, data sources will be very difficult to identify Only add data to the warehouse that will answer key, corporate questions asked by the corporate sponsor. Otherwise, you will have a data dump12
Data WarehouseBuilding a Useful Data Warehouse You really need: strong executive sponsorship good knowledge of the data sound software engineering stability from source systems users who want a success A 75 percent failure rate is often cited 13
Data WarehouseEnterprise Information SystemAn EIS (Enterprise Information System) allows users to query data in a data warehouse Now users can access key, corporate data in the data warehouse 14
Data WarehouseUsers of an Enterprise Information System (1/3)multiple EIS (or different graphical interfaces) are needed to satisfy different types of users General userswant a tool that provides detailed data, but is easy to use Want access to the data warehouse to do routine tasks such as Find me Joes phone number, etc. Simple application, not focused on large reports 15
Data WarehouseUsers of an Enterprise Information System (2/3)ExecutivesWant a high-level, summary data (and a simple tool)Must be easy to use, users want to click a few buttons and get data they want Results must be graphs Users should be able to drill-down into key areas. 16
Data WarehouseUsers of an Enterprise Information System (3/3)Analystswant a flexible, more detailed tool Often very knowledgeable about the data Willing to do more work to learn about the data Sometimes even learn SQL to issue their own ad-hoc queries 17
Data WarehouseNeed for Data WarehousesData warehouses provide a single place to store key corporate data users can go one place to find this key data using an enterprise information system (EIS) also a place to store and access historical data Users measure performance goals for their company over a period of timeCompany statistics are available Data not stored in the same place is difficult to locate and compare, easily lost Single query can be used to access key data18
Data WarehouseSecurity & Data WarehousesBuilding a data warehouse does increase security risk because key, corporate information are all in one place Risk reduction: database system components can be used to protect the data warehouse. These include Views Access control Security Administration Encryption Audit (logging of all accesses) 19
Data WarehouseMoving Data into the Data WarehouseMoving data from source OLTP systems to the data warehouse is one of the hardest tasks in data warehousing Updates to the data warehouse are performed periodically weekly , nightly, monthly …Occasionally, real-time data is needed in a data warehouse, but this is not very common see the document about ETL, too!20
Data WarehouseData MartA data mart is a subset of the data warehouse that may make it simpler for users to access key corporate data Sometimes, users only need a piece of data from the data warehouse The data mart is typically fed from the data warehouse 21
ReferencesData Warehouse Books and ReferencesRalph Kimball, MargyRossl: The Data Warehouse Toolkit, 2nd Ed., John Wiley & Sons 2002 (Lists ofpitfalls, verydetailedforseveralapplicationslike CRM, HR, Insurances)W. H. Inmon: Buildingthe Data Warehouse, 3rd Ed., John Wiley & Sons 2002 (DW design, migration, techicaldetails)Claudia Imhoff, Nicholas Galemmo, Jonathan G. Geiger: Mastering Data Warehouse Design, John Wiley & Sons 2003 (Technical andbusinessview, design, optimization)Donald K. Burleson, W. H. Inmon, Joseph Hudicka: The Data Warehouse eBusiness DBA Handbook, BMC Software andDBAzine/RampantTechpress 2003 (AvailableaseBook, technicaldetails, eBusiness, focus on Oracle, DB/DW administration, tools)22
ReferencesData Warehouse Books and ReferencesMaria Sueli Almeida, MissaoIshikawa, Joerg Reinschmidt, Torsten Roeber: GettingStartedwithDataWarehouseand Business Intelligence, www.redbooks.ibm.com, 1999 (eBookfrom IBM, focus on DB2, verytechnical)Mark W. Humphries, Michael W. Hawkins, Michelle C. Dy: Data Warehousing, Pearson Education, 1998 (Verytechnical, incl. projectmgmt., architecture, hardwareand parallel computing)Chris Todman: Designing a Data Warehouse, Prentice Hall 2000 (Introduction, not verydetailed)23
THANK YOUFOR YOUR ATTENTION24
ReferencesData Warehouse Books and ReferencesDavid Grossman, Ophir Frieder: Introductionto Data Warehouse, Illinois Institute of Technology 2005Dr. Andreas Geppert, Credit Suisse: Data Warehousing - Data-Warehouse-Entwurf, 2006, http://arvo.ifi.unizh.ch/dbtg/Classes/DWH/Slides/dwh-04-sl.pdf (p31: Explainationofstar/snowflake/galaxyscheme, in German)Carmela R. Balassiano: Data Warehouse Design Feb. 2007, http://academic.brooklyn.cuny.edu/cis/cbalassiano/CIS717-2%20course%20documents/week2/Data%20Warehouse%20primer.ppt (p12, p18: Explainationofstar/snowflake/galaxyscheme, in English)25

More Related Content

DMDW Lesson 03 - Data Warehouse Theory

  • 1. STUDIERENUND DURCHSTARTEN.Author I: Dip.-Inf. (FH) Johannes HoppeAuthor II: M.Sc. Johannes HofmeisterAuthor III: Prof. Dr. Dieter HomeisterDate: 18.03.2011
  • 2. Data WarehouseAuthor I: Dip.-Inf. (FH) Johannes HoppeAuthor II: M.Sc. Johannes HofmeisterAuthor III: Prof. Dr. Dieter Homeister Date: 18.03.2011
  • 4. Definition DW“A data warehouse is a single source for key, corporate information needed to enable business decisions .”Dieter Homeister (his DM Script)4
  • 5. Data WarehouseOLTP and DSS DefinedAn application that updates is called an on-line transaction processing (OLTP) application An application that issues queries to the readonly database is called a decision support system (DSS) 5
  • 6. Data WarehouseStovepipe vs. IntegrationWhen systems stand by themselves they are often referred to as stovepipes Systems that easily share data are called well integrated systems 6
  • 7. Data WarehouseProblems with Stovepipe Architecture (1/2)ProblemsUsers who wish to access data must query several different DSS to find it Data may have fundamental conflicts between DSS a department code table in one DSS may differ in another DSS a measurement may be stored in meters in one DSS and yards in another7
  • 8. Data WarehouseProblems with Stovepipe Architecture (2/2)Solution:Use a data warehouse, where data is integrated from the several different stovepipe systems Data warehouse is really sharing-lite -- you don’t have toco-ordinate as much when applications are built and you still reap the benefits of data sharing 8
  • 9. Data WarehouseData Warehouse SolutionA data warehouse is an attempt to integrate separate DSS so that users can query one place to find the answers to their questions A data warehouse has the key, corporate data in the organization A data warehouse tracks historical data 9
  • 10. 02Selling the Data Warehouse10
  • 11. Data WarehouseSelling the Data Warehouse(1/2)A data warehouse project will fail without corporate sponsorship Preferably, the project should be sponsored by the CEO The CEO must be sold on the value to the business to improve competitive advantage by deploying a data warehouse 11
  • 12. Data WarehouseSelling the Data Warehouse (2/2)If an active, corporate sponsor does not exist, data sources will be very difficult to identify Only add data to the warehouse that will answer key, corporate questions asked by the corporate sponsor. Otherwise, you will have a data dump12
  • 13. Data WarehouseBuilding a Useful Data Warehouse You really need: strong executive sponsorship good knowledge of the data sound software engineering stability from source systems users who want a success A 75 percent failure rate is often cited 13
  • 14. Data WarehouseEnterprise Information SystemAn EIS (Enterprise Information System) allows users to query data in a data warehouse Now users can access key, corporate data in the data warehouse 14
  • 15. Data WarehouseUsers of an Enterprise Information System (1/3)multiple EIS (or different graphical interfaces) are needed to satisfy different types of users General userswant a tool that provides detailed data, but is easy to use Want access to the data warehouse to do routine tasks such as Find me Joes phone number, etc. Simple application, not focused on large reports 15
  • 16. Data WarehouseUsers of an Enterprise Information System (2/3)ExecutivesWant a high-level, summary data (and a simple tool)Must be easy to use, users want to click a few buttons and get data they want Results must be graphs Users should be able to drill-down into key areas. 16
  • 17. Data WarehouseUsers of an Enterprise Information System (3/3)Analystswant a flexible, more detailed tool Often very knowledgeable about the data Willing to do more work to learn about the data Sometimes even learn SQL to issue their own ad-hoc queries 17
  • 18. Data WarehouseNeed for Data WarehousesData warehouses provide a single place to store key corporate data users can go one place to find this key data using an enterprise information system (EIS) also a place to store and access historical data Users measure performance goals for their company over a period of timeCompany statistics are available Data not stored in the same place is difficult to locate and compare, easily lost Single query can be used to access key data18
  • 19. Data WarehouseSecurity & Data WarehousesBuilding a data warehouse does increase security risk because key, corporate information are all in one place Risk reduction: database system components can be used to protect the data warehouse. These include Views Access control Security Administration Encryption Audit (logging of all accesses) 19
  • 20. Data WarehouseMoving Data into the Data WarehouseMoving data from source OLTP systems to the data warehouse is one of the hardest tasks in data warehousing Updates to the data warehouse are performed periodically weekly , nightly, monthly …Occasionally, real-time data is needed in a data warehouse, but this is not very common see the document about ETL, too!20
  • 21. Data WarehouseData MartA data mart is a subset of the data warehouse that may make it simpler for users to access key corporate data Sometimes, users only need a piece of data from the data warehouse The data mart is typically fed from the data warehouse 21
  • 22. ReferencesData Warehouse Books and ReferencesRalph Kimball, MargyRossl: The Data Warehouse Toolkit, 2nd Ed., John Wiley & Sons 2002 (Lists ofpitfalls, verydetailedforseveralapplicationslike CRM, HR, Insurances)W. H. Inmon: Buildingthe Data Warehouse, 3rd Ed., John Wiley & Sons 2002 (DW design, migration, techicaldetails)Claudia Imhoff, Nicholas Galemmo, Jonathan G. Geiger: Mastering Data Warehouse Design, John Wiley & Sons 2003 (Technical andbusinessview, design, optimization)Donald K. Burleson, W. H. Inmon, Joseph Hudicka: The Data Warehouse eBusiness DBA Handbook, BMC Software andDBAzine/RampantTechpress 2003 (AvailableaseBook, technicaldetails, eBusiness, focus on Oracle, DB/DW administration, tools)22
  • 23. ReferencesData Warehouse Books and ReferencesMaria Sueli Almeida, MissaoIshikawa, Joerg Reinschmidt, Torsten Roeber: GettingStartedwithDataWarehouseand Business Intelligence, www.redbooks.ibm.com, 1999 (eBookfrom IBM, focus on DB2, verytechnical)Mark W. Humphries, Michael W. Hawkins, Michelle C. Dy: Data Warehousing, Pearson Education, 1998 (Verytechnical, incl. projectmgmt., architecture, hardwareand parallel computing)Chris Todman: Designing a Data Warehouse, Prentice Hall 2000 (Introduction, not verydetailed)23
  • 24. THANK YOUFOR YOUR ATTENTION24
  • 25. ReferencesData Warehouse Books and ReferencesDavid Grossman, Ophir Frieder: Introductionto Data Warehouse, Illinois Institute of Technology 2005Dr. Andreas Geppert, Credit Suisse: Data Warehousing - Data-Warehouse-Entwurf, 2006, http://arvo.ifi.unizh.ch/dbtg/Classes/DWH/Slides/dwh-04-sl.pdf (p31: Explainationofstar/snowflake/galaxyscheme, in German)Carmela R. Balassiano: Data Warehouse Design Feb. 2007, http://academic.brooklyn.cuny.edu/cis/cbalassiano/CIS717-2%20course%20documents/week2/Data%20Warehouse%20primer.ppt (p12, p18: Explainationofstar/snowflake/galaxyscheme, in English)25