SlideShare a Scribd company logo
Trends and Issues Impacting
              Database Management Systems
                       Circa 2004


Craig S. Mullins
Director, Technology Planning/BMC Software, Inc.
http://www.craigsmullins.com
General Industry Trends
• Complexity
     • Heterogeneity
                                      • Speed-to-Market
     • Rapid change                   • Buy versus Build
     • Consolidation                  • Follow the Leader
• Lack of Resources
     • Skilled Technicians
        – A lot of PFCSKs and IROCs
     • Time
        – Already overworked
     • Budget
DBMS Industry Trends
•   Rapid DBMS Versioning
•   Enabling for the Internet
•   Online, real-time …
•   Java and .Net
•   XML
•   Multimedia
•   Procedural logic
•   ERP and CRM
•   Mo’ data! Mo’ data!
•   Integration
New DBMS Versions
• Analysis of New Features
  – Check all Requirements
     • Hardware and Software
• Planning the Upgrade
  – Impact to system, applications
  – Scheduling
• Fallback Strategy
• Migration Verification
DBMS Subsumes Functionality
•   XML
•   ETL and Propagation
•   OLAP
•   Multimedia
•   Objects
•   Logic/Code
    – triggers, UDFs, stored procedures
The Internet
• From DBA to eDBA
 Internet-Age DBA Skills
It’s About Change...




Source: Gartner Group
Increasingly Complex Enterprise
         Infrastructure
A Virtual, Extended Infrastructure
Internet Infrastructure Weaknesses
              Problem           Symptom              Effect

                             Sporadic crashes     Unplanned
            Unreliable       for no apparent      outages
                             reason

                             Operators do not     Simple problems
            Complex
                             understand how to    result in long
                             resolve problems     outages

            Fragile          IT mgrs. Must        Long debugging
                             debug innocuous      cycles for new
                             changes              releases

                                                  Systems must be
            Vulnerable       Viruses and bugs     rolled back to
                             attack all systems   clean backups
                             at once

Source: Forrester Research
The Cost of an Outage
E-Business Applications Fail to
  Deliver Service Because...
Unplanned outages - Unplanned Outages
        Planned and            Planned outages -
 driven by problems hr. workdays
  x Staff Shortages - 18       driven by change
  x   Short Implementation Times x Management
  x   Application failure
       x Executive Demands for Web Presence
                                    x Maintenance
  x
  x   Element Failure Scalability
      Management Tool               x Migrations
  x   Performance Keep Up With Own Scripts
       x Unable to
  x
  x   Capacity limitsWeb Load/Growth Version
      Unpredictable
                                    x

  x   Transaction Success of Site     Management
       x Unknown
      Backouts                      x Propagation
  x   Customized Environments
       x Need Flexibility of Management
Where is Application Downtime?
                                            30% of
• Planned vs. Unplanned                    Outages

  – Planned outages represents   70% of
                                 Outages
    70% of application
    downtime.
  – Just 30% is due to
    unplanned outages and
    50% of the unplanned
    downtime is due to
    problems during planned
    downtime.
Impact on eDBAs
• Downtime is not tolerated
  – Downtime was never “good” but it was tolerated
  – No longer, though as we move from availability to
    e-vailability with intelligent techniques & solutions
• Avoiding downtime with automated tools
  – “On the fly” operational tuning
     • ALTER SYSTEM - Oracle9i
     • SET SYSPARM - DB2 V7
  – Monitor performance across multiple platforms
  – Redundant systems – HACMP, RAID, etc.
Example: Keeping Your Systems Up!

                                     Monitor
                      SYSPLEX
                                    ARCHIVE FAILED?
        Action         DB2A         ADD LOG
                                    NOTIFY AML
Change ZPARM           DB2B

       Monitor         DB2C
                                               Action

   EDM POOL FAILED?               LOGS
                                 LOGS
                                LOGS     +      LOG
Example: Recover with no Down Time!
       LOGS
      LOGS
     LOGS


                                           Undo/Redo SQL
                          Log
                         Analysis

                                                 DB2A

                                                 DB2B
       No need to
       take the data                Data
       off-line to run
                                                 DB2C
       SQL against it!
Online and Real-time
• The need for more and more availability drives
  online and real-time maintenance
   – The DBMS begins to allow for more changes to be
     made during normal operations
   – The DBMS begins to gather statistics and performance
     metrics during normal operations
   – ISVs deliver more online, real-time features and
     functionality that the DBMS does not yet deliver
   – Less manual-intervention required
Database Design and Web Time
 • When the Web is involved everything becomes
   “rush-rush” - do it now!
 • Don’t let database design suffer - take your time
   and do it right.
 • Apps are temporary but data is forever!
    – If you do not believe this, then consider: “How
      often has your organization re-entered or re-
      keyed data into a new database when the data
      already exists elsewhere?”
Prepare for Global Scope

 Who is accessing the
  database?
      Internal and External users
      Local, National, and
       International
      Now versus then




  24 x 7 x 365¼
Impact on the DBA
             Where is the performance problem?
               Most experts agree that 75% to 80% of

                performance problems in relational
                applications is caused by poor SQL or
      gateway                                                        HTML
                application code, but on the web . . .
      ISP
                                       XML       init.ora        ZPARMs
        connection           CGI
  3GL                               bridge/router/hub
             DB2      Java                                operating system
            Connect                     DNS
ASP                                                                hardware
                                                          HTTP
        network cabling
                                                 Java       application code
      SQL
                                   SQL*Net       applet
            database schema
                                                              network software
Java’s Popularity is Skyrocketing

                  Java Software Market ($U.S. Millions)
        2000
        1750
        1500
        1250
        1000
          750
          500
          250
              0
               1997   1998    1999   2000    2001   2002

Source: IDC
Why Java?
•   Portability
•   Make web pages active and dynamic
•   Like C/C++ with a smaller footprint
•   Lower cost of change (DLLs)
•   Download changes from the web
•   But
    – slower - interpreted, not compiled
Java and Databases:
        Two Methods
• JDBC
 – Enables Dynamic SQL from Java
 – Uses API (CLI)

• SQLJ
 – Enables Static SQL for Java
 – Uses embedded SQL
Java Alphabet Soup
• J2EE - Java 2 Enterprise Edition
  – Standard services and specifications for making
    Java highly available, secure, reliable, and
    scalable for enterprise adoption
• EJB - Enterprise Java Beans
  – Components that contain the business logic for
    a J2EE application
Impact of Java on DBA
• Application tuning
  – Must understand Java
     • To provide guidance during design reviews
        – Is the problem in the SQL or the application
     • How can you tune the application if you do not
       understand the language (Java)?
        – Optimizing SQL is not enough since it may be embedded
          in poor application code
  – Must understand the SQL techniques used
     • JDBC and SQLJ
Microsoft .NET
• ... is a set of Microsoft technologies for
  connecting people, systems, and devices
• ... allows Internet Servers to expose
  functions to any client named as .NET web
  services
• … enables software to be delivered as a
  service over the web
• … is designed to let many different services
  and systems interact
Java versus .Net
• ...designed to enable   • …designed to enable
  applications to be        development in
  deployed on any           multiple languages as
  platform as long as       long as the application
  they are written in       is deployed on
  Java                      Windows
The Rush to XML
What is XML?
• XML stands for eXtensible Markup Language.
   – Like HTML, XML is based on SGML
     (Standard Generalized Markup Language)
   – HTML uses tags to describe the appearance of data on a page,
     whereas XML uses tags to describe the data itself, instead of its
     appearance.
   – Allows documents to be self-describing, through the specification
     of tag sets and the structural relationships between the tags.
XML is a Meta Language
• XML is actually a meta language - a language used to
  define other languages.
   – These languages are collected in dictionaries called Document
     Type Definitions (DTDs).
   – The DTD stores definitions of tags for specific industries or fields
     of knowledge. So, the meaning of a tag must be defined in a DTD
     before it can be used.
   – The DTD for an XML document can be either part of the
     document or stored in an external file.
Sample XML DTD
<!DOCTYPE CUSTOMER [
<!ELEMENT CUST (first_name, middle_initial, last_name,
                  company_name, street_address, city,
                  state, zip_code, country*)>
<!ELEMENT first_name (#PCDATA)>
<!ELEMENT middle_initial (#PCDATA)>
<!ELEMENT last_name (#PCDATA)>
<!ELEMENT company_name (#PCDATA)>
<!ELEMENT street_address (#PCDATA)>
<!ELEMENT city (#PCDATA)>
<!ELEMENT state (#PCDATA)>
<!ELEMENT zip_code (#PCDATA)>
<!ELEMENT country (#PCDATA)>
]
Sample XML
<CUSTOMER>
<first_name>Craig</first_name>
<middle_initial>S.</middle_initial>
<last_name>Mullins</last_name>
<company_name>BMC Software, Inc.</company_name>
<street_address>2101 CityWest Blvd.</street_address>
<city>Houston</city>
<state>TX</state>
<zip_code>77042</zip_code>
<country>U.S.A.</country>
</CUSTOMER>
Why is XML Important?
• XML is used for exchanging and sharing data
   – Inter- and intra-organization
• XML may be used to define database structures
   – and vice versa


         DOCTYPE
         XML definition
                                     e   DDL
         lorem ipsum . . .   Generat             Physical
                                                 Database


                                       e   XML
                               Generat
XML to DBMS
s   Add XML functionality to the DBMS
     • Extender capabilities similar to IBM DB2 video, image, audio,
       and other multimedia data types
     • Combine UDT, UDFs, and triggers for functionality
s   XML document stored in a column -or-
s   XML components stored as parts of multiple columns
    in multiple tables
s   Formulate XML documents from existing tables
s   Search XML documents text and sections
s   XQuery capabilities
XML and Data Management
•   Middleware: Software called from your               •   Wrappers: Software that treats XML
    application to transfer data between XML                documents as a source of relational data.
    documents and databases. For data-                      These products typically query XML
    centric applications.                                   documents using SQL. For data-centric
•   XML-Enabled Databases: Databases                        applications.
    with extensions for transferring data               •   Content Management Systems:
    between XML documents and                               Applications built on top of native XML
    themselves. Primarily for data-centric                  databases and/or the file system for
    applications.                                           content/document management. For
•   Native XML Databases: Databases that                    document-centric applications.
    store XML in "native" form. For data- and           •   XML Query Engines: Standalone engines
    document-centric applications.                          that can query XML documents. For
•   XML Servers: XML-aware J2EE servers,                    data- and document-centric applications.
    Web application servers, integration                •   XML Data Binding: Products that can
    engines, and custom servers. For data-                  bind XML documents to objects. Some of
    and document-centric applications.                      these can also store/retrieve objects from
                                                            the database. For data-centric
                                                            applications.
    Source: XML Database Products by Ronald Bourret
    http://www.rpbourret.com/xml/XMLDatabaseProds.htm
Impact of XML on DBA
• Database definition
   – Perhaps a new way to create databases
       • DTDs to build a database schema
       • Create DTD from a database schema
       • Interface to data modeling tools
• Data access
   – Database extenders
   – XML defines the data it contains
• New XML DBMS products?
   – Tamino (Software AG)
   – Xperanto (IBM)
• Over-enthusiasm!
Logic and
the DBMS
Triggers vs. Functions vs. Procs
U
P   I
    N   D
D                            Code
                             Code
                             Code
    S   E
A
    E   L
T                 Triggers
    R   E
E
    T   T
                                     UDFs
        E                             function( )
                                        if this then that
                                        else
                                          do this stuff
                                        return x
                      Stored          end
                    Procedures



                   →          Code
                              Code
                              Code                          SQL
Procedural DBA Duties

        Stored
      Procedures                  Performance
                                    Monitors


D
E                  D
V                  E
                        Admin.                  DB2
E      Triggers    B    Process
L                  U
O                  G
P                                           External
                       Design               Libraries
                       Review
      Functions
Role of the Procedural DBA
                              DBCO Impl
                              (COMMIT in
                                        em          entation          Ensuring
                                           proc, write o
                                                           r guide)   Reuse
                                                        n
                                        m    stratio
                                          ini set)           Des
                                                                i
                                CO Ad rder, proc
                           DB r firing o                    Rev gn
                              igge
                                                               iews
                           (tr                 EX
                                             An PLAI
                            On Call               a ly
                                                       sis
                                                           N     Coding
                            for DBCO                             Complex
                            Abends                                Queries
                                                       QL
                                               i n gS       Debuggi
                                                                    ng
                       Schema               un              SQL
                                           T
                       R esolution

DBCO = Database Code Object
Non-Traditional Data
“Universal” Data
                                       Complex
                                                         Compound Docs
                                   Graphics
                  Video


                                                         Design Data

                                                                Spatial Data
                          Images
                                                                       Temporal Data
                                                 Text
Unstructured                                                                Structured

          Audio
                                                     Existing
                                                    Databases
                    Seismic Data


                                        Simple

                                                                   Source: Gartner Group
Multimedia = Big Databases
   Object                  Typical Size
   HD TV                   200 MB/second
   Feature-length,
   high-resolution movie   5-6 GB
   High-resolution video   3 GB/hour
   Feature-length movie    2 GB
   Video                   1 GB/hour
   Radiologic image        40-60 MB
   Color image             20-40 MB
   Large image             200 KB-3 MB
   Text                    30-40 KB/page
   Check image             45 KB
Integration and Federation
• DBMSs are adapting to “handle” more
  types of non-traditional data
  – Spreadsheets
  – Word documents
  – Presentations
• How?
  – Integrate the data into the DBMS
  – Federate and manage the data “where it lies”
DBMS to Manage
                   All Kinds of Data
              Federated DBMS

                                • A federated
                                  approach allows
                                  the DBMS to
                                  manage data
                                  where it exists

Spreadsheet
DBMS to Manage
                    All Kinds of Data
              Or integration…

                                 • An alternate
                                   approach “sucks”
                                   the non-relational
                                   data into the
                                   DBMS to be
                                   managed

Spreadsheet
Autonomic Computing

Real Time       Self
Statistics    Managing
                          Intelligent
                         Automation




 Virtual
 Reorg



             Database      Correct and
             Wizards         Notify
Data Keeps Growing
Phenomenal Data Growth
• “Global 2000 companies double the amount of data they
  own every year, while the average dot-com’s data doubles
  every 90 days.”
            – Mike Ruettgers, CEO of EMC Corp., Oracle Open World 2001
• “Inside IBM we talk about 10 times more connected
  people, 100 time more network speed, a 1000 times more
  devices and a million times more data.”
            – Lou Gerstner, CEO of IBM Corp., eBusiness Expo 2000
• A recent research note from Giga Information Group
  estimates that there are about 201,000TB, or about 197
  petabytes, on the planet. Of course, this is just an estimate that
   Giga deems to be accurate within an order of magnitude (that is,
   within a factor of 10).
• McKinsey & Company reports a CAGR of 76% for data
  storage.
Relational Database Size




Source: Gartner Group
Database Size Issues
• Technology enables larger databases
• Web, multimedia, data warehousing, and data mining
  drive up database size
• Disk drives increase in capacity but
  speed of access does not keep up with
  capacity increases
• Cost of storage decreasing; so
  why not store more data? But...
   – What data do users need to store?
   – How long must it be maintained?
   – What are they willing
     to pay?                                                             0010
                                                                    001010100
                                                                101011101011
                                                             101011010010101
                                                     01001001101010111000011
                                                 11001010100100101000100101
The Database Environment
  • A lot of choices!
        – Vendor, platform, and architecture of DBMS

                               MVS, OS/390, z/OS                                   Enterprise
                               Windows NT / 2000 / XP                              - Parallel Edition
                               Unix                                                Departmental
                                     AIX                                           Personal
                                     Sun Solaris
                                                                                   Mobile (PDA)
                                     HP-UX
                                     Linux
                                         others?
                               Others (VSE, VMS, MPE, OS/400, etc.)
                               Desktop OS
                                     Windows 98 / ME / XP
                                     Linux
                                     Mac?

Adabas, Teradata, PostgreSQL, Supra, Compaq Non-Stop SQL, Ingres, IMS, IDMS, Datacom, Teradata, others...
Heterogeneity Plus!
             • The DBA’s
               knowledge must span
               the entire enterprise
                – Operating Systems
                – Networking Protocols
                – Programming
                  Languages
                – Business Objectives
                – And so on…
Impact on the DBA
• Unrealistic Expectations
   – Impossible to master everything
       • Education is the first thing cut!
   – Impossible to specialize in a heterogeneous shop
• Reactive mode is encouraged
   – Even though proactive mode is optimal
   – Who looks for more problems when they don’t have enough time
     in the day to solve the problems of those complaining the loudest.
   – DBA uses YBWJ method
Worldwide Spending for ERP
            Packaged Software
  $30


  $25


  $20
                                                            Manufacturing
                                                            Distribution
  $15
                                                            Human Resources
                                                            Financial
  $10


   $5


   $0
    1995   1996   1997   1998   1999   2000   2001   2002




Source: IDC
The Computing “Platform” Evolution

                                        This is the current
         “Customer Service Apps”        battleground,
                                        NOT the DBMS
2000+
           “Base” Applications     SAP, Oracle, Baan, PeopleSoft

                The DBMS           DB2, Oracle, SQL Server

          Network (LAN/WAN)        TCP/IP, Novell, WinNT

             Communications        SNA, TCP/IP, NFS

          The Operating System     MVS, UNIX, WIN (NT,95)

                                   Proprietary (IBM+) to Open
60’s            Hardware           (Wintel, HP/Intel)
The Data Continuum
                                                 o   n
                                             pti
                                        Perce
                           ence
                       eri
                                                     Wisdom
                      p
                   Ex             Knowledge
                                                   Integration
         ext                                       › Multiple points
    Cont       Information
                              Integration            of knowledge
                              › Systems            Understanding
                              › People             › Knowledge
               Business       Systems                applied
    Data       › Data in      › Technology to
                 Context        store and report
                              People
Simple
› Numbers                     › Identify
› Characters                  › Capture
› Date                        › Maintain
› Time
Complex
› BLOBs
› CLOBs
Varied
› Content
  Management
The Data Continuum

                                                 Wisdom

                                  Knowledge
                                                 Knowledge +
                                                 Application
                 Information      Technology +   Systems
                                  People
                Data Models
                                  Knowledge
     Data       Data Dictionary
                                  Management
                Repository
DBMS
Files
Spreadsheets
Graphics
Folders
Etc.
The Data Continuum
                                                  o   n
                                              pti
An Example:                               rce
                                       Pe
                          ence
                      eri
                                                      Wisdom

                  Exp            Knowledge

      ext
 Cont         Information



Data
Other DBMS Market Trends

• ODBMS – no longer a threat
  to become “mainstream” but…
• “Post-Relational” DBMS?
  – XML DBMS
  – The Associative Model
  – Multivalue DBMS


• In Memory DBMS
Enter the DBA

• The job of database
  administration is getting
  increasingly more difficult
  as database technology
  rapidly advances adding
  new functionality, more
  options, and more
  complex and comp-
  licated capabilities...
The DBA is a “Jack of all Trades”
        OS/390               C++       Windows             V$ Tables
Linux
                   Oracle     SQL                  XML                  Java
 DB2                                                       Informix     applet
                         Unix                        MQ
Connect         DB2
                                                          DNS
      gateway                                                         HTML
            VTAM       CGI                                  TCP/IP
ISP
        connection                                               ZPARMs
      3GL                                                 database schema
                COBOL
ASP                                                  operating system
                Java
                                                         HTTP     hardware
      SQL Server                network software
            bridge/router/hub                             application code
 VB                                    network cabling
                CICS         SQL*Net
                                                   JCL
So What is Needed?
• Intelligent automation of DBA tasks
   – because no one has all the skilled resources
     they need
   – frees up more DBA time



                               They work hard so you
                               don’t have tooooo…


                                     “Scrubbing Bubbles”
Intelligent   Automated
A DBA Control Panel




DBA-focused on DBA tasks, not focused on systems management
tasks like framework products… but integrated with systems
management functionality where it makes sense.
Contact Information
http://www.craigsmullins.com/dba_book.htm
                                            Craig S. Mullins
                                            Technology Planning
                                            Craig_Mullins@BMC.com
                                            http://www.craigsmullins.com



                                            http://www.bmc.com



 http://www.craigsmullins.com/cm-book.htm

More Related Content

Db trends final

  • 1. Trends and Issues Impacting Database Management Systems Circa 2004 Craig S. Mullins Director, Technology Planning/BMC Software, Inc. http://www.craigsmullins.com
  • 2. General Industry Trends • Complexity • Heterogeneity • Speed-to-Market • Rapid change • Buy versus Build • Consolidation • Follow the Leader • Lack of Resources • Skilled Technicians – A lot of PFCSKs and IROCs • Time – Already overworked • Budget
  • 3. DBMS Industry Trends • Rapid DBMS Versioning • Enabling for the Internet • Online, real-time … • Java and .Net • XML • Multimedia • Procedural logic • ERP and CRM • Mo’ data! Mo’ data! • Integration
  • 4. New DBMS Versions • Analysis of New Features – Check all Requirements • Hardware and Software • Planning the Upgrade – Impact to system, applications – Scheduling • Fallback Strategy • Migration Verification
  • 5. DBMS Subsumes Functionality • XML • ETL and Propagation • OLAP • Multimedia • Objects • Logic/Code – triggers, UDFs, stored procedures
  • 6. The Internet • From DBA to eDBA Internet-Age DBA Skills
  • 9. A Virtual, Extended Infrastructure
  • 10. Internet Infrastructure Weaknesses Problem Symptom Effect Sporadic crashes Unplanned Unreliable for no apparent outages reason Operators do not Simple problems Complex understand how to result in long resolve problems outages Fragile IT mgrs. Must Long debugging debug innocuous cycles for new changes releases Systems must be Vulnerable Viruses and bugs rolled back to attack all systems clean backups at once Source: Forrester Research
  • 11. The Cost of an Outage
  • 12. E-Business Applications Fail to Deliver Service Because... Unplanned outages - Unplanned Outages Planned and Planned outages - driven by problems hr. workdays x Staff Shortages - 18 driven by change x Short Implementation Times x Management x Application failure x Executive Demands for Web Presence x Maintenance x x Element Failure Scalability Management Tool x Migrations x Performance Keep Up With Own Scripts x Unable to x x Capacity limitsWeb Load/Growth Version Unpredictable x x Transaction Success of Site Management x Unknown Backouts x Propagation x Customized Environments x Need Flexibility of Management
  • 13. Where is Application Downtime? 30% of • Planned vs. Unplanned Outages – Planned outages represents 70% of Outages 70% of application downtime. – Just 30% is due to unplanned outages and 50% of the unplanned downtime is due to problems during planned downtime.
  • 14. Impact on eDBAs • Downtime is not tolerated – Downtime was never “good” but it was tolerated – No longer, though as we move from availability to e-vailability with intelligent techniques & solutions • Avoiding downtime with automated tools – “On the fly” operational tuning • ALTER SYSTEM - Oracle9i • SET SYSPARM - DB2 V7 – Monitor performance across multiple platforms – Redundant systems – HACMP, RAID, etc.
  • 15. Example: Keeping Your Systems Up! Monitor SYSPLEX ARCHIVE FAILED? Action DB2A ADD LOG NOTIFY AML Change ZPARM DB2B Monitor DB2C Action EDM POOL FAILED? LOGS LOGS LOGS + LOG
  • 16. Example: Recover with no Down Time! LOGS LOGS LOGS Undo/Redo SQL Log Analysis DB2A DB2B No need to take the data Data off-line to run DB2C SQL against it!
  • 17. Online and Real-time • The need for more and more availability drives online and real-time maintenance – The DBMS begins to allow for more changes to be made during normal operations – The DBMS begins to gather statistics and performance metrics during normal operations – ISVs deliver more online, real-time features and functionality that the DBMS does not yet deliver – Less manual-intervention required
  • 18. Database Design and Web Time • When the Web is involved everything becomes “rush-rush” - do it now! • Don’t let database design suffer - take your time and do it right. • Apps are temporary but data is forever! – If you do not believe this, then consider: “How often has your organization re-entered or re- keyed data into a new database when the data already exists elsewhere?”
  • 19. Prepare for Global Scope  Who is accessing the database?  Internal and External users  Local, National, and International  Now versus then 24 x 7 x 365¼
  • 20. Impact on the DBA  Where is the performance problem?  Most experts agree that 75% to 80% of performance problems in relational applications is caused by poor SQL or gateway HTML application code, but on the web . . . ISP XML init.ora ZPARMs connection CGI 3GL bridge/router/hub DB2 Java operating system Connect DNS ASP hardware HTTP network cabling Java application code SQL SQL*Net applet database schema network software
  • 21. Java’s Popularity is Skyrocketing Java Software Market ($U.S. Millions) 2000 1750 1500 1250 1000 750 500 250 0 1997 1998 1999 2000 2001 2002 Source: IDC
  • 22. Why Java? • Portability • Make web pages active and dynamic • Like C/C++ with a smaller footprint • Lower cost of change (DLLs) • Download changes from the web • But – slower - interpreted, not compiled
  • 23. Java and Databases: Two Methods • JDBC – Enables Dynamic SQL from Java – Uses API (CLI) • SQLJ – Enables Static SQL for Java – Uses embedded SQL
  • 24. Java Alphabet Soup • J2EE - Java 2 Enterprise Edition – Standard services and specifications for making Java highly available, secure, reliable, and scalable for enterprise adoption • EJB - Enterprise Java Beans – Components that contain the business logic for a J2EE application
  • 25. Impact of Java on DBA • Application tuning – Must understand Java • To provide guidance during design reviews – Is the problem in the SQL or the application • How can you tune the application if you do not understand the language (Java)? – Optimizing SQL is not enough since it may be embedded in poor application code – Must understand the SQL techniques used • JDBC and SQLJ
  • 26. Microsoft .NET • ... is a set of Microsoft technologies for connecting people, systems, and devices • ... allows Internet Servers to expose functions to any client named as .NET web services • … enables software to be delivered as a service over the web • … is designed to let many different services and systems interact
  • 27. Java versus .Net • ...designed to enable • …designed to enable applications to be development in deployed on any multiple languages as platform as long as long as the application they are written in is deployed on Java Windows
  • 28. The Rush to XML
  • 29. What is XML? • XML stands for eXtensible Markup Language. – Like HTML, XML is based on SGML (Standard Generalized Markup Language) – HTML uses tags to describe the appearance of data on a page, whereas XML uses tags to describe the data itself, instead of its appearance. – Allows documents to be self-describing, through the specification of tag sets and the structural relationships between the tags.
  • 30. XML is a Meta Language • XML is actually a meta language - a language used to define other languages. – These languages are collected in dictionaries called Document Type Definitions (DTDs). – The DTD stores definitions of tags for specific industries or fields of knowledge. So, the meaning of a tag must be defined in a DTD before it can be used. – The DTD for an XML document can be either part of the document or stored in an external file.
  • 31. Sample XML DTD <!DOCTYPE CUSTOMER [ <!ELEMENT CUST (first_name, middle_initial, last_name, company_name, street_address, city, state, zip_code, country*)> <!ELEMENT first_name (#PCDATA)> <!ELEMENT middle_initial (#PCDATA)> <!ELEMENT last_name (#PCDATA)> <!ELEMENT company_name (#PCDATA)> <!ELEMENT street_address (#PCDATA)> <!ELEMENT city (#PCDATA)> <!ELEMENT state (#PCDATA)> <!ELEMENT zip_code (#PCDATA)> <!ELEMENT country (#PCDATA)> ]
  • 32. Sample XML <CUSTOMER> <first_name>Craig</first_name> <middle_initial>S.</middle_initial> <last_name>Mullins</last_name> <company_name>BMC Software, Inc.</company_name> <street_address>2101 CityWest Blvd.</street_address> <city>Houston</city> <state>TX</state> <zip_code>77042</zip_code> <country>U.S.A.</country> </CUSTOMER>
  • 33. Why is XML Important? • XML is used for exchanging and sharing data – Inter- and intra-organization • XML may be used to define database structures – and vice versa DOCTYPE XML definition e DDL lorem ipsum . . . Generat Physical Database e XML Generat
  • 34. XML to DBMS s Add XML functionality to the DBMS • Extender capabilities similar to IBM DB2 video, image, audio, and other multimedia data types • Combine UDT, UDFs, and triggers for functionality s XML document stored in a column -or- s XML components stored as parts of multiple columns in multiple tables s Formulate XML documents from existing tables s Search XML documents text and sections s XQuery capabilities
  • 35. XML and Data Management • Middleware: Software called from your • Wrappers: Software that treats XML application to transfer data between XML documents as a source of relational data. documents and databases. For data- These products typically query XML centric applications. documents using SQL. For data-centric • XML-Enabled Databases: Databases applications. with extensions for transferring data • Content Management Systems: between XML documents and Applications built on top of native XML themselves. Primarily for data-centric databases and/or the file system for applications. content/document management. For • Native XML Databases: Databases that document-centric applications. store XML in "native" form. For data- and • XML Query Engines: Standalone engines document-centric applications. that can query XML documents. For • XML Servers: XML-aware J2EE servers, data- and document-centric applications. Web application servers, integration • XML Data Binding: Products that can engines, and custom servers. For data- bind XML documents to objects. Some of and document-centric applications. these can also store/retrieve objects from the database. For data-centric applications. Source: XML Database Products by Ronald Bourret http://www.rpbourret.com/xml/XMLDatabaseProds.htm
  • 36. Impact of XML on DBA • Database definition – Perhaps a new way to create databases • DTDs to build a database schema • Create DTD from a database schema • Interface to data modeling tools • Data access – Database extenders – XML defines the data it contains • New XML DBMS products? – Tamino (Software AG) – Xperanto (IBM) • Over-enthusiasm!
  • 38. Triggers vs. Functions vs. Procs U P I N D D Code Code Code S E A E L T Triggers R E E T T UDFs E function( ) if this then that else do this stuff return x Stored end Procedures → Code Code Code SQL
  • 39. Procedural DBA Duties Stored Procedures Performance Monitors D E D V E Admin. DB2 E Triggers B Process L U O G P External Design Libraries Review Functions
  • 40. Role of the Procedural DBA DBCO Impl (COMMIT in em entation Ensuring proc, write o r guide) Reuse n m stratio ini set) Des i CO Ad rder, proc DB r firing o Rev gn igge iews (tr EX An PLAI On Call a ly sis N Coding for DBCO Complex Abends Queries QL i n gS Debuggi ng Schema un SQL T R esolution DBCO = Database Code Object
  • 42. “Universal” Data Complex Compound Docs Graphics Video Design Data Spatial Data Images Temporal Data Text Unstructured Structured Audio Existing Databases Seismic Data Simple Source: Gartner Group
  • 43. Multimedia = Big Databases Object Typical Size HD TV 200 MB/second Feature-length, high-resolution movie 5-6 GB High-resolution video 3 GB/hour Feature-length movie 2 GB Video 1 GB/hour Radiologic image 40-60 MB Color image 20-40 MB Large image 200 KB-3 MB Text 30-40 KB/page Check image 45 KB
  • 44. Integration and Federation • DBMSs are adapting to “handle” more types of non-traditional data – Spreadsheets – Word documents – Presentations • How? – Integrate the data into the DBMS – Federate and manage the data “where it lies”
  • 45. DBMS to Manage All Kinds of Data Federated DBMS • A federated approach allows the DBMS to manage data where it exists Spreadsheet
  • 46. DBMS to Manage All Kinds of Data Or integration… • An alternate approach “sucks” the non-relational data into the DBMS to be managed Spreadsheet
  • 47. Autonomic Computing Real Time Self Statistics Managing Intelligent Automation Virtual Reorg Database Correct and Wizards Notify
  • 49. Phenomenal Data Growth • “Global 2000 companies double the amount of data they own every year, while the average dot-com’s data doubles every 90 days.” – Mike Ruettgers, CEO of EMC Corp., Oracle Open World 2001 • “Inside IBM we talk about 10 times more connected people, 100 time more network speed, a 1000 times more devices and a million times more data.” – Lou Gerstner, CEO of IBM Corp., eBusiness Expo 2000 • A recent research note from Giga Information Group estimates that there are about 201,000TB, or about 197 petabytes, on the planet. Of course, this is just an estimate that Giga deems to be accurate within an order of magnitude (that is, within a factor of 10). • McKinsey & Company reports a CAGR of 76% for data storage.
  • 51. Database Size Issues • Technology enables larger databases • Web, multimedia, data warehousing, and data mining drive up database size • Disk drives increase in capacity but speed of access does not keep up with capacity increases • Cost of storage decreasing; so why not store more data? But... – What data do users need to store? – How long must it be maintained? – What are they willing to pay? 0010 001010100 101011101011 101011010010101 01001001101010111000011 11001010100100101000100101
  • 52. The Database Environment • A lot of choices! – Vendor, platform, and architecture of DBMS MVS, OS/390, z/OS Enterprise Windows NT / 2000 / XP - Parallel Edition Unix Departmental AIX Personal Sun Solaris Mobile (PDA) HP-UX Linux others? Others (VSE, VMS, MPE, OS/400, etc.) Desktop OS Windows 98 / ME / XP Linux Mac? Adabas, Teradata, PostgreSQL, Supra, Compaq Non-Stop SQL, Ingres, IMS, IDMS, Datacom, Teradata, others...
  • 53. Heterogeneity Plus! • The DBA’s knowledge must span the entire enterprise – Operating Systems – Networking Protocols – Programming Languages – Business Objectives – And so on…
  • 54. Impact on the DBA • Unrealistic Expectations – Impossible to master everything • Education is the first thing cut! – Impossible to specialize in a heterogeneous shop • Reactive mode is encouraged – Even though proactive mode is optimal – Who looks for more problems when they don’t have enough time in the day to solve the problems of those complaining the loudest. – DBA uses YBWJ method
  • 55. Worldwide Spending for ERP Packaged Software $30 $25 $20 Manufacturing Distribution $15 Human Resources Financial $10 $5 $0 1995 1996 1997 1998 1999 2000 2001 2002 Source: IDC
  • 56. The Computing “Platform” Evolution This is the current “Customer Service Apps” battleground, NOT the DBMS 2000+ “Base” Applications SAP, Oracle, Baan, PeopleSoft The DBMS DB2, Oracle, SQL Server Network (LAN/WAN) TCP/IP, Novell, WinNT Communications SNA, TCP/IP, NFS The Operating System MVS, UNIX, WIN (NT,95) Proprietary (IBM+) to Open 60’s Hardware (Wintel, HP/Intel)
  • 57. The Data Continuum o n pti Perce ence eri Wisdom p Ex Knowledge Integration ext › Multiple points Cont Information Integration of knowledge › Systems Understanding › People › Knowledge Business Systems applied Data › Data in › Technology to Context store and report People Simple › Numbers › Identify › Characters › Capture › Date › Maintain › Time Complex › BLOBs › CLOBs Varied › Content Management
  • 58. The Data Continuum Wisdom Knowledge Knowledge + Application Information Technology + Systems People Data Models Knowledge Data Data Dictionary Management Repository DBMS Files Spreadsheets Graphics Folders Etc.
  • 59. The Data Continuum o n pti An Example: rce Pe ence eri Wisdom Exp Knowledge ext Cont Information Data
  • 60. Other DBMS Market Trends • ODBMS – no longer a threat to become “mainstream” but… • “Post-Relational” DBMS? – XML DBMS – The Associative Model – Multivalue DBMS • In Memory DBMS
  • 61. Enter the DBA • The job of database administration is getting increasingly more difficult as database technology rapidly advances adding new functionality, more options, and more complex and comp- licated capabilities...
  • 62. The DBA is a “Jack of all Trades” OS/390 C++ Windows V$ Tables Linux Oracle SQL XML Java DB2 Informix applet Unix MQ Connect DB2 DNS gateway HTML VTAM CGI TCP/IP ISP connection ZPARMs 3GL database schema COBOL ASP operating system Java HTTP hardware SQL Server network software bridge/router/hub application code VB network cabling CICS SQL*Net JCL
  • 63. So What is Needed? • Intelligent automation of DBA tasks – because no one has all the skilled resources they need – frees up more DBA time They work hard so you don’t have tooooo… “Scrubbing Bubbles” Intelligent Automated
  • 64. A DBA Control Panel DBA-focused on DBA tasks, not focused on systems management tasks like framework products… but integrated with systems management functionality where it makes sense.
  • 65. Contact Information http://www.craigsmullins.com/dba_book.htm Craig S. Mullins Technology Planning Craig_Mullins@BMC.com http://www.craigsmullins.com http://www.bmc.com http://www.craigsmullins.com/cm-book.htm

Editor's Notes

  1. As we discussed, the infrastructure that may be needed to provide a specific service or create a new business model can be very complex. This complexity can be evident both physically and logically. In addition, this infrastructure may change rapidly as the business changes. The first steps in managing a critical business service surround identifying the specific infrastructure that the service needs to run. Essentially, we need to map out the path that the business transactions travel along to better understand how to manage the service. (next)
  2. Depending on what business model is being adopted, a company may be required to integrate into a “virtual” enterprise to deliver specific services or business models. For example: If an organization has acquired an EProcurement application such as Ariba, the “service” will extend into more than just the “in-house” enterprise. You would want to manage the supplier’s system (as it relates to the transaction). You would want to have information from the Shipping company, etc.. No longer can we only think in terms of the “enterprise” we must always look further to determine if the service in question requires integration to a virtual enterprise and determine where management makes sense. Now, we will present the BMC Software approach to Systems Management in this new Era. (next)
  3. Internet commerce services fail many times because there are weak infrastructures supporting them. 1.) Unreliability - The best of the largest retail e-commerce sites achieve 99.4% availability (Keynote Systems, benchmarker). The worst sites achieve only 86.7% availability and costs retailers millions of dollars in revenues. 2.) Complexity - If something crashes, it can take IT personnel hours to figure out where the problem is - routers, servers, operating systems, middleware, etc. 3.) Fragile - Simple upgrades and bug fixes can leave systems in limbo for weeks while IT personnel figure our which configurations won&apos;t crash every few hours. 4.) Vulnerable - Infrastructure today still has weaknesses in which a single event can bring down an entire system. For example, standards on single machine types, like NT, leave whole web sites vulnerable to viruses.
  4. Unplanned outages - unexpected problems. End up having to recover the environment: the longer the recovery, the longer the unplanned outage becomes. Point to customer : Industry is focused on unplanned outages on open systems and trying to minimize number of surprises in the environment. Most companies say that more outage time is spent on planned outages than on unplanned outages. Planned outages are driven by change in the environment. This is becoming a much larger issue for customers. If you could reduce the amount of time on the planned outages, you can have a significant impact on the amount of downtime. BMC is the only vendor in industry that focuses on both planned and unplanned outages, and can reduce the overall impact of outages from an application perspective for a customer.
  5. Unplanned outages - driven by problems Element failure Performance degradation Capacity limitation Application logic error Transaction backout Data corruption Planned outages - driven by change Database maintenance Application migrations Configuration upgrade Data propagation Source of 70% and 30% is CIO feed back during Customer briefings and meetings.
  6. There are two methods of accessing your DB2 databases using Java: JDBC SQLJ
  7. XML is getting a lot of publicity these days. If you believe everything you read, then XML is going to solve all of our interoperability problems, completely replace SQL, and possibly even deliver world peace. Okay, that last one is an exaggeration, but you get the point. In actuality, XML stands for eXtensible Markup Language. The need for extensibility, structure, and validation is the basis for the evolution of the web towards XML. XML, like HTML, is based upon SGML (Standard Generalized Markup Language) which allows documents to be self-describing, through the specification of tag sets and the structural relationships between the tags. HTML is a small, specifically defined set of tags and attributes, enabling users to bypass the self-describing aspect for a document. XML, on the other hand, retains the key SGML advantage of self-description, while avoiding the complexity of full-blown SGML.
  8. XML provides a standard for data interchange. By incorporating XML into DB2 you can more directly and quickly access the XML documents. With DB2 you can search and store entire XML documents using SQL. You also have the option of combining XML documents with traditional data stored in relational tables. When you store or compose a document you can invoke DBMS functions to trigger an event to automate the interchange of data between applications.
  9. However, there are some problems with XML. For example, standard web browsers do not currently understand the descriptive tags. This problem will be alleviated in time as XML-capable web browsers come to market. Another problem with XML is not really the fault of XML, but of market hype. There is a lot of confusion surrounding XML in the industry. Some folks believe that XML will provide metadata where none currently exists or that XML will replace SQL as a data access method for relational data. Neither of these assertions are true. There is no way that any technology, XML included, can conjure up information that does not exist. Humans must create the metadata tags in XML for the data to be described. XML enables self-describing documents. It does not describe your data for you. And XML does not do what SQL does. Hence, XML cannot replace SQL. SQL is the standard access method for relational data. It is used to “tell” a relational DBMS what data is to be retrieved. XML is a document description language. It describes the contents of data. XML may be useful for defining databases, but not for accessing them.