SlideShare a Scribd company logo

Exadata Patching Demystified

•
•
E
Enkitec

This document provides an overview of Exadata patching. It discusses that patching has improved over time. Oracle will patch Exadata systems for customers with support contracts. Exadata patches are applied using patchmgr and involve pushing new OS images to storage cells which reboot multiple times. Database servers are patched using yum. Quarterly database patches contain RDBMS, CRS, and Diskmon patches applied together using opatch. It is important to test patches in non-production first and have a patching plan.

Related slideshows

Whats new in Autonomous Database in 2022
Whats new in Autonomous Database in 2022Whats new in Autonomous Database in 2022
Whats new in Autonomous Database in 2022
 
Real-Time Query for Data Guard
Real-Time Query for Data Guard Real-Time Query for Data Guard
Real-Time Query for Data Guard
 
Oracle RAC 19c: Best Practices and Secret Internals
Oracle RAC 19c: Best Practices and Secret InternalsOracle RAC 19c: Best Practices and Secret Internals
Oracle RAC 19c: Best Practices and Secret Internals
 
1 of 32
Exadata Patching Demystified Presented by: Andy Colvin August 14, 2012
About Me  Working around Oracle since 1999  Background in systems, network, database  6 years at Enkitec  Working on Exadata for 2+ years
Exadata Patching Survey  How many people patch themselves?  How many have had problems patching?  How many never patch?
A Quick Disclaimer  Patching was very rough in the beginning  It’s much better now!
 Do I have to patch every month?  Oracle will patch my Exadata for me*  If I don’t patch my Exadata, Oracle won’t support me  Exadata patches are scary
Why Patch?  Bug fixes  New features  Security updates
Where to Start?  MOS Note #888828.1  Latest patch updates  New patching methods  Links to patch notes for other products
Oracle-Provided Patches Compute Nodes Exadata Storage  Oracle Stack Servers  Linux/Solaris  Intelligent storage  Always run Linux  Proprietary Infiniband Switches application - cellsrv  Custom CentOS
Exadata Storage Server Patches Exadata Storage Server  Single patch  Operating system •  Firmware Single patch download • SameDriversapplies to all hardware  patch • Exadata software, OS, ILOM, firmware  Updates storage and compute nodes • README and Support Note  Released quarterly (usually) # imageinfo Active image version: 11.2.2.3.2.110520
Exadata Storage Server Versions 11.2.3.1.1 DB Major Release Maintenance Release DB Minor Release Feature Release DB Patchset
Exadata Storage Server Versions  What version am I running? [root@enkcel03 ~]# imageinfo Kernel version: 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 Cell version: OSS_11.2.3.1.1_LINUX.X64_120607 Cell rpm version: cell-11.2.3.1.1_LINUX.X64_120607-1 Active image version: 11.2.3.1.1.120607 Active image activated: 2012-06-28 21:45:30 -0500 Active image status: success Active system partition on device: /dev/md6 Active software partition on device: /dev/md8
Exadata Storage Server Patches  Apply with patchmgr – included with the patch release  Run from compute node  Uses dcli and SSH keys to deploy patch across storage grid
What Happens During Patching?  New OS image pushed to inactive partitions on each cell  Filesystem labels are switched to newly patched partitions  The cell reboots and firmware is patched – multiple reboots are common  Post-install validation checks are run  USB recovery media is updated to match the new software version
Rolling Storage Server Patching offline disks  No outage required online cel01 disks  Single command ASM resync Patch Staging end-to-end offline disks online cel02  Consider your disks workload ASM resync  6 hours – ¼ rack offline disks online celNN disks ASM resync
Non-Rolling Storage Server Patching  Full outage cel01  1.5 – 3 hours Patch Staging  All cells in parallel cel02  Shorter maintenance window celNN
Patching Database Servers  Database servers updated with storage servers  Utilize yum for updates  Each version has a yum channel  exadata_dbserver_11.2_x86_64_latest  exadata_dbserver_11.2.3.1.0_x86_64_base  exadata_dbserver_11.2.3.1.0_x86_64_patch
Patching Database Servers  Database servers updated along with storage servers  Utilize yum for updates  Replaces old “minimal pack”  Each version has a yum channel  exadata_dbserver_11.2_x86_64_latest  exadata_dbserver_11.2.3.1.0_x86_64_base  exadata_dbserver_11.2.3.1.0_x86_64_patch
Patching Database Servers  Compute nodes must be running OEL 5.5  V2 Exadata owners – see MOS note #1284070.1  Several options for installation  Update directly from ULN  Configure local yum repository  Mount ISO file containing RPMs
Quarterly Database Patch for Exadata  Patches for RDBMS & Grid Infrastructure  Applied using OPatch  Each BP/QDPE contains 3 patches, applied together  RDBMS  CRS  Diskmon  Patches are cumulative
Quarterly Database Patch for Exadata  Originally called bundle patches  Release frequency  Current patchset (11.2.0.3) – quarterly recommended patches, with monthly bundles  Previous patchset (11.2.0.2) - quarterly  Oracle recommends quarterly patching according to CPU/PSU cycle
QDPE / BP Versions  Check patch version with “opatch lsinventory” [oracle@enkdb01 ~]$ opatch lsinventory Interim patches (3) : Patch 13688022 : applied on Fri Apr 27 14:52:31 CDT 2012 Unique Patch ID: 14639565 Patch description: "Diskmon Patch for Exadata (APR 2012 - 11.2.0.3.5) : (13688022)” Patch 13696251 : applied on Fri Apr 27 14:51:36 CDT 2012 Unique Patch ID: 14639565 Patch description: "CRS Patch for Exadata (APR 2012 - 11.2.0.3.5) : (13696251)” Patch 13734832 : applied on Fri Apr 27 14:49:34 CDT 2012 Unique Patch ID: 14639565 Patch description: "Database Patch for Exadata (APR 2012 - 11.2.0.3.5) : (13734832)”
QDPE / BP Application  Always rolling patches  Applied one node at a time  Use opatch auto to apply  Requires root privileges  sudo is acceptable  Post-install database scripts
Patchset Upgrades  Check MOS #888828.1 for certification  Read upgrade-specific notes before upgrading  11.2.0.2 – MOS #1315926.1  11.2.0.3 – MOS #1373255.1  GI upgrade always rolling  Database upgrade requires an outage*
Patchset Upgrades  Minimum cell version requirements  11.2.0.2 – cell version 11.2.2.1.1+  11.2.0.3 – cell version 11.2.2.4.0+  Install latest BP on current release before upgrading GI/RDBMS  Apply BP/QDPE after installing software, before running upgrade scripts
Infiniband Patches  Released yearly  Infiniband switches run CentOS Linux  Patch method depends on version  Rolling patches – no outage required  Spine switch first, followed by leaf switches
What’s a QFSDP?  Quarterly Full Stack Download Patch  Released January, April, July, October  Contains latest patches for all stacks  Infrastructure  Database  OEM  Patches still installed individually  You may not need to apply all patches (Infiniband, PDU, etc)
Minimizing Patching Risk  Create a concise patching plan  Read all of the notes before starting  Patch a non-production system first  Patch the same way on all systems
Standby First Patch Apply  Apply patches to standby system  Perform testing with snapshot standby or active data guard  Dataguard switchover  Patch previous production system  Switch back to production or leave as is for next patch cycle
OPlan  Used with BP/QDPE deployment  Creates patch deployment report customized to your environment  Includes steps for in place, out of place patch and rollback  MOS Note #1306814.1  Patch #11846294  Available for 11.2.0.2 and 11.2.0.3 bundle patches or QDPEs
A Quick Note on Platinum Services  Read the T’s and C’s before you trust sales  Oracle will patch for free (to a certain extent)  Monitoring is great
Useful MOS Notes  Supported versions (11.2) - #888828.1  e-Business Suite patch compatibility - #1392527.1  OPlan – #1306814.1  Exachk – #1070954.1  11.2.0.2 upgrade – #1315926.1  11.2.0.3 upgrade – #1373255.1
Questions? Contact Information: Andy Colvin email - andy.colvin@enkitec.com web - http://www.enkitec.com blog – http://blog.oracle-ninja.com twitter - @acolvin

More Related Content

Exadata Patching Demystified

  • 1. Exadata Patching Demystified Presented by: Andy Colvin August 14, 2012
  • 2. About Me  Working around Oracle since 1999  Background in systems, network, database  6 years at Enkitec  Working on Exadata for 2+ years
  • 3. Exadata Patching Survey  How many people patch themselves?  How many have had problems patching?  How many never patch?
  • 4. A Quick Disclaimer  Patching was very rough in the beginning  It’s much better now!
  • 5.  Do I have to patch every month?  Oracle will patch my Exadata for me*  If I don’t patch my Exadata, Oracle won’t support me  Exadata patches are scary
  • 6. Why Patch?  Bug fixes  New features  Security updates
  • 7. Where to Start?  MOS Note #888828.1  Latest patch updates  New patching methods  Links to patch notes for other products
  • 8. Oracle-Provided Patches Compute Nodes Exadata Storage  Oracle Stack Servers  Linux/Solaris  Intelligent storage  Always run Linux  Proprietary Infiniband Switches application - cellsrv  Custom CentOS
  • 9. Exadata Storage Server Patches Exadata Storage Server  Single patch  Operating system •  Firmware Single patch download • SameDriversapplies to all hardware  patch • Exadata software, OS, ILOM, firmware  Updates storage and compute nodes • README and Support Note  Released quarterly (usually) # imageinfo Active image version: 11.2.2.3.2.110520
  • 10. Exadata Storage Server Versions 11.2.3.1.1 DB Major Release Maintenance Release DB Minor Release Feature Release DB Patchset
  • 11. Exadata Storage Server Versions  What version am I running? [root@enkcel03 ~]# imageinfo Kernel version: 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64 Cell version: OSS_11.2.3.1.1_LINUX.X64_120607 Cell rpm version: cell-11.2.3.1.1_LINUX.X64_120607-1 Active image version: 11.2.3.1.1.120607 Active image activated: 2012-06-28 21:45:30 -0500 Active image status: success Active system partition on device: /dev/md6 Active software partition on device: /dev/md8
  • 12. Exadata Storage Server Patches  Apply with patchmgr – included with the patch release  Run from compute node  Uses dcli and SSH keys to deploy patch across storage grid
  • 13. What Happens During Patching?  New OS image pushed to inactive partitions on each cell  Filesystem labels are switched to newly patched partitions  The cell reboots and firmware is patched – multiple reboots are common  Post-install validation checks are run  USB recovery media is updated to match the new software version
  • 14. Rolling Storage Server Patching offline disks  No outage required online cel01 disks  Single command ASM resync Patch Staging end-to-end offline disks online cel02  Consider your disks workload ASM resync  6 hours – ¼ rack offline disks online celNN disks ASM resync
  • 15. Non-Rolling Storage Server Patching  Full outage cel01  1.5 – 3 hours Patch Staging  All cells in parallel cel02  Shorter maintenance window celNN
  • 16. Patching Database Servers  Database servers updated with storage servers  Utilize yum for updates  Each version has a yum channel  exadata_dbserver_11.2_x86_64_latest  exadata_dbserver_11.2.3.1.0_x86_64_base  exadata_dbserver_11.2.3.1.0_x86_64_patch
  • 17. Patching Database Servers  Database servers updated along with storage servers  Utilize yum for updates  Replaces old “minimal pack”  Each version has a yum channel  exadata_dbserver_11.2_x86_64_latest  exadata_dbserver_11.2.3.1.0_x86_64_base  exadata_dbserver_11.2.3.1.0_x86_64_patch
  • 18. Patching Database Servers  Compute nodes must be running OEL 5.5  V2 Exadata owners – see MOS note #1284070.1  Several options for installation  Update directly from ULN  Configure local yum repository  Mount ISO file containing RPMs
  • 19. Quarterly Database Patch for Exadata  Patches for RDBMS & Grid Infrastructure  Applied using OPatch  Each BP/QDPE contains 3 patches, applied together  RDBMS  CRS  Diskmon  Patches are cumulative
  • 20. Quarterly Database Patch for Exadata  Originally called bundle patches  Release frequency  Current patchset (11.2.0.3) – quarterly recommended patches, with monthly bundles  Previous patchset (11.2.0.2) - quarterly  Oracle recommends quarterly patching according to CPU/PSU cycle
  • 21. QDPE / BP Versions  Check patch version with “opatch lsinventory” [oracle@enkdb01 ~]$ opatch lsinventory Interim patches (3) : Patch 13688022 : applied on Fri Apr 27 14:52:31 CDT 2012 Unique Patch ID: 14639565 Patch description: "Diskmon Patch for Exadata (APR 2012 - 11.2.0.3.5) : (13688022)” Patch 13696251 : applied on Fri Apr 27 14:51:36 CDT 2012 Unique Patch ID: 14639565 Patch description: "CRS Patch for Exadata (APR 2012 - 11.2.0.3.5) : (13696251)” Patch 13734832 : applied on Fri Apr 27 14:49:34 CDT 2012 Unique Patch ID: 14639565 Patch description: "Database Patch for Exadata (APR 2012 - 11.2.0.3.5) : (13734832)”
  • 22. QDPE / BP Application  Always rolling patches  Applied one node at a time  Use opatch auto to apply  Requires root privileges  sudo is acceptable  Post-install database scripts
  • 23. Patchset Upgrades  Check MOS #888828.1 for certification  Read upgrade-specific notes before upgrading  11.2.0.2 – MOS #1315926.1  11.2.0.3 – MOS #1373255.1  GI upgrade always rolling  Database upgrade requires an outage*
  • 24. Patchset Upgrades  Minimum cell version requirements  11.2.0.2 – cell version 11.2.2.1.1+  11.2.0.3 – cell version 11.2.2.4.0+  Install latest BP on current release before upgrading GI/RDBMS  Apply BP/QDPE after installing software, before running upgrade scripts
  • 25. Infiniband Patches  Released yearly  Infiniband switches run CentOS Linux  Patch method depends on version  Rolling patches – no outage required  Spine switch first, followed by leaf switches
  • 26. What’s a QFSDP?  Quarterly Full Stack Download Patch  Released January, April, July, October  Contains latest patches for all stacks  Infrastructure  Database  OEM  Patches still installed individually  You may not need to apply all patches (Infiniband, PDU, etc)
  • 27. Minimizing Patching Risk  Create a concise patching plan  Read all of the notes before starting  Patch a non-production system first  Patch the same way on all systems
  • 28. Standby First Patch Apply  Apply patches to standby system  Perform testing with snapshot standby or active data guard  Dataguard switchover  Patch previous production system  Switch back to production or leave as is for next patch cycle
  • 29. OPlan  Used with BP/QDPE deployment  Creates patch deployment report customized to your environment  Includes steps for in place, out of place patch and rollback  MOS Note #1306814.1  Patch #11846294  Available for 11.2.0.2 and 11.2.0.3 bundle patches or QDPEs
  • 30. A Quick Note on Platinum Services  Read the T’s and C’s before you trust sales  Oracle will patch for free (to a certain extent)  Monitoring is great
  • 31. Useful MOS Notes  Supported versions (11.2) - #888828.1  e-Business Suite patch compatibility - #1392527.1  OPlan – #1306814.1  Exachk – #1070954.1  11.2.0.2 upgrade – #1315926.1  11.2.0.3 upgrade – #1373255.1
  • 32. Questions? Contact Information: Andy Colvin email - andy.colvin@enkitec.com web - http://www.enkitec.com blog – http://blog.oracle-ninja.com twitter - @acolvin