Exadata: How to manage OS dependencies when upgrading to 19c

Upgrading to Exadata 19c is a significant milestone, mostly because it marks the transition from Oracle Linux 6 to Oracle Linux 7. While it’s an exciting jump forward, that OS shift introduces some specific "dependency drama" that can stall your progress if you aren't prepared for it.

I've organized the content below with clear headings to help you navigate these specific dependency hurdles.

Navigating the Linux 7 Transition in Exadata 19c Upgrades

Exadata 19c comes with Linux 7 and Exadata 12c and 18c come with Linux 6. There will then be an upgrade to Linux 7 when upgrading from 12c or 18c to Exadata 19c or above. In this specific case, you may face specific dependencies issues which I will explain how to deal with. This can be split into two more general cases:

• When there is no OS upgrade, I have documented the way of dealing with dependencies in [this blog].
• If you have an OS upgrade — if you are upgrading from 12c or 18c to 19c and above — you will have to follow a specific way of dealing with this due to the upgrade to Linux 7 which I describe below.

Identifying Dependency Conflicts During Pre-checks

When running the usual database servers pre-requisites while upgrading to 19c, the dependencies error message will be slightly different than the usual one (when no OS upgrade happens):

[root@exacel01 dbserver_patch_19.190411]# ./patchmgr -dbnodes ~/dbs_group -precheck -iso_repo /tmp/SAVE/p29542809_*_Linux-x86-64.zip -target_version 19.2.1.0.0.190410 -allow_active_network_mounts  ************************************************************************************************************  NOTE patchmgr release: 19.190411 (always check MOS 1553103.1 for the latest release of dbserver.patch.zip)  ...  2019-05-07 19:57:29 -0500 :ERROR : dbnodeupdate.sh precheck failed on one or more nodes   SUMMARY OF WARNINGS AND ERRORS FOR exadb01:   exadb01: ERROR: Custom packages found that require removal before updating to another major Oracle Linux release, see exadb01:/var/log/cellos/unkown_packages-rpt.070519194937.txt for more details.   exadb01: WARNING: This Exadata update includes an Oracle Linux 6 to Oracle Linux 7 update. 

Analyzing the Custom Package Report

And when looking at the unkown_packages-rpt file:

[root@exadb01]# cat /var/log/cellos/unkown_packages-rpt.070519194937.txt  #################################################################################  # File initialized at 070519_195509 (runid :070519194937) by dbnodeupdate.sh 19.190411  # NOTE: This list contains rpms which are seen as custom rpms  #################################################################################  . . .  list of customs RPMs here  . . .  #################################################################################  # ALERT : Custom packages found (see above)  # These custom packages MUST be removed before proceeding a major Oracle Linux upgrade. 

Strategies for Removing Custom RPMs

In this case, clearly the customs RPMs MUST be removed to be able to upgrade to 19c, and then to Linux 7. We have a few ways of achieving this:

1. Using a generated script (/var/log/cellos/remove_unkown_packages...sh) right now.
2. Using a generated script just before the upgrade.
3. Run dbnodeupdate.sh with the additional -R flag at update time to have these packages automatically removed — this is the patchmgr option -force_remove_custom_rpms.

Recommendation: Automating Removal with patchmgr

I personally recommend going with the patchmgr option -force_remove_custom_rpms, which would do everything for us at upgrade time.

One thing: if we check the patchmgr documentation, -force_remove_custom_rpms is still technically described for the upgrade from Linux 5 to Linux 6. Oracle may have missed updating the patchmgr help, but you can indeed use -force_remove_custom_rpms for the move from OL6 to OL7 as well.

In the case of an upgrade to Exadata 19c, simply add the flag to your command:

[root@exacel01 dbserver_patch_19.190411]# nohup ./patchmgr -dbnodes ~/dbs_group -upgrade -iso_repo /tmp/SAVE/p29542809_*_Linux-x86-64.zip -target_version 19.2.1.0.0.190410 -allow_active_network_mounts -force_remove_custom_rpms -rolling & 

Summary Rule for Exadata Patching

Following these steps will get you past the pre-check errors. Just remember to organize the re-installation of your custom RPMs after patching (or use this as an excuse for a cleanup!).

• No OS upgrade: Follow the standard dependency handling.
• OS upgrade (OL6 to OL7): Add -force_remove_custom_rpms to your patchmgr command.

Enjoy your Exadata in 19c!