Weblogic patch rollback issues in Oracle EBS R12.2

When you try to rollback a weblogic patch in Oracle EBS R12.2, you might run into issues similar to below:

$ ./bsu.sh -remove -patchlist=YIJF -prod_dir=/u01/install/VISION/fs2/FMW_Home/wlserver_10.3 -verbose  Checking for conflicts..  No conflict(s) detected Starting removal of Patch ID: YIJF Restoring /d1/V1223B5B/fs1/FMW_Home/modules/com.bea.core.datasource6.binding_1.10.0.0.jar from /u01/install/VISION/fs2/FMW_Home/patch_wls1036/backup/backup.jar Result: Failure Failure condition follows: An error occured while removing patches – run with logging to obtain detailed information java.io.FileNotFoundException: /d1/V1223B5B/fs1/FMW_Home/modules/com.bea.core.datasource6.binding_1.10.0.0.jar (No such file or directory) Result: Failure Failure condition follows: An error occured while removing patches – run with logging to obtain detailed information java.io.FileNotFoundException: /d1/V1223B5B/fs1/FMW_Home/modules/com.bea.core.datasource6.binding_1.10.0.0.jar (No such file or directory) 

1. Analyzing the BSU Rollback Failure

Understanding the Hardcoded Path Issue

If you observe the errors, it’s trying to restore the jar file to a location that is not present in the current filesystem. This error was captured in a VISION instance created from Oracle VM Template. So you can see that the file path seems to be a path used by an Oracle internal system where the VM template is created. There is not much harm caused by the above issue to the instance.

2. Potential Impact on Production Environments

Risks of Rollback in Dual File System Architectures

Now consider this hypothetical situation, where you are trying to rollback a weblogic patch in patch fs ( fs2 ) that was previously applied to fs1 in Production Oracle E-Business Suite R12.2. So the patch history in the patch fs ( fs2) will still have file paths of fs1. So when you rollback a patch it will restore the files to RUN FS ( fs1). This can cause a havoc and a big outage to online users.

3. Preventive Measures and Recommendations

Verifying the patch-backup.xml Registry

As of now bsu utility doesn’t seem to handle issues this kind of situation. So all Oracle EBS Apps DBAs out there, make sure to check the below file for correct paths, before you try to rollback any weblogic patch.

$ vi $FMW_HOME/patch_wls1036/registry/patch-backup.xml 

Looking Toward a Permanent Fix

A permanent solution to this problem would be to update bsu patch utility to not restore or touch files outside the weblogic home. Hope Oracle Development notices this issue before it effects some customers EBS production instance.

Oracle Database Consulting Services

Ready to optimize your Oracle Database for the future?