Blog | Pythian

How To Downgrade Grid 19c Clusterware to 12.1

Written by Michael Dinh | Dec 18, 2019 5:00:00 AM
There may be many reasons for downgrading from Grid 19c to 12.1; however, the reason I am downgrading is to rerun a test requested by Oracle support from the 12.1 clusterware. Here are the steps performed.

Check cluster:

oracle@ol7-121-rac2 ~]$ crsctl check cluster -all
 **************************************************************
 ol7-121-rac1:
 CRS-4537: Cluster Ready Services is online
 CRS-4529: Cluster Synchronization Services is online
 CRS-4533: Event Manager is online
 **************************************************************
 ol7-121-rac2:
 CRS-4537: Cluster Ready Services is online
 CRS-4529: Cluster Synchronization Services is online
 CRS-4533: Event Manager is online
 **************************************************************
 [oracle@ol7-121-rac2 ~]$

Downgrade Node2:

(Can run the downgrade script in parallel on all cluster nodes, but one.) 
[root@ol7-121-rac2 ~]# env|egrep -i 'ORA|HOME|TNS'
 HOME=/root
 
 [root@ol7-121-rac2 ~]# ls /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh
 /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh
 
 [root@ol7-121-rac2 ~]# /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh -downgrade
 Using configuration parameter file: /u01/app/19.3.0.0/grid/crs/install/crsconfig_params
 The log of current session can be found at:
  /u01/app/oracle/crsdata/ol7-121-rac2/crsconfig/crsdowngrade_ol7-121-rac2_2019-12-09_10-58-49PM.log
 2019/12/09 23:04:14 CLSRSC-4006: Removing Oracle Trace File Analyzer (TFA) Collector.
 2019/12/09 23:05:04 CLSRSC-4007: Successfully removed Oracle Trace File Analyzer (TFA) Collector.
 2019/12/09 23:05:04 CLSRSC-591: successfully downgraded Oracle Clusterware stack on this node
 [root@ol7-121-rac2 ~]#

Downgrade Node1:

(Downgrade the last node after downgrade all other nodes) 
[root@ol7-121-rac1 ~]# env|egrep -i 'ORA|HOME|TNS'
 HOME=/root
 
 [root@ol7-121-rac1 ~]# ls /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh
 /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh
 
 [root@ol7-121-rac1 ~]# /u01/app/19.3.0.0/grid/crs/install/rootcrs.sh -downgrade
 Using configuration parameter file: /u01/app/19.3.0.0/grid/crs/install/crsconfig_params
 The log of current session can be found at:
  /u01/app/oracle/crsdata/ol7-121-rac1/crsconfig/crsdowngrade_ol7-121-rac1_2019-12-09_11-06-04PM.log
 CRS-4123: Oracle High Availability Services has been started.
 CRS-2672: Attempting to start 'ora.evmd' on 'ol7-121-rac1'
 CRS-2672: Attempting to start 'ora.mdnsd' on 'ol7-121-rac1'
 CRS-2676: Start of 'ora.evmd' on 'ol7-121-rac1' succeeded
 CRS-2676: Start of 'ora.mdnsd' on 'ol7-121-rac1' succeeded
 CRS-2672: Attempting to start 'ora.gpnpd' on 'ol7-121-rac1'
 CRS-2676: Start of 'ora.gpnpd' on 'ol7-121-rac1' succeeded
 CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ol7-121-rac1'
 CRS-2672: Attempting to start 'ora.gipcd' on 'ol7-121-rac1'
 CRS-2676: Start of 'ora.cssdmonitor' on 'ol7-121-rac1' succeeded
 CRS-2676: Start of 'ora.gipcd' on 'ol7-121-rac1' succeeded
 CRS-2672: Attempting to start 'ora.cssd' on 'ol7-121-rac1'
 CRS-2672: Attempting to start 'ora.diskmon' on 'ol7-121-rac1'
 CRS-2676: Start of 'ora.diskmon' on 'ol7-121-rac1' succeeded
 CRS-2676: Start of 'ora.cssd' on 'ol7-121-rac1' succeeded
 CRS-2672: Attempting to start 'ora.crf' on 'ol7-121-rac1'
 CRS-2672: Attempting to start 'ora.ctssd' on 'ol7-121-rac1'
 CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'ol7-121-rac1'
 CRS-2676: Start of 'ora.crf' on 'ol7-121-rac1' succeeded
 CRS-2676: Start of 'ora.ctssd' on 'ol7-121-rac1' succeeded
 CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'ol7-121-rac1' succeeded
 CRS-2672: Attempting to start 'ora.asm' on 'ol7-121-rac1'
 CRS-2676: Start of 'ora.asm' on 'ol7-121-rac1' succeeded
 CRS-2672: Attempting to start 'ora.storage' on 'ol7-121-rac1'
 CRS-2676: Start of 'ora.storage' on 'ol7-121-rac1' succeeded
 CRS-2672: Attempting to start 'ora.crsd' on 'ol7-121-rac1'
 CRS-2676: Start of 'ora.crsd' on 'ol7-121-rac1' succeeded
 2019/12/09 23:18:00 CLSRSC-338: Successfully downgraded OCR to version 12.1.0.2.0
 CRS-2672: Attempting to start 'ora.crf' on 'ol7-121-rac1'
 CRS-2672: Attempting to start 'ora.storage' on 'ol7-121-rac1'
 CRS-2676: Start of 'ora.storage' on 'ol7-121-rac1' succeeded
 CRS-2676: Start of 'ora.crf' on 'ol7-121-rac1' succeeded
 CRS-2672: Attempting to start 'ora.crsd' on 'ol7-121-rac1'
 CRS-2676: Start of 'ora.crsd' on 'ol7-121-rac1' succeeded
 2019/12/09 23:18:23 CLSRSC-4006: Removing Oracle Trace File Analyzer (TFA) Collector.
 2019/12/09 23:18:49 CLSRSC-4007: Successfully removed Oracle Trace File Analyzer (TFA) Collector.
 2019/12/09 23:18:50 CLSRSC-591: successfully downgraded Oracle Clusterware stack on this node
 2019/12/09 23:18:51 CLSRSC-640: To complete the downgrade operation, ensure that the node inventory on all nodes points to the configured Grid Infrastructure home '/u01/app/12.1.0.2/grid'.
 2019/12/09 23:18:52 CLSRSC-592: Run 'crsctl start crs' from home /u01/app/12.1.0.2/grid on each node to complete downgrade.
 2019/12/09 23:19:13 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
 [root@ol7-121-rac1 ~]#

Remove Grid 19c from Clusterware:

[oracle@ol7-121-rac1 ~]$ /u01/app/19.3.0.0/grid/oui/bin/runInstaller -nowait -waitforcompletion -ignoreSysPrereqs \
 > -updateNodeList -silent CRS=false ORACLE_HOME=/u01/app/19.3.0.0/grid \
 > "CLUSTER_NODES=ol7-121-rac1, ol7-121-rac2" -doNotUpdateNodeList
 Starting Oracle Universal Installer...
 
 Checking swap space: must be greater than 500 MB. Actual 18431 MB Passed
 The inventory pointer is located at /etc/oraInst.loc
 You can find the log of this install session at:
  /u01/app/oraInventory/logs/UpdateNodeList2019-12-09_11-57-44PM.log
 'UpdateNodeList' was successful.
 [oracle@ol7-121-rac1 ~]$

Configure Grid 12c with Clusterware:

[oracle@ol7-121-rac1 ~]$ /u01/app/12.1.0.2/grid/oui/bin/runInstaller -nowait -waitforcompletion -ignoreSysPrereqs \
 > -updateNodeList -silent CRS=true ORACLE_HOME=/u01/app/12.1.0.2/grid \
 > "CLUSTER_NODES=ol7-121-rac1, ol7-121-rac2"
 Starting Oracle Universal Installer...
 
 Checking swap space: must be greater than 500 MB. Actual 18431 MB Passed
 The inventory pointer is located at /etc/oraInst.loc
 'UpdateNodeList' was successful.
 [oracle@ol7-121-rac1 ~]$
### Start crs for Node1 
[root@ol7-121-rac1 ~]# . oraenv <<< +ASM1
 ORACLE_SID = [root] ? The Oracle base has been set to /u01/app/oracle
 
 [root@ol7-121-rac1 ~]# crsctl start crs
 CRS-4123: Oracle High Availability Services has been started.
 [root@ol7-121-rac1 ~]#

Start crs for Node2:

[root@ol7-121-rac2 ~]# . oraenv <<< +ASM2
 ORACLE_SID = [root] ? The Oracle base has been set to /u01/app/oracle
 
 [root@ol7-121-rac2 ~]# crsctl start crs
 CRS-4123: Oracle High Availability Services has been started.
 [root@ol7-121-rac2 ~]#

Validate Cluster:

[oracle@ol7-121-rac2 ~]$ olsnodes
 ol7-121-rac1
 ol7-121-rac2
 
 [oracle@ol7-121-rac2 ~]$ crsctl check cluster -all
 **************************************************************
 ol7-121-rac1:
 CRS-4537: Cluster Ready Services is online
 CRS-4529: Cluster Synchronization Services is online
 CRS-4533: Event Manager is online
 **************************************************************
 ol7-121-rac2:
 CRS-4537: Cluster Ready Services is online
 CRS-4529: Cluster Synchronization Services is online
 CRS-4533: Event Manager is online
 **************************************************************
 
 [oracle@ol7-121-rac2 ~]$ crsctl query crs softwareversion
 Oracle Clusterware version on node [ol7-121-rac2] is [12.1.0.2.0]
 
 [oracle@ol7-121-rac2 ~]$ crsctl query crs softwarepatch
 Oracle Clusterware patch level on node ol7-121-rac2 is [365281826].
 
 [oracle@ol7-121-rac2 ~]$ crsctl query crs releaseversion
 Oracle High Availability Services release version on the local node is [12.1.0.2.0]
 
 [oracle@ol7-121-rac2 ~]$ crsctl query crs releasepatch
 Oracle Clusterware release patch level is [365281826] and the complete list of patches [28553832 ] have been applied on the local node.
 
 [oracle@ol7-121-rac2 ~]$ crsctl query crs activeversion -f
 Oracle Clusterware active version on the cluster is [12.1.0.2.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [365281826].
 [oracle@ol7-121-rac2 ~]$

Validate inventory:

[oracle@ol7-121-rac2 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -all|head -50
 Oracle Interim Patch Installer version 12.2.0.1.18
 Copyright (c) 2019, Oracle Corporation. All rights reserved.
 
 
 Oracle Home : /u01/app/12.1.0.2/grid
 Central Inventory : /u01/app/oraInventory
  from : /u01/app/12.1.0.2/grid/oraInst.loc
 OPatch version : 12.2.0.1.18
 OUI version : 12.1.0.2.0
 Log file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/opatch2019-12-10_00-22-52AM_1.log
 
 Lsinventory Output file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/lsinv/lsinventory2019-12-10_00-22-52AM.txt
 --------------------------------------------------------------------------------
 Local Machine Information::
 Hostname: ol7-121-rac2.localdomain
 ARU platform id: 226
 ARU platform description:: Linux x86-64
 
 List of Oracle Homes:
  Name Location
  OraGI12Home1 /u01/app/12.1.0.2/grid
  OraDB12Home1 /u01/app/oracle/product/12.1.0.2/dbhome_1
  OraGI19Home1 /u01/app/19.3.0.0/grid
 
 Installed Top-level Products (1):
 
 Oracle Grid Infrastructure 12c 12.1.0.2.0
 There are 1 products installed in this Oracle Home.
 
 
 Interim patches (1) :
 
 Patch 28553832 : applied on Fri Dec 06 16:38:26 UTC 2019
 Unique Patch ID: 22732107
 Patch description: "OCW Interim patch for 28553832"
  Created on 11 Feb 2019, 02:30:42 hrs PST8PDT
  Bugs fixed:
  21339083, 24831217, 18589889, 20768643, 19061429, 26512962, 26147987
  19133945, 19341538, 19521169, 20011424, 19361757, 19187207, 21123932
  19302350, 19130141, 23340259, 19530755, 21441387, 19028836, 18701017
  19699720, 19620918, 19168690, 18899171, 19244316, 17005215, 19653795
  19471722, 18634372, 19536337, 19184188, 26973611, 19131709, 20160490
  20995001, 21131037, 24509984, 19380733, 20985167, 18943696, 20552947
  19591984, 18135723, 18461246, 20014326, 19524857, 25250824, 22999793
  18890943, 19154753, 25055154, 24808260, 19499021, 25206628, 20923930
  19046190, 25074091, 20028666, 23170674, 24451580, 19051385, 19682695
  19050688, 19831611, 22891868, 21607972, 26755613, 18922918, 21087799
  25956724, 18956780, 19683886, 23058136, 23757020, 19026993, 20110156
  21269977, 18261648, 18362971, 19558324, 22651583, 20185476, 18882642
  19478119, 17447588, 19414274, 19262534, 20752167, 19147513, 22475834
 [oracle@ol7-121-rac2 ~]$

Deinstall Grid 19c Clusterware for Node1:

[oracle@ol7-121-rac1 ~]$ /u01/app/19.3.0.0/grid/oui/bin/runInstaller -detachHome -silent ORACLE_HOME=/u01/app/19.3.0.0/grid
 Starting Oracle Universal Installer...
 
 Checking swap space: must be greater than 500 MB. Actual 18431 MB Passed
 The inventory pointer is located at /etc/oraInst.loc
 You can find the log of this install session at:
  /u01/app/oraInventory/logs/DetachHome2019-12-10_12-24-09AM.log
 'DetachHome' was successful.
 [oracle@ol7-121-rac1 ~]$

Deinstall Grid 19c Clusterware for Node2:

[oracle@ol7-121-rac2 ~]$ /u01/app/19.3.0.0/grid/oui/bin/runInstaller -detachHome -silent ORACLE_HOME=/u01/app/19.3.0.0/grid
 Starting Oracle Universal Installer...
 
 Checking swap space: must be greater than 500 MB. Actual 18431 MB Passed
 The inventory pointer is located at /etc/oraInst.loc
 You can find the log of this install session at:
  /u01/app/oraInventory/logs/DetachHome2019-12-10_12-24-39AM.log
 'DetachHome' was successful.
 [oracle@ol7-121-rac2 ~]$

Check inventory for Node1:

[oracle@ol7-121-rac1 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -all|head -50
 Oracle Interim Patch Installer version 12.2.0.1.18
 Copyright (c) 2019, Oracle Corporation. All rights reserved.
 
 
 Oracle Home : /u01/app/oracle/product/12.1.0.2/dbhome_1
 Central Inventory : /u01/app/oraInventory
  from : /u01/app/oracle/product/12.1.0.2/dbhome_1/oraInst.loc
 OPatch version : 12.2.0.1.18
 OUI version : 12.1.0.2.0
 Log file location : /u01/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/opatch/opatch2019-12-10_00-34-03AM_1.log
 
 Lsinventory Output file location : /u01/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2019-12-10_00-34-03AM.txt
 --------------------------------------------------------------------------------
 Local Machine Information::
 Hostname: ol7-121-rac1.localdomain
 ARU platform id: 226
 ARU platform description:: Linux x86-64
 
 List of Oracle Homes:
  Name Location
  OraGI12Home1 /u01/app/12.1.0.2/grid
  OraDB12Home1 /u01/app/oracle/product/12.1.0.2/dbhome_1
 
 Installed Top-level Products (1):
 
 Oracle Database 12c 12.1.0.2.0
 There are 1 products installed in this Oracle Home.
 
 
 Interim patches (1) :
 
 Patch 28553832 : applied on Fri Dec 06 16:54:30 UTC 2019
 Unique Patch ID: 22732107
 Patch description: "OCW Interim patch for 28553832"
  Created on 11 Feb 2019, 02:30:42 hrs PST8PDT
  Bugs fixed:
  21339083, 24831217, 18589889, 20768643, 19061429, 26512962, 26147987
  19133945, 19341538, 19521169, 20011424, 19361757, 19187207, 21123932
  19302350, 19130141, 23340259, 19530755, 21441387, 19028836, 18701017
  19699720, 19620918, 19168690, 18899171, 19244316, 17005215, 19653795
  19471722, 18634372, 19536337, 19184188, 26973611, 19131709, 20160490
  20995001, 21131037, 24509984, 19380733, 20985167, 18943696, 20552947
  19591984, 18135723, 18461246, 20014326, 19524857, 25250824, 22999793
  18890943, 19154753, 25055154, 24808260, 19499021, 25206628, 20923930
  19046190, 25074091, 20028666, 23170674, 24451580, 19051385, 19682695
  19050688, 19831611, 22891868, 21607972, 26755613, 18922918, 21087799
  25956724, 18956780, 19683886, 23058136, 23757020, 19026993, 20110156
  21269977, 18261648, 18362971, 19558324, 22651583, 20185476, 18882642
  19478119, 17447588, 19414274, 19262534, 20752167, 19147513, 22475834
  18893463, 21242118, 19514085, 25496699, 20758997, 27738131, 27006180
 [oracle@ol7-121-rac1 ~]$

Check inventory for Node2:

[oracle@ol7-121-rac2 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -all|head -50
 Oracle Interim Patch Installer version 12.2.0.1.18
 Copyright (c) 2019, Oracle Corporation. All rights reserved.
 
 
 Oracle Home : /u01/app/12.1.0.2/grid
 Central Inventory : /u01/app/oraInventory
  from : /u01/app/12.1.0.2/grid/oraInst.loc
 OPatch version : 12.2.0.1.18
 OUI version : 12.1.0.2.0
 Log file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/opatch2019-12-10_00-24-54AM_1.log
 
 Lsinventory Output file location : /u01/app/12.1.0.2/grid/cfgtoollogs/opatch/lsinv/lsinventory2019-12-10_00-24-54AM.txt
 --------------------------------------------------------------------------------
 Local Machine Information::
 Hostname: ol7-121-rac2.localdomain
 ARU platform id: 226
 ARU platform description:: Linux x86-64
 
 List of Oracle Homes:
  Name Location
  OraGI12Home1 /u01/app/12.1.0.2/grid
  OraDB12Home1 /u01/app/oracle/product/12.1.0.2/dbhome_1
 
 Installed Top-level Products (1):
 
 Oracle Grid Infrastructure 12c 12.1.0.2.0
 There are 1 products installed in this Oracle Home.
 
 
 Interim patches (1) :
 
 Patch 28553832 : applied on Fri Dec 06 16:38:26 UTC 2019
 Unique Patch ID: 22732107
 Patch description: "OCW Interim patch for 28553832"
  Created on 11 Feb 2019, 02:30:42 hrs PST8PDT
  Bugs fixed:
  21339083, 24831217, 18589889, 20768643, 19061429, 26512962, 26147987
  19133945, 19341538, 19521169, 20011424, 19361757, 19187207, 21123932
  19302350, 19130141, 23340259, 19530755, 21441387, 19028836, 18701017
  19699720, 19620918, 19168690, 18899171, 19244316, 17005215, 19653795
  19471722, 18634372, 19536337, 19184188, 26973611, 19131709, 20160490
  20995001, 21131037, 24509984, 19380733, 20985167, 18943696, 20552947
  19591984, 18135723, 18461246, 20014326, 19524857, 25250824, 22999793
  18890943, 19154753, 25055154, 24808260, 19499021, 25206628, 20923930
  19046190, 25074091, 20028666, 23170674, 24451580, 19051385, 19682695
  19050688, 19831611, 22891868, 21607972, 26755613, 18922918, 21087799
  25956724, 18956780, 19683886, 23058136, 23757020, 19026993, 20110156
  21269977, 18261648, 18362971, 19558324, 22651583, 20185476, 18882642
  19478119, 17447588, 19414274, 19262534, 20752167, 19147513, 22475834
  18893463, 21242118, 19514085, 25496699, 20758997, 27738131, 27006180
 [oracle@ol7-121-rac2 ~]$

Remove Grid 19c directory for Node1:

[root@ol7-121-rac1 ~]# ls -ld /u01/app/19.3.0.0
 drwxr-xr-x. 3 root oinstall 18 Dec 6 17:30 /u01/app/19.3.0.0
 [root@ol7-121-rac1 ~]# rm -rf /u01/app/19.3.0.0
 [root@ol7-121-rac1 ~]# ls -ld /u01/app/19.3.0.0
 ls: cannot access /u01/app/19.3.0.0: No such file or directory
 [root@ol7-121-rac1 ~]# mkdir -p /u01/app/19.3.0.0
 [root@ol7-121-rac1 ~]# chmod -R 775 /u01/app/19.3.0.0
 [root@ol7-121-rac1 ~]# chown -R oracle:oinstall /u01/app/19.3.0.0
 [root@ol7-121-rac1 ~]# ls -ld /u01/app/19.3.0.0
 drwxrwxr-x. 2 oracle oinstall 6 Dec 10 00:26 /u01/app/19.3.0.0
 [root@ol7-121-rac1 ~]#

Remove Grid 19c directory for Node2:

[root@ol7-121-rac2 ~]# ls -ld /u01/app/19.3.0.0
 drwxr-xr-x. 3 root oinstall 18 Dec 9 15:47 /u01/app/19.3.0.0
 [root@ol7-121-rac2 ~]# rm -rf /u01/app/19.3.0.0
 [root@ol7-121-rac2 ~]# ls -ld /u01/app/19.3.0.0
 ls: cannot access /u01/app/19.3.0.0: No such file or directory
 [root@ol7-121-rac2 ~]# mkdir -p /u01/app/19.3.0.0
 [root@ol7-121-rac2 ~]# chmod -R 775 /u01/app/19.3.0.0
 [root@ol7-121-rac2 ~]# chown -R oracle:oinstall /u01/app/19.3.0.0
 [root@ol7-121-rac2 ~]# ls -ld /u01/app/19.3.0.0
 drwxrwxr-x. 2 oracle oinstall 6 Dec 10 00:29 /u01/app/19.3.0.0
 [root@ol7-121-rac2 ~]#

Hopefully the steps provided will be useful in the event that a downgrade is necessary.

Oracle Database Consulting Services

Ready to optimize your Oracle Database for the future?

 

 
View full post