During RAC online patching, GoldenGate resource is required to be relocated to surviving node. When relocation fails, there are a few logs which can be useful for troubleshooting. Oracle Grid Infrastructure Standalone Agents for Oracle Clusterware (XAG) was not installed and using existing version from $GRID_HOME. Unfortunately, I don't have the exact version and would suspect the version is probably same as:
[oracle@racnode-dc1-1 ~]$ $GRID_HOME/bin/agctl query releaseversion
The Oracle Grid Infrastructure Agents release version is 3.1.0
[oracle@racnode-dc1-1 ~]$ $GRID_HOME/bin/srvctl -V
srvctl version: 12.1.0.2.0
[oracle@racnode-dc1-1 ~]$
The name of the GoldenGate instance/resource (instance_name): gg_xx GoldenGate resources can be determined using the following command:
$GRID_HOME/bin/crsctl stat res -t|grep -A2 xag
xag.gg_xx-vip.vip
1 ONLINE ONLINE racnode-dc1-2 STABLE
xag.gg_xx.goldengate
1 ONLINE ONLINE racnode-dc1-2 STABLE
Here is the output for GoldenGate relocation and can be performed from any node.
$GRID_HOME/bin/agctl relocate goldengate gg_xx --node racnode-dc1-1
CRS-2673: Attempting to stop 'xag.gg_xx.goldengate' on 'racnode-dc1-2'
CRS-2677: Stop of 'xag.gg_xx.goldengate' on 'racnode-dc1-2' succeeded
CRS-2673: Attempting to stop 'xag.gg_xx-vip.vip' on 'racnode-dc1-2'
CRS-2677: Stop of 'xag.gg_xx-vip.vip' on 'racnode-dc1-2' succeeded
CRS-2672: Attempting to start 'xag.gg_xx-vip.vip' on 'racnode-dc1-1'
CRS-2676: Start of 'xag.gg_xx-vip.vip' on 'racnode-dc1-1' succeeded
CRS-2672: Attempting to start 'xag.gg_xx.goldengate' on 'racnode-dc1-1'
CRS-2674: Start of 'xag.gg_xx.goldengate' on 'racnode-dc1-1' failed
CRS-2679: Attempting to clean 'xag.gg_xx.goldengate' on 'racnode-dc1-1'
CRS-2681: Clean of 'xag.gg_xx.goldengate' on 'racnode-dc1-1' succeeded
CRS-2564: Failed to relocate resource 'xag.gg_xx.goldengate'. Will attempt to restore it on 'racnode-dc1-2' now.
CRS-2673: Attempting to stop 'xag.gg_xx-vip.vip' on 'racnode-dc1-1'
CRS-2677: Stop of 'xag.gg_xx-vip.vip' on 'racnode-dc1-1' succeeded
CRS-2672: Attempting to start 'xag.gg_xx-vip.vip' on 'racnode-dc1-2'
CRS-2676: Start of 'xag.gg_xx-vip.vip' on 'racnode-dc1-2' succeeded
CRS-2672: Attempting to start 'xag.gg_xx.goldengate' on 'racnode-dc1-2'
CRS-2676: Start of 'xag.gg_xx.goldengate' on 'racnode-dc1-2' succeeded
CRS-4000: Command Relocate failed, or completed with errors.
$GRID_HOME/bin/agctl status goldengate gg_xx
Goldengate instance 'gg_xx' is running on racnode-dc1-2
Here are the logs to check in order of personal preference. Check GoldenGate Error Log: $GG_HOME/ggserr.log Check XAG Agent trace file: GoldenGate user is ggsuser $ORACLE_BASE/diag/crs/`hostname -s`/crs/trace/crsd_scriptagent_ggsuser.trc Check alert.log for crs: $ORACLE_BASE/diag/crs/`hostname -s`/crs/trace/alert.log In summary, there are multiple logs to check and when checking the right one first will make troubleshooting more efficient.
No Comments Yet
Let us know what you think