A simple command that saved me
So there I was, reviewing the environment to create an action plan for patching. To my surprise, I noticed the following when checking DBFS: [code] Oracle Instance not alive for sid "DBFS2" [/code] I will demonstrate the simple command I used to ensure that all services that are supposed to be running are running. What happened? It turns out there was a network outage and when starting resources, DBFS was missing. It's much harder to know what are all the resources that need to be started for a complex environment with RAC, GoldenGate, DataGuard, ACFS, DBFS. crsctl stat res -t -w '((TARGET != ONLINE) or (STATE != ONLINE)' [code] -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.xxxxxxxx.yy_users_dr.svc 1 OFFLINE OFFLINE STABLE 2 OFFLINE OFFLINE STABLE -------------------------------------------------------------------------------- [/code] From the command above, Target=State=OFFLINE which is good. crsctl stat res -t -w '((TARGET != ONLINE) or (STATE != ONLINE)' [code] [code] crsctl stat res -t -w '((TARGET != ONLINE) or (STATE != ONLINE)' -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- dbfs_mount OFFLINE ONLINE hostxxxx01 STABLE OFFLINE OFFLINE hostxxxx02 STABLE -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.dbfs.db 2 OFFLINE OFFLINE Instance Shutdown,STABLE ora.xxxxxxxx.yy_users_dr.svc 1 OFFLINE OFFLINE STABLE 2 OFFLINE OFFLINE STABLE xag.ggue_src.goldengate 1 OFFLINE ONLINE hostxxxx01 STABLE -------------------------------------------------------------------------------- [/code]
The above is not good because State=ONLINE but Target=OFFLINE for dbfs_mount at hostxxxx02. It seems Oracle is seldom intuitive as checking the condition (TARGET != STATE) does not return any results. [code][/code] crsctl stat res -t -w '((TARGET != STATE)' [code][/code] In conclusion, always check TARGET=STATE for CRS to avoid having services down that are supposed to be running. Alternatively, I suppose just restarting CRS versus starting each service individually might be better, provided resources are integrated with CRS.
