Using RDA 4.11 in a RAC Environment
RDA 4.11 is out, with a couple of new features. “Oracle Database Diagnostics Collector” (ORADDC) is one of those. It allows you to easily activate all kinds of traces, dumps, or stack collections.
Once RDA 4.11 installed, run ./rda.pl -T oraddc to collect whatever you need. The tool will guide you. To learn more about its features, you can display its man page: run ./rda.pl -M oraddc or ./rda.pl -M.
This may become one of the most used RDA modules for Oracle Support Services and Oracle database administrators stuck in different situations. For now, let’s start with a more basic question: “How to leverage RDA in a RAC environment ?”
About RDA
RDA (Remote Diagnostic Agent) is one of the primary tools used by Oracle Support Services to diagnose any problem related to the Oracle stacks (from the Virtual Machine to the CRM). You can leverage this tool for your own needs and to speed up your problem resolution.
Using RDA before you install the Clusterware
Installing the Oracle Clusterware is an error-prone process, and in most cases, the errors come from unmet prerequisites, Fortunately, the CVU (Cluster Verification Utility) helps diagnose any problem in the initial setup you’ve performed. The best way to use it is probably to download the latest version and to validate the prerequisites according to the release you want to install. Once you’ve downloaded and unzipped the latest release from OTN, you can for example check if the prerequisites are met before you install Clusterware, with a command like the one below:
$ su - oracle
$ mkdir cvu
$ cd cvu
$ # Change the URL accordingly to the CVU latest
$ # release and your platform
$ wget http://download.oracle.com/otndocs/products/clustering/cvu/cvu_linux_64_112007.zip
$ unzip cvu_*.zip
$ cd bin
$ ./cluvfy stage -pre crsinst \
-n node1,node2,node3,node4 \
-r 10gR2 -verbose
Using the latest CVU will reduce the number of errors, and the checks are more accurate with the latest releases. However, you can also use RDA on a per-node basis to check the Oracle prerequisites. In order to do that, use the HCVE (Health Check/Validation Engine) tool. Below is an example of this use of RDA HCVE:
$ su - oracle $ mkdir rda-4.11 $ cd rda-4.11 $ # Change the distribution accordingly to the latest $ # RDA release and platform. Use your Metalink Credentials $ ftp updates.oracle.com ftp> cd 6854532 ftp> ls ftp> get p6854532_4110_Linux-x86.zip ftp> bye $ unzip 6854532*.zip $ cd rda $ ./rda.pl -T hcve Processing HCVE tests … Available Pre-Installation Rule Sets: 1. Oracle Database 10g R1 (10.1.0) PreInstall (Linux-x86) 2. Oracle Database 10g R1 (10.1.0) PreInstall (Linux AMD64) 3. Oracle Database 10g R1 (10.1.0) PreInstall (IA-64 Linux) 4. Oracle Database 10g R2 (10.2.0) PreInstall (Linux AMD64) 5. Oracle Database 10g R2 (10.2.0) PreInstall (IA-64 Linux) 6. Oracle Database 10g R2 (10.2.0) PreInstall (Linux-x86) 7. Oracle Database 11g R1 (11.1.0) PreInstall (Linux AMD64) 8. Oracle Database 11g R1 (11.1.0) PreInstall (Linux-x86) 9. Oracle Application Server 10g (9.0.4) PreInstall (Linux) 10. Oracle Application Server 10g R2 (10.1.2) PreInstall (Linux) 11. Oracle Application Server 10g R3 (10.1.3) PreInstall (Linux AMD64) 12. Oracle Application Server 10g R3 (10.1.3) PreInstall (IA-64 Linux) 13. Oracle Application Server 10g R3 (10.1.3) PreInstall (Linux-x86) 14. Oracle Portal PreInstall (Generic) Available Post-Installation Rule Sets: 15. Oracle Portal PostInstall (generic) 16. RAC 10G DB and OS Best Practices (Linux) 17. Data Guard PostInstall (Generic) Enter the HCVE rule set number Hit ‘Return’ to accept the default (1) > 4 Enter value for < Planned ORACLE_HOME location or if set > Hit ‘Return’ to accept the default ($ORACLE_HOME) > Test “Oracle Database 10g R2 (10.2.0) PreInstall (Linux AMD64)” executed at Sat Mar 15 15:52:17 2008 Test Results ~~~~~~~~~~~~ ID NAME RESULT VALUE ===== ==================== ====== ======================================== 10 OS Certified? FAILED Not certified [Red Hat version] 20 User in /etc/passwd? PASSED userOK 30 Group in /etc/group? PASSED GroupOK 40 Input ORACLE_HOME RECORD $ORACLE_HOME 50 ORACLE_HOME Valid? PASSED OHexists 60 O_H Permissions OK? FAILED NoWritePerm 70 Umask Set to 022? PASSED UmaskOK 80 LDLIBRARYPATH Unset? FAILED IsSet 90 JAVA_HOME Unset? FAILED IsSet 100 Other O_Hs in PATH? FAILED OratabEntryInPath 110 oraInventory Permiss PASSED oraInventoryOK 120 /tmp Adequate? PASSED TempSpaceOK 130 Swap (in MB) PASSED 3906 140 RAM (in MB) FAILED 1003 150 Swap OK? PASSED SwapToRAMOK 160 Disk Space OK? PASSED DiskSpaceOK 170 Kernel Parameters OK PASSED KernelOK 180 Got ld,nm,ar,make? PASSED ld_nm_ar_make_found 190 ulimits OK? FAILED TimeNotDefined StackTooSmall NoFilesN..> 200 EL4 RPMs OK? PASSED NotEnterprise 204 RHEL3 RPMs OK? PASSED NotRHEL3 205 RHEL4 RPMs OK? PASSED NotRHEL4 209 SUSE SLES9 RPMs OK? PASSED NotSuSE 210 SUSE SLES10 RPMs OK? PASSED NotSuSE 214 ip_local_port_range PASSED ip_local_port_rangeOK 220 Tainted Kernel? PASSED NotVerifiable 230 Other OUI Up? PASSED NoOtherOUI Result file: /home/arkzoyd/Customers/TPG/rda/output/RDA_HCVE_A201DB10R2_lna_res.htm
Run an RDA collection once the Clusterware installed
Once the Clusterware correctly is installed and configured, RDA can easily be installed in the cluster (Check Metalink Note note 359395.1 for more details). It enables you to run a RDA collection cluster-wide. To proceed with such an installation, just run the command below, once the cluster is configured:
$ ./rda.pl -vX Remote setup_cluster-----------------------------------------------------Requesting common information-----------------------------------------------------Where RDA should be installed on the remote nodes? Hit ‘Return’ to accept the default (/home/oracle/rda-4.11/rda) > Where setup files and reports should be stored on the remote nodes? Hit ‘Return’ to accept the default (/home/oracle/rda-4.11/rda) > Should an alternate login be used to execute remote requests (Y/N)? Hit ‘Return’ to accept the default (N) > Enter an Oracle User ID (userid only) to view DBA_ and V$ tables. If RDA will be run under the Oracle software owner’s ID, enter a ‘/’ here, and select Y at the SYSDBA prompt to avoid being prompted for the database password at runtime. Hit ‘Return’ to accept the default (system) > / Is ‘/’ a sysdba user (will connect as sysdba) (Y/N)? Hit ‘Return’ to accept the default (N) > Y-----------------------------------------------------Requesting information for node node1-----------------------------------------------------Enter the Oracle Home to be analyzed on the node node1 Hit ‘Return’ to accept the default (/u01/app/oracle/product/10.2.0/db_1) > Enter the Oracle SID to be analyzed on the node node1 Hit ‘Return’ to accept the default (racdb1) > racdb1-----------------------------------------------------Requesting information for node node2-----------------------------------------------------Enter the Oracle Home to be analyzed on the node node2 Hit ‘Return’ to accept the default (/u01/app/oracle/product/10.2.0/db_1) > Enter the Oracle SID to be analyzed on the node node2 Hit ‘Return’ to accept the default (racdb2) > racdb2-----------------------------------------------------RAC Setup Summary-----------------------------------------------------Nodes: . NOD001 node1/racdb1 . NOD002 node2/racdb2 2 nodes found-----------------------------------------------------S909RDSP: Produces the Remote Data Collection Reports-----------------------------------------------------Updating the setup file …
Once RDA is installed, you can run a cluster-wide collection with the command below:
$ ./rda.pl -v -e REMOTE_TRACE=1
The two options are used to display the progress of the various operations executed by RDA. Once the collection is done, you can get the .zip file to review the results. It contains a directory called remote, with the RDA collection for each one of the nodes. Here is an example of the output:
Use RDA to compare cluster nodes or node evolutions
RDA contains another useful tool that you can leverage with a cluster (see previous section to install RDA in a cluster). This tool is named diff. It allows you to collect Oracle-related system information for one or all the nodes of a cluster. When RDA is installed in the cluster, you can compare two nodes or one node at two different points in time. In order to use this tool, just run a collection. Here is an example of how to run a collection for a set named myset and for the sample 1 of it:
$ ./rda.pl -T diff:A:myset,1
You can also run it in interactive mode with the command below:
$ ./rda.pl -T diff
To generate the output, you can run ./rda.pl -T diff in interactive mode. The -C option will guide you through the process of generating the report you need.
Use RDA to check the RAC database settings
Once the database software is installed and the database is created, you can check the RAC settings with the RDA HCVE tool. In order to proceed, you need to make sure you have the SYSTEM password and you’ve set the ORACLE_HOME, PATH, and ORACLE_SID environment variables.
$ su - oracle $ cd rda-4.11 $ . oraenv ORACLE_SID = [RACDB] ? $ export ORACLE_SID=RACDB1 $ ./rda.pl -T hcve Processing HCVE tests … Available Pre-Installation Rule Sets: 1. Oracle Database 10g R1 (10.1.0) PreInstall (Linux-x86) 2. Oracle Database 10g R1 (10.1.0) PreInstall (Linux AMD64) 3. Oracle Database 10g R1 (10.1.0) PreInstall (IA-64 Linux) 4. Oracle Database 10g R2 (10.2.0) PreInstall (Linux AMD64) 5. Oracle Database 10g R2 (10.2.0) PreInstall (IA-64 Linux) 6. Oracle Database 10g R2 (10.2.0) PreInstall (Linux-x86) 7. Oracle Database 11g R1 (11.1.0) PreInstall (Linux AMD64) 8. Oracle Database 11g R1 (11.1.0) PreInstall (Linux-x86) 9. Oracle Application Server 10g (9.0.4) PreInstall (Linux) 10. Oracle Application Server 10g R2 (10.1.2) PreInstall (Linux) 11. Oracle Application Server 10g R3 (10.1.3) PreInstall (Linux AMD64) 12. Oracle Application Server 10g R3 (10.1.3) PreInstall (IA-64 Linux) 13. Oracle Application Server 10g R3 (10.1.3) PreInstall (Linux-x86) 14. Oracle Portal PreInstall (Generic) Available Post-Installation Rule Sets: 15. Oracle Portal PostInstall (generic) 16. RAC 10G DB and OS Best Practices (Linux) 17. Data Guard PostInstall (Generic) Enter the HCVE rule set number Hit ‘Return’ to accept the default (1) > 16 Enter the password for ‘SYSTEM’: Please re-enter it to confirm: Test “RAC 10G DB and OS Best Practices (Linux)” executed at Sat Mar 15 16:08:08 2008 Test Results ~~~~~~~~~~~~ ID NAME RESULT VALUE ===== ==================== ====== ======================================== 10 ORA_CRS_HOME RECORD /u01/app/oracle/product/10.2.0/crs 100 Database Name RECORD RACDB 102 Database Version RECORD 10.2.0.3.0 104 Interconnect Network RECORD 106 DB Block Size RECORD 8192 108 DB File Multiblock R RECORD 16 120 Max Commit Propagati PASSED 0 130 SYS.AUDSES$ Cache Si PASSED 10000 132 SYS.IDGEN1$ Cache Si FAILED 20 140 Parallel Execution M FAILED 2152 150 Min Parallel Servers RECORD 3 152 Min Parallel Servers FAILED 0 200 $ORA_CRS_HOME Define PASSED Found 210 Remote Access PASSED All loaded 220 _USR_ORA_DEBUG / CRS FAILED node1:? node2:? 230 _USR_ORA_DEBUG / ORA FAILED node1:? node2:? 240 rmem_max PASSED OK 250 UDP Buffer Size PASSED OK 260 wmem_max PASSED OK 270 rmem_default PASSED OK 280 wmem_default PASSED OK 290 Sysrq Magic Keys PASSED OK 300 Oracle Executable Li PASSED linked 310 hangcheck-timer FAILED node1:Unknown node2:Unknown 320 aio-max-size Setting FAILED node1:Unknown node2:Unknown 330 Memory (32-bit) PASSED OK 340 Swap (32-bit) PASSED OK 350 Swap (64-bit) FAILED [node1:]Swap<2RAM [node2:..> 360 Patch List PASSED Complete Result file: /home/oracle/rda-4.11/rda/output/RDA_HCVE_P400RAC_lnx_res.htm
RDA can probably do much more for you, but this is another story. If you want to learn more about Metalink tools and practices, Chris Warticki’s Blog - Oracle Support is very worthwhile.
