Changing Hostnames in Oracle RAC

Jun 11, 2007 / By Alex Gorbachev

Tags: ,

Update: this procedure is for Linux and should work on any UNIX OS. Ron supplied how he did this on Windows in the comment below. Thanks Ron.

Sometimes there is a desperate need to change hostnames for one or all nodes of an Oracle RAC cluster. However, this operation is not officially supported. From Metalink Note 220970.1 RAC Frequently Asked Questions:

Can I change the public hostname in my Oracle Database 10g Cluster using Oracle Clusterware?

Hostname changes are not supported in Oracle Clusterware (CRS), unless you want to perform a deletenode followed by a new addnode operation.
The hostname is used to store among other things the flag files and CRS stack will not start if hostname is changed.

One way to do it is to remove a node from a cluster, change its hostname, and then add it back to the cluster as a new node. You will need to make sure that ORACLE_HOME is also added to this node as well as the database instance configuration.

If you are brave enough, there is another way to do this. It’s not described anywhere on the Metalink, but there are no major hacks needed to implement it. The idea is to simply re-run the configuration of CRS (including re-formatting OCR and voting disks) and re-create the CRS resources after that. Obviously, this is not an online operation and the whole cluster is down for the duration of rename.

I assume that we have an Oracle RAC cluster running, with database(s) running already, optionally including ASM instances.

Capture Resource Definitions

Before doing anything, we should capture resource definitions from the current CRS resources. This is an optional step, but it will simplify configuration later.

A single resource definition can be captured with a command $ORA_CRS_HOME/bincrs_stat -p . Here is a small shell script to capture that for every resource and save it into a .cap file. As you will see later these files can be used to easily recreate resources:

for res in `$ORA_CRS_HOME/bin/crs_stat -p | grep "^NAME=" | cut -d = -f 2` ; do
    $ORA_CRS_HOME/bin/crs_stat -p $res >/opt/oracle/resources/$res.cap
done

Here is an example list of files created:

# ls -l /opt/oracle/resources
total 52
-rw-r--r--  1 root root 813 Feb 27 12:06 ora.A.A1.inst.cap
-rw-r--r--  1 root root 813 Feb 27 12:06 ora.A.A2.inst.cap
-rw-r--r--  1 root root 789 Feb 27 12:06 ora.A.db.cap
-rw-r--r--  1 root root 809 Feb 27 12:06 ora.vs10a.ASM1.asm.cap
-rw-r--r--  1 root root 799 Feb 27 12:06 ora.vs10a.gsd.cap
-rw-r--r--  1 root root 832 Feb 27 12:06 ora.vs10a.LISTENER_VS10A.lsnr.cap
-rw-r--r--  1 root root 804 Feb 27 12:06 ora.vs10a.ons.cap
-rw-r--r--  1 root root 828 Feb 27 12:06 ora.vs10a.vip.cap
-rw-r--r--  1 root root 809 Feb 27 12:06 ora.vs11a.ASM2.asm.cap
-rw-r--r--  1 root root 799 Feb 27 12:06 ora.vs11a.gsd.cap
-rw-r--r--  1 root root 832 Feb 27 12:06 ora.vs11a.LISTENER_VS11A.lsnr.cap
-rw-r--r--  1 root root 804 Feb 27 12:06 ora.vs11a.ons.cap
-rw-r--r--  1 root root 828 Feb 27 12:06 ora.vs11a.vip.cap

Sample content from one file:

# cat /opt/oracle/resources/ora.A.db.cap
NAME=ora.A.db
TYPE=application
ACTION_SCRIPT=/opt/oracle/A/product/10.2.0/crs/bin/racgwrap
ACTIVE_PLACEMENT=0
AUTO_START=1
CHECK_INTERVAL=600
DESCRIPTION=CRS application for the Database
FAILOVER_DELAY=0
FAILURE_INTERVAL=60
FAILURE_THRESHOLD=1
HOSTING_MEMBERS=
OPTIONAL_RESOURCES=
PLACEMENT=balanced
REQUIRED_RESOURCES=
RESTART_ATTEMPTS=1
SCRIPT_TIMEOUT=600
START_TIMEOUT=0
STOP_TIMEOUT=0
UPTIME_THRESHOLD=7d
USR_ORA_ALERT_NAME=
USR_ORA_CHECK_TIMEOUT=0
USR_ORA_CONNECT_STR=/ as sysdba
USR_ORA_DEBUG=0
USR_ORA_DISCONNECT=false
USR_ORA_FLAGS=
USR_ORA_IF=
USR_ORA_INST_NOT_SHUTDOWN=
USR_ORA_LANG=
USR_ORA_NETMASK=
USR_ORA_OPEN_MODE=
USR_ORA_OPI=false
USR_ORA_PFILE=
USR_ORA_PRECONNECT=none
USR_ORA_SRV=
USR_ORA_START_TIMEOUT=0
USR_ORA_STOP_MODE=immediate
USR_ORA_STOP_TIMEOUT=0
USR_ORA_VIP=

Stop Clusterware

Now we can stop Oracle Clusterware on all nodes using $ORA_CRS_HOME/bin/crsctl stop crs, and then change hostnames. Note that this will stop all databases, listeners, and other resources registered within CRS, so this is the time when outage starts.

Rename Hosts

I won’t discuss changing the hostname itself it here — it’s a straight-forward SA task.

A few things to pay special attention to:

  1. Make sure that aliases in /etc/hosts are amended.
  2. Don’t forget to change aliases for VIPs and private IPs. This is not strictly required but you are better off following the standard naming convention (-priv and -vip for interconnect and virtual IP respectively) unless you have really good reason not to. Note that at this stage you should be also able to change IP addresses as well. I didn’t try it, but it should work.
  3. Also make sure DNS configuration is also changed by your SA, if your applications use DNS to resolve hostnames.

Modify $ORA_CRS_HOME/install/rootconfig

$ORA_CRS_HOME/install/rootconfig is called as part of the root.sh script run after Oracle Clusterware installation. We have to modify it so that it uses different host names.

Generally, you would simply change every appearance of the old hostnames to the new hostnames. If you want to do that in vi, use :%s/old_node/new_node/g. Be careful not to change existing non-relevant parts of the script matching your old hostname. The variables that should be changed are

  • CRS_HOST_NAME_LIST
  • CRS_NODE_NAME_LIST
  • CRS_PRIVATE_NAME_LIST
  • CRS_NODELIST
  • CRS_NODEVIPS

The latter might need modification if you also change IPs.

At this stage, you can also change your OCR and voting disks locations. The following lines should be changed:

CRS_OCR_LOCATIONS={OCR path},{OCR mirror path}
CRS_VOTING_DISKS={voting disk1 path},{voting disk2 path},{voting disk3 path}

You can also change your cluster name via the variable CRS_CLUSTER_NAME.

Cleanup OCR and Voting Disks

You should clear OCR and voting disks, otherwise, the script will refuse to format them (well, it will unless you use a special force option, but that’s more hassle). This can be done using dd. In the example below I have mirrored OCR and 3 voting disks:

dd if=/dev/zero of= bs=1024k
dd if=/dev/zero of={OCR2 path} bs=1024k
dd if=/dev/zero of={voting1 path} bs=1024k
dd if=/dev/zero of={voting2 path} bs=1024k
dd if=/dev/zero of={voting3 path} bs=1024k

“Break” Clusterware Configuration

rootconfig has some protection from idiots (as we say in Russia) built in — it checks if Clusterware has been already configured and, if it has, it exits without doing any harm. One way to “break” the configuration and make this script run for a second time is to delete the file /etc/oracle/ocr.loc. (Note that this is a Linux-specific location; other Unix variants might have different path. On HP-UX, for example, it’s something like /var/opt/oracle/ocr.log if I recall correctly.)

Run $ORA_CRS_HOME/install/rootconfig

If everything has gone alright, you should be able to run $ORA_CRS_HOME/install/rootconfig as the root user without any issues. If there are problems, follow the standard CRS troubleshooting procedure — checking /var/log/messages and $ORA_CRS_HOME/log/{nodename} et cetera.

Note that this should be done on every node one by one — sequentially. On the last node of the cluster, the script will try to configure the VIPs, and there is a known bug here if you use a private range IP for VIP. This can be easily fixed by running $ORA_CRS_HOME/bin/vipca manually in graphical mode (i.e. you will need $DISPLAY configured correctly).

Verify Clusterware Configuration and Status

This is a simple check to make sure that all nodes are up and have VIP components configured correctly:

[root@vs10 bin]# $ORA_CRS_HOME/bin/crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.vs10.gsd   application    ONLINE    ONLINE    vs10
ora.vs10.ons   application    ONLINE    ONLINE    vs10
ora.vs10.vip   application    ONLINE    ONLINE    vs10
ora.vs11.gsd   application    ONLINE    ONLINE    vs11
ora.vs11.ons   application    ONLINE    ONLINE    vs11
ora.vs11.vip   application    ONLINE    ONLINE    vs11

Adding Listener Resources to CRS

There are two ways to do this — you can either use netca to configure the listener from scratch (you might need to clean it up from listener.ora first), or you can change the configuration manually and register it with CRS from command line. I’ll show how to that manually — obviously, the preferred way when it comes to the real environments. ;-)

First of all, we will need to change the $ORACLE_HOME/network/admin/listener.ora file, and you will probably want to change tnsnames.ora at the same time. You need to replace old node aliases with new ones, and change the IPs if they are used instead of aliases, and if you changed them above during clusterware reconfiguration.

Note that depending on how your LOCAL_LISTENER and REMOTE_LISTENER init.ora parameters are set, you might need to change them: if they reference connections descriptors from tnsname.ora, then only the latter should be changed, but if there are full connection descriptors, they should also be modified).

You should also change listener names to reflect new hostnames. Usually, listeners are named as LISTENER_{hostname}, and you should keep this convention again unless you have a very good reason not to. Do that on both nodes if you don’t have a shared ORACLE_HOME.

Now it’s time to get back to the .cap files with the CRS resource definitions we captured when we began. The files we are interested in are in format ora.{hostname}.LISTENER_{HOSTNAME}.lsnr.cap. In my case, one of them is ora.vs10a.LISTENER_VS10A.lsnr (my old hostname was vs10a). If you changed listener names above, you would need to amend it there as well — NAME=ora.vs10.LISTENER_VS10.lsnr, and rename the file according to the new host name following the same naming convention.

Your VIP name has probably changed, so this line should be modified as well: REQUIRED_RESOURCES=ora.vs10.vip. And finally, the hosting member will change: HOSTING_MEMBERS=vs10. Check the whole file carefully — you should simply modify the old hostname to the new one in both lower and upper case.

Now it’s time to register the resource — the crs_register command does just that. This command specifies the resource name to register and the directory where the .cap file is located. It should be named exactly like resource name plus a “.cap” extension. Each node’s listener can be added from the same node. It’s important that the content of the .cap file is modified appropriately. Assuming I have files ora.vs10.LISTENER_VS10.lsnr and ora.vs11.LISTENER_VS11.lsnr in directory /opt/oracle/A/resources, I run:

$ORA_CRS_HOME/bin/crs_register ora.vs10.LISTENER_VS10.lsnr -dir /opt/oracle/A/resources
$ORA_CRS_HOME/bin/crs_register ora.vs11.LISTENER_VS11.lsnr -dir /opt/oracle/A/resources

Now the output from crs_stat -t should be:

Name           Type           Target    State     Host
------------------------------------------------------------
ora....10.lsnr application    OFFLINE   OFFLINE
ora.vs10.gsd   application    ONLINE    ONLINE    vs10
ora.vs10.ons   application    ONLINE    ONLINE    vs10
ora.vs10.vip   application    ONLINE    ONLINE    vs10
ora....11.lsnr application    OFFLINE   OFFLINE
ora.vs11.gsd   application    ONLINE    ONLINE    vs11
ora.vs11.ons   application    ONLINE    ONLINE    vs11
ora.vs11.vip   application    ONLINE    ONLINE    vs11

It’s now time to start the listeners:

$ORA_CRS_HOME/bin/srvctl start nodeapps -n vs10
$ORA_CRS_HOME/bin/srvctl start nodeapps -n vs11

crs_stat -t should show the listeners online:

Name           Type           Target    State     Host
------------------------------------------------------------
ora....10.lsnr application    ONLINE    ONLINE    vs10
ora.vs10.gsd   application    ONLINE    ONLINE    vs10
ora.vs10.ons   application    ONLINE    ONLINE    vs10
ora.vs10.vip   application    ONLINE    ONLINE    vs10
ora....11.lsnr application    ONLINE    ONLINE    vs11
ora.vs11.gsd   application    ONLINE    ONLINE    vs11
ora.vs11.ons   application    ONLINE    ONLINE    vs11
ora.vs11.vip   application    ONLINE    ONLINE    vs11

Adding ASM Instances to CRS

This step is optional, and it you don’t use ASM, skip it.

Unfortunately, we can’t simply use .cap files to register ASM resources. There are more pieces required and the only way I could find to register ASM instances is to use srvctl which is, actually, a more supported option. This is simple:

$ORACLE_HOME/bin/srvctl add asm -n vs10 -i ASM1 -o $ORACLE_HOME
$ORACLE_HOME/bin/srvctl add asm -n vs11 -i ASM1 -o $ORACLE_HOME
$ORACLE_HOME/bin/srvctl start asm -n vs10
$ORACLE_HOME/bin/srvctl start asm -n vs11

There is a catch — sometimes I had to prefix the name of the ASM instance with a “+” (i.e. making it like -i +ASM1) and sometimes no plus-sign was required. I couldn’t figure out why, so if you have a clue, let me know.

crs_stat -t should show now:

Name           Type           Target    State     Host
------------------------------------------------------------
ora....SM1.asm application    ONLINE    ONLINE    vs10
ora....10.lsnr application    ONLINE    ONLINE    vs10
ora.vs10.gsd   application    ONLINE    ONLINE    vs10
ora.vs10.ons   application    ONLINE    ONLINE    vs10
ora.vs10.vip   application    ONLINE    ONLINE    vs10
ora....SM2.asm application    ONLINE    ONLINE    vs11
ora....11.lsnr application    ONLINE    ONLINE    vs11
ora.vs11.gsd   application    ONLINE    ONLINE    vs11
ora.vs11.ons   application    ONLINE    ONLINE    vs11
ora.vs11.vip   application    ONLINE    ONLINE    vs11

Register Databases

For each database, you need to register a database resource. Then, for every instance, you need to register an instance resource. So for database A, my two-node cluster, I use:

$ORACLE_HOME/bin/srvctl add database -d A -o $ORACLE_HOME
$ORACLE_HOME/bin/srvctl add instance -d A -i A1 -n vs10
$ORACLE_HOME/bin/srvctl add instance -d A -i A2 -n vs11
$ORACLE_HOME/bin/srvctl start database -d A

Finally, crs_stat -t should show all resources online:

Name           Type           Target    State     Host
------------------------------------------------------------
ora.A.A1.inst  application    ONLINE    ONLINE    vs10
ora.A.A2.inst  application    ONLINE    ONLINE    vs11
ora.A.db       application    ONLINE    ONLINE    vs10
ora....SM1.asm application    ONLINE    ONLINE    vs10
ora....10.lsnr application    ONLINE    ONLINE    vs10
ora.vs10.gsd   application    ONLINE    ONLINE    vs10
ora.vs10.ons   application    ONLINE    ONLINE    vs10
ora.vs10.vip   application    ONLINE    ONLINE    vs10
ora....SM2.asm application    ONLINE    ONLINE    vs11
ora....11.lsnr application    ONLINE    ONLINE    vs11
ora.vs11.gsd   application    ONLINE    ONLINE    vs11
ora.vs11.ons   application    ONLINE    ONLINE    vs11
ora.vs11.vip   application    ONLINE    ONLINE    vs11

Other Resources

If you had other resources like services, user VIPs, or user-defined resources, you will probably be fine using the crs_register command to get them back into CRS. I didn’t try it, but it should work.

Final Check

To make sure that everything is working, you should at least reboot every node and see if everything comes up.

I don’t know if that operation is considered to be supported. The only slippery bit is modifying the $ORA_CRS_HOME/install/rootconfig file, because it’s usually created by the Universal Installer. Another tricky place is the “unusual” registration of listeners. Otherwise, all the commands are pretty much usual stuff, I think. Good luck!

29 Responses to “Changing Hostnames in Oracle RAC”

  • Doug Burns says:

    I recognise those hostnames ;-)

  • Shu says:

    I had to change a few other files in $CRS_HOME before it works for me, such as
    ./inventory/Components21/oracle.has.crs/10.2.0.1.0/context.xml
    ./inventory/Components21/oracle.crs/10.2.0.1.0/installIP1Log.xml
    ./inventory/Components21/oracle.crs/10.2.0.1.0/context.xml
    ./inventory/ContentsXML/oraclehomeproperties.xml
    ./install/paramfile.crs
    But overall, this is a great recipe, played magics for me.
    Thanks a lot!

  • Glad to hear it worked for you, Shu. Thanks for feedback!
    Indeed, inventory needs to be changed as well. I don’t see you mentioned file {inventory_dir}/ContentsXML/inventory.xml – it has cluster nodes in it.

  • P Kumar says:

    Alex,
    I have tried your method and am running into a problem with the ASM on one node. I get a “PRKS 1009: Failed to start asm interface” error and when I try to start or stop it, I get a “unrecoverable failure. Human intervention required” message. The other node is fine.

    I have tried it a few times from scratch with the same result, even to the point of doing it in reverse and going back to the old hostnames. Everything comes up fine with the old hostnames. With the new hostnames, I consistently get this ASM error on one node. Can you think of anything that I might be doing wrong?

    Thanks
    PKumar

  • PKumar,

    PRKS error doesn’t provide enough details to find out what’s wrong.

    First thing I would do is to try starting ASM instance manually – set your environment and use sqlplus. If that doesn’t work – you’d see the error. Maybe local_listener or remote_listener is set to an unresolvable value (I think just a hostname can be used with EZconnect) or something else host related.

    If that works fine, then you need to check the trace file for ora.{nodename}.asm resource. First, make sure you are using srvctl command from ASM home (if it’s different from database home). Than in that home, see the log file $ORACLE_HOME/log/{nodename}/racg/ora.{nodename}.ora.log. You should see the real reason behind the failure. If that doesn’t uncover enough, you can enable CRS tracing on that resource. The easy way to do that would be:
    crsctl debug log res ora.{nodename}.vip:1

    Disable tracing:
    crsctl debug log res ora.node1.vip:0

    Let me know how it goes. Good Luck!

  • P Kumar says:

    Sorry could get nothing in the logs. I didn’t see any error anywhere other than when I tried to start it up. I couldn’t spend too much time on it and went ahead and changed the ASM instance to 3 instead of 1, and that worked. Do you know why that would happen? Is the ASM info stored somewhere, perhaps tied to the hostname, that I needed to clear?

  • Mark J. Bobak says:

    Worked great, thanks Alex!

  • rush2me says:

    Can you tell me about – change in ip adress?
    i installed orcle 10 g RAC- without RAC option(single node) on hp unix 11i v3.
    had to change its ip adress.
    now all seems ok only NETCA is not working..
    am even able to create a database and connect to it…

  • rush2me:
    this article addresses RAC environment. It doesn’t apply to non-RAC installations and change of IP address in single node should be a matter of changing listener configuration unless you are using more advanced features.

  • HunterX says:

    Thank you for good stuff.
    I used this stuff to test with 11G RAC.

  • HunterX says:

    thank you again.
    I used idea on this article to change hostnames on RAC 10Gr2 (Sparc).

  • Glad it helped. Please don’t forget that this is a “creative” way of doing it and might render your environment unsupported. On the other hand, pretty much all operations are “legitimate”.

  • Ron says:

    Hi i have done the operation on a Oracle Window Cluster with the info you give us. They are some variant in the step. It work fine for us (change of all Ip address and subnet mask) it can be use for a hostname change like you have said.

    Thanks.

    Here we go (for a 2 nodes cluster 10.2.0.3)

    Stop all Oracle service on all nodes before the change of the address, the low format of the ocrcfg and vote disk

    change the address in the file host in C:\WINDOWS\system32\drivers\etc with the new address

    change the ip address in the window network definition connection with the new address

    ocopy \\.\ocrcfg d:\backup\ocrcfg.dmp
    ocopy \\.\votedsk1 d:\backup\votedsk1.dmp

    rem go to directory of the cluster binary crs\bin

    cd /D D:\Oracle\Ora10g\crs/bin

    rem Format raw ocrcfg and vote disk

    LogPartFormat.exe \\.\ocrcfg /q
    LogPartFormat.exe \\.\votedsk1 /q

    rem Run in prompt mode the cluster setup
    rem crssetup.exe config -nn %s_nodenumberlist% -pn %s_privatenamelist% -ch %ORA_CRS_HOME% -oh %s_OracleHome% -c %s_clustername% -l “%s_languageid%” -q %s_votedisklist% -g %s_usergroup%
    rem example for 2 node

    crssetup.exe config -nn host_name_node_1 ,1,host_name_node_2,2 -pn host_name_node_1-priv,1,host_name_node_2-priv,2 -ch D:\Oracle\Ora10g\crs -oh D:\Oracle\Ora10g\crs -c crs_lab -l “CANADIAN FRENCH_CANADA.WE8MSWIN1252″; -q “\\.\votedsk1″

    rem ensures that all nodes are configured with the same port for the ONS

    racgons.exe add_config host_name_node_1:6200 host_name_node_2:6200

    rem oifcfg to direct components to use specific network interfaces
    oifcfg.exe setif -global “Public”/10.200.5.0:public “Prive”/192.169.255.0:cluster_interconnect

    rem to rebuild the vip address vipca in wondow componant and make the incription in the cluster (in prompt mode or online)
    rem at the same time it will create the gsd and ons service

    vipca.bat -silent -nodelist “host_name_node_1,host_name_node_2″ -nodevips “host_name_node_1/host_name_node_1-vip/255.255.255.0/Public,host_name_node_2/host_name_node_2-vip/255.255.255.0/Public”;

    rem vipca
    rem check the state of the crs services (inscription)

    crs_stat -t

    rem now make the registration of the listener in the cluster.
    rem important don’t use the prompt command below, it will give you a crs-0223 error at the startup service
    Rem to solve this problem use netca on line

    rem crs_register ora.host_name_node_1.LISTENER_host_name_node_1.lsnr -dir D:\Oracle\Ora10g\crs\crs\profile
    rem crs_register ora.host_name_node_2.LISTENER_host_name_node_2.lsnr -dir D:\Oracle\Ora10g\crs\crs\profile
    rem srvctl start nodeapps -n host_name_node_1
    ren srvctl start nodeapps -n host_name_node_2

    netca

    rem check the state of the crs services (inscription)
    crs_stat -t

    rem start oracle object service and cluster volume before registering and accessing to asm services
    sc start ”Oracle Object Service” host_name_node_1
    sc start OracleClusterVolumeService host_name_node_1
    sc start ”Oracle Object Service” host_name_node_2
    sc start OracleClusterVolumeService host_name_node_2

    rem go the the rdbms directory to make the registration of asm and the database
    set ORACLE_HOME=D:\Oracle\Ora10g\rdbms

    rem add the service asm in the cluster in my case +asm1 and +asm2 for my 2 nodes
    srvctl add asm -n host_name_node_1 -i +ASM1 -o %ORACLE_HOME%
    srvctl add asm -n host_name_node_2 -i +ASM2 -o %ORACLE_HOME%

    rem start the asm service on each node
    srvctl start asm -n host_name_node_1
    srvctl start asm -n host_name_node_2

    rem check the state of the crs services (inscription)
    crs_stat -t

    rem go to the bin directory of rdbms directory to make the registration of the database
    cd /D D:\Logiciels\Oracle\Ora10g\rdbms/bin

    rem register the database (in my case labotary lab) and the to database service (lab1 and lab2) of each node
    srvctl add database -d lab -o %ORACLE_HOME%
    srvctl add instance -d lab -i lab1 -n host_name_node_1
    srvctl add instance -d lab -i lab2 -n host_name_node_2

    rem finally start the database
    srvctl start database -d lab

    rem go to directory of the cluster binary crs\bin
    rem all the services
    cd /D D:\Logiciels\Oracle\Ora10g\crs/bin

    rem you can go in the database with sqlplus and check all your components

  • Good job Ron.
    Thanks. I added the ref link to your commend in the main post.

  • Yibin Dong says:

    Ron,

    I tried in Windows x64 EE RAC RDBMS 11g1 and it worked, good job.

    I have a few modifications and comments:
    (1) Do not use OCOPY to backup OCR.
    Instead, using OCRCONFIG to avoid risk of corruption.
    For example,
    ocrconfig -export C:\oracle\product\11.1.0\admin\ocrbackup\ocr.exp
    Metalink note 779149.1 How to Export/Import an OCR with Ocrconfig

    (2) When running C:\oracle\product\11.1.0\crs\bin\crssetup.exe config…
    The following errors might happened:
    Step 5: Starting up CRS stack on all nodes
    node1 service OracleCSService in improper PENDING state, err(0)
    node2 service OracleCSService in improper PENDING state, err(997)

    This is expected.
    The workaround was to manually start Oracle Object Service then tried to restart OracleCSService.
    %CRS_HOME%\BIN>net start “Oracle Object Service”
    %CRS_HOME%\BIN>net start “OracleCSService”

    (3) Before configuring the listener, run cluvfy to confirm post CRS installation and this will fail because global inventory has not been updated.
    The following errors will occur
    ERROR:
    CRS is not installed on any of the nodes.
    Verification cannot proceed.
    The solution is to execute SETUP with updateNodeList option for OraCrsHome/OraAsmHome/OraDbHome to reflect the new public hostname in /inventory.xml

    (4) As a final insurance, take a new OCR export backup after all done.

    Best Regards,
    Yibin

  • Alex Gorbachev says:

    Yibin, well done doing it on Windows and thanks for posting your feedback.

  • Alan Kloss says:

    I think i can shed some light on P Kumar´s issue about ASM not starting via srvctl:

    I had the same problem, the ASM resource was not starting via srvctl on any node,
    not a single message on any log file besides PRKS-1009 and CRS-0215. Both arent explanatory at all.

    After toying around with the RAC scripts(this was a test environment), decided to try to start ASM via the racgrap script, this is the script srvctl calls(you can see the racgwrap path in the .cap files from the rac resources files).

    calling $ORACLE_HOME/bin/racgwrap with no parameters returned that the correct usage was:
    $ORACLE_HOME/bin/racgmain start|stop|check

    so i did that:
    $ORACLE_HOME/bin/racgmain ora.vs10.ASM1.asm start

    And guess what, it gave me the error message i was looking for.
    It pointed out that the ASM instance couldnt be started because the script did not have write permission on the directory $ORACLE_HOME/log/

    Checking this directory permissions showed me that it was owned by root.

    Changing the permissions with chown -R to oracle:oinstall did the trick.

    Now srvctl starts ASM peacefully.

    Cheers,

    Alan

  • monto says:

    Alex-Thanks for this wonderful article,i got a question.We have OS imaging/SAN sync(srdf) from an already existing cluster(rac1/rac2) to another cluster called as node1/node2,the hostnames are different along with the ip’s between them,i have made respective changes in DNS and /etc/hosts and ran roodelete.sh on both nodes and rootdeinstall.sh on primary node.

    I changed the hostname in rootconfig file on both nodes,can i run root.sh instead of rootconfig?I’m i on the right tract please let me as this is my first time with this..

    Thanks

  • @monto: I wouldn’t rely on this for DR. It makes DR activation very convoluted process. A more reliable approach is to only replicate database and have clusterware ready pre-installed and functioning.

  • Oracle Notes » How to Clean Up After a Failed 10g or 11.1 Oracle Clusterware Installation says:

    […] To Change RAC hostname Please review this Note […]

  • Joaquin says:

    Hi Alex

    One of our clients wants to change the hostname of the machine only but of course this will imply changing as well the Oracle RAc hostnames (even though I left the old ones in the /etc/hosts). The commands that you execute will NOT erase any of the onformation store on the database right?

  • INDRANIL says:

    Hi Alex,
    The above article is execellent! I was able to derive in VM Ware , a 2 node RAC – STDB1 instance is running in node rac3 ,STDB2 is running in rac4 from a 2 node RAC – instance LIVE1 in rac1 , LIVE2 instance in LIVE2. Now, I want to use those as like LIVE ( LIVE1(rac1),LIVE2(rac2) ) – Primary Database and STDB ( STDB1(rac3),STDB2(rac4)… Alex, could you please hlep me for that ? Thanks in Advance, INDRANIL,IBM,INDIA

  • INDRANIL says:

    Hi Alex,
    It was as like-
    # Before I read your ARTICLE on “hostname change in CRS”, I had setup a 2 node RAC with host name- rac1 and rac2 and DB Name was LIVE with two instances LIVE1 and LIVE2. This was done using VM Ware in Windows Host. In VM ware, I installed Oracle Linux a flavour from ReD HAT Linux.
    In that I did the setup for 2 Virtual nodes rac1 and rac2 and installed CRS, DB etc.
    #Now After I read your article, what I did is, I copied the VM nodes rac1, rac2 as windows file system and again get2 node RAC with the same host names as rac1,rac2 and LIVE(LIVE1,LIVE2) as DB.
    Then I follow your article and modify the hostnames rac1 , rac2 to rac3, rac4 and modify the DB name LIVE(LIVE1,LIVE2) to STDB(STDB1,STDB2).
    # So, Now I am having 4 nodes in VM ware where – there are two RAC VM machines – say – VM1 and VM2.
    where VM1 is of 2 rac nodes- rac1,rac2
    DB Name= LIVE ( LIVE1, LIVE2)
    AND
    VM2 is of 2 rac nodes – rac3, rac4
    DB Name = STDB (STDB1,STDB2).
    # Now, I need help from you to setup the DATA Guard using those as –
    Primary Database – LIVE ( LIVE1(rac1),LIVE2(rac2))
    and Stand By Database as STDB ( STDB1(rac3),STDB(rac4).
    This was my query to you.
    Thanks again
    INDRANIL,IBM,INDIA,9433056179,indrdas2@gmail.com

  • Paresh says:

    Alex,

    Thank you, saved a ton of time and efforts.
    Here is something additional that needs to be done as part of the hostname change. Re-establish ssh equivalency between nodes (only the part to accept the key) by manually running ssh to other node/s from each node as user oracle.

    I ran into this issue because of need to run vipca and also had to manually add the listeners using netca (the hostname was changed by SA before I got a chance to extract .cap files). Thanks to Brian Lucas who posted the tip on a unrelated thread at – https://forums.oracle.com/forums/message.jspa?messageID=9886788#9886788

    BTW – I also changed all of the IP addresses successfully.

    Thanks,
    Paresh

  • Diego says:

    Alex,

    What are the steps for 11gR2 RAC? I need to change the IP addresses and the domain names.

    Thanks,
    Diego

  • How to Change IP Address? - Bizzteams says:

    […] Cluster … that's solution or run to recreate OCR + Voting Below Site .. can help Sure! Changing Hostnames in Oracle RAC | The Pythian Blog Surachart Opun's Blog: Change hostnames oracle RAC Or just change VIP hostname Surachart Opun's […]

Leave a Reply

  • (will not be published)

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>