Anybody who has tried this multi-node to single-node cloning in 11i knows that it’s difficult and very error-prone. If we outline the Apps Tier cloning process, it looks like this (supposing we have a two-node instance with the DB and CM on one node, and Web and Forms on the other):

# CM Node
# copy all files to the target
$ perl adpreclone.pl appsTier merge

# Web Node
# copy $COMMON_TOP/clone/appl directory only to the target
$ perl adpreclone.pl appltop merge

# Target Node
$ perl adcfgclone.pl appsTier

This process is called merging appltops. It’s not as easy as it looks. Many times, the production environment might not have proper values in the fnd_nodes table, which makes adcfgclone.pl fail to properly recognize the appltops for merging.

But in R12, life is made easy. (more…)

One of the main goals in architecting a Disaster Recovery (DR) solution is to make a DR failover transparent to the end users. Too often, users must reboot their desktops, clear their browser cache and the jinitiator jar cache, and so on, even when we have made sure that the post-failover URL of the 11i instance is the same. After a failover of an 11i instance from a primary site to a DR site, if the user can operate without changing anything in his desktop, only then can we say that the goal is achieved.

In most cases the culprits are: forgetting the DNS setup for the hostnames of Middle Tiers, or the load balancer, if one is used; and the caching of DNS entries at the different levels in the network. A quick look at the caching section of Wikipedia’s page on DNS gives some idea of I’m talking about. Because of the default settings, the old IP address gets cached in the user’s desktop and in caching DNS servers in the network. As a result, the user’s desktop is still trying to reach the old server, which is now offline.

The best fix for these kind of DNS side effects is to change the TTL (Time To Live) parameter of the DNS entry for the hostname from the default value to a smaller one. I prefer setting it to a value a little smaller than the time you take to failover. That is, if you take 60 minutes to failover from Primary to Secondary datacenter, then set the TTL to 50 minutes.

Let’s take an example here. Let’s say our 11i instance has the URL http://apps.example.com:8000, the primary instance being windsor, the secondary ottawa. And we have two load balancers: one at primary site and one at the secondary, with hostnames lb.windsor.example.com and lb.ottawa.example.com respectively. If the DNS is set up with default values, it will look like this:

(more…)

Oracle R12 is certified to install on 64-bit Linux for X86_64 platforms. Yeah, I know that’s old news. The interesting part here is it’s not “pure” 64-bit software like the Oracle Database on Linux 64-bit: R12 is taking advantage of the OS’s capability to run 32-bit executables.

If we look at metalink note 416305.1, there is a big list of 32-bit RPMs in the pre-req packages list for R12 on 64-bit Linux. This is not something new — 11i on Solaris, HP-UX, and AIX also had the same behaviour.

11i doesn’t take advantage of 64-bit capabilities of the OS at full scale. It runs in 32-bit mode even on a 64-bit OS, as all its executables are 32-bit. One place where 11i does use the 64-bit feature is the “Memory Based Planner” program, which is part of the Advanced Supply Chain Planning (ASCP) module. Oracle shipped a 64-bit version of the program only for the Solaris, AIX, and HP-UX platforms, while the Linux version had just the 32-bit version. I have seen customers who added a Solaris Middle Tier to their Linux 11i instances just to run this 64-bit Planner program, spending all the dollars they saved with Linux middle tiers thus far, in consulting fees to add that extra Solaris middle tier node.

The good news for R12 customers is that, we don’t need a Solaris, AIX or HP-UX middle tier any more to run the 64-bit Planner program. Oracle has started shipping 64-bit Planner (MSONWL64.exe) for Linux 64-bit from version 12.0.3 onwards. Check Metalink note 254810.1 for details.

If you are a early adopter of R12 with a version earlier than 12.0.3, its time to patch up to 12.0.3 or 12.0.4. and migrate to the Linux 64-bit platform. This migration to Linux 64-bit should not be a big hassle, as it is binary-compatible with Linux 32-bit. I expect it be as simple as 1) copy, 2) relink, and 3) startup. I will, however, find out the exact method, and post here in the blog.

From my previous post on TXK rollup patch, you already know the significance of adctxinf.tmp file in $AD_TOP/admin/template directory. It has wealth of information about different XML tags in the Context XML file of an Apps 11i instance. In relation to the same file, now I want to share with you all a small XSL (XML style sheet) file I wrote back in 2005. It makes adctxinf.tmp much more readable; all tags are presented in a tabular format in the browser.

For users of Microsoft Internet Explorer, Firefox, and Safari, go directly to this link to view the file. Those of you who want to have an offline copy, download that file and vasu.xsl to same directory and open adctxinf.xml in any app of your choice. Its just a very simple XSL file, but it should make life a little easier. My goal when I wrote this stylesheet was to make the Context XML file itself much more human-readable. But I never got much time to work on that.

It’s true — XML is for machines; HTML is for humans.

You might have already seen the blog update on Steven Chan’s site. TXK AutoConfig and Templates Rollup Patch S (6372396) was released on May 5th.

This patch differs from traditional TXK autoconfig template patch releases in that the ATG team decided to include some other important TXK patches also with this release. One of these is TXK Advanced Utilities Rollup Patch C (5011249).

This Advanced Utilities patch brings some import scripts that can be used to implement advanced topologies. The important benefit here is that these scripts can be run from command line. For complete details, refer to Metalink note 277574.1, Running Configuration Wizards from the Command Line in Oracle Applications 11i.

As a side effect of this generous inclusion of import updates, the patch size has increased from 16mb (RUP R) to 65mb (RUP S).

This Rollup Patch has also brought in some new Context variables. These new XML tags in Rollup Patch S are:

1. s_jdbc_connect_descriptor_generation: The value of this variable determines whether the jdbc connect descriptor is regenerated and whether the value of the context variable s_apps_jdbc_connect_descriptor is updated. Acceptable values for this context variable are true (the default) or false. Set this value to false when using a custom apps jdbc connect descriptor
2. s_apcprestart: This variable is used to specify the complete path of the script to be executed before the Apache service is started.
3. s_ccmmaxsyspathlen: (Windows-specific) This context variable is used for setting the maximum length of the Windows System Path.

(more…)

Today I ran into an 11i E-Business Suite instance which is using Apache/JServ to do forms load-balancing. In fact, it’s my first customer to implement and successfully use Apache/JServ for load-balancing. Here is quick sketch of the instance architecture:

                  Users
                    | 11i URL:
                    | http://appsrv1.pythian.com:8000/
                    |
             ----------------
             |   appsrv1    |
             | apache/jserv |
             |  formservlet |
             ----------------
            /                \
           /                  \
----------------             ----------------
|   appsrv2    |             |   appsrv3    |
| apache/jserv |             | apache/jserv |
|  formservlet |             |  formservlet |
----------------             ----------------

            -------------------
            |    dbsrv01      |
            | CM/RepSrv       |
            |  Database       |
            -------------------

Almost all the customers that I have worked on till now, with a requirement for multiple middletier, have implemented load balancing using hardware load-balancers like Cisco Content Switch or F5 Big-IP.

This customer is running forms in servlet mode, a prerequisite for using Jserv to do load-balancing. If somebody wants to do forms load-balancing in socket mode and opts not to use any hardware LBRs, then the only option is a Metrics server.

Here is brief overview on how forms load-balancing happens in above architecture. (more…)

It is quite common for Oracle Tech support, while troubleshooting any 11i E-Business Suite Self Service Applications (SSWA) related error messages, to ask to enable debug logging in JServ configuration files.

The procedure to enable debug logging in JServ is to update following line in file $IAS_ORACLE_HOME/Apache/Jserv/etc/jserv.properties from log=false to log=true, and then bounce Apache using:

$ cd $COMMON_TOP/admin/scripts/$CONTEXT_NAME
$ adapcctl.sh stop
$ adapcctl.sh start

After the bounce, we start seeing debug messages in the log file $IAS_ORACLE_HOME/Apache/Jserv/logs/jserv.log.

Please note, the logfile’s and config file’s locations can be different from those mentioned above. In case you have a shared tech stack implemented, replace $IAS_ORACLE_HOME with $CONF_TOP/iAS.

The main problem with this procedure is the bounce of Apache. In cases where the client is not a heavy SSWA user and the issue isn’t affecting a significant number of users, then often we will be asked to wait until off-business hours to do the bounce of Apache. These delays can make your metalink SR hop between different Oracle support offices in different timezones, which can lead to delays in resolution of the issue.

Let’s have quick review of JServ configuration files which are of interest to us. jserv.conf — this is the configuration file of mod_jserv module in Apache. Here’s a snippet from jserv.conf: (more…)

My name is Vasu Balla, and I’ve been with Pythian for about four months now. I have worked on Oracle E-Business Suite instances for over five years, and I’ve never had a moment where I felt bored.

I am constantly challenged with new technologies and new issues, and my Pythian team’s clients continue to present interesting issues. The most difficult are the ones that are very intermittent or not easily reproducible. If you try to get answers for these kind of issues from Oracle Tech Support by opening a SR, your chances are next to none. Our team recently encountered an issue a client had had with Oracle Configurator for two years. They had followed up with Oracle Tech Support for over a year and had eventually learned just to live with the problem. When Pythian came in, we were shocked to learn the history of the problem. It turned out to be one of the most exciting problems we’ve resolved.

I am new to blogging, but quite experienced in reading blog posts. My Google Reader collects over 100 articles per day, mostly around my interests, which are:

• Oracle 11i E-Business Suite Advanced topologies
  • Hardware Load Balancing
  • DNS based Load Balancing
  • Jserv Load Balancing
• Oracle 11i SSL Implementation at Load balancer level
• Forms load balancing using Jserv + forms servlet mode
• Oracle Concurrent Managers PCP setup
• Oracle 11i Apps with RAC configuration

Signing off, now — with a promise to post some interesting stuff on topics such as these. Stay tuned.