Why is my Forms 12c Application so slow?
After a recent upgradeUsers cannot work with the forms application anymore; it has been a few weeks since an upgrade to Forms 12c. The usual suspects have been investigated.
- The database is not waiting on forms transactions
- The application server is not in any distress, and yes, we have fluctuations in server CPU usage, but well within limits
- The user PCs (running the forms in a Browser) are not seeing any issues like CPU/memory limitations
- The forms are very slow in navigation and at odd times during the day they "freeze"
So why is it slow?In essence, if the application is not responding quickly we have these classical areas to review
- The applet is slow to respond to the user input
- The frmweb process is slow to respond to the events received from the Applet
- The frmweb process is waiting on the database to provide it data
- Other unknown reasons
[oracle@oraserver1 karun]$ ls -ltr $DOMAIN_HOME/system_components/FORMS/forms1/trace | tail -3 ... -rw-r----- 1 oracle dba 5882480 May 6 10:50 forms_26737.trc -rw-r----- 1 oracle dba 759326 May 6 18:00 forms_17824.trc -rw-r----- 1 oracle dba 1209818 May 6 18:15 forms_16979.trc [oracle@oraserver1 karun]$The easy way to start a trace Use the Enterprise Manager Control: Select a session and click on Enable Tracing Choose the tracegroup (I usually choose to run with debug as this has all the details). After a few minutes of tracing you can click on Disable Tracing button. And EM can also show you the trace file contents. The other easy way to start a trace This is what we can also do when the EM is not accessible ( or you do not really like the pointy-clicky stuff) Start the form on the client PC with this URL:
https://<server>:<port>/forms/frmservlet?config=<your_app_config>&record=forms&tracegroup=debug Work for a few minutes and close the application. Having generated trace files, we now have to read them on the forms application server after we translate them to text. $JAVA_HOME/bin/java -classpath $ORACLE_HOME/jlib/frmxlate.jar oracle.forms.diagnostics.Xlate datafile=$DOMAIN_HOME/system_components/FORMS/forms1/trace/forms_17824.trc outputfile=trace_my_session_20180506.txt outputclass=WriteOutTEXT I can also use different options if I want to translate the trace file to HTML or XML with
What can traces tell usA typical trace file is an engineer's delight. To give you an idea, this is a snippet
9615 [BUILTIN.END,3] Timestamp=53160, StartEvent=9614, Duration=0 9616 [BUILTIN.START,3] Timestamp=53160, EndEvent=9617, Name=SET_ITEM_INSTANCE_PROPERTY [Arguments] Type=In Position=1 DataType=INTEGER Value=131081 Type=In Position=2 DataType=NUMBER Value=0 Type=In Position=3 DataType=NUMBER Value=1526 Type=In Position=4 DataType=STRING Value=BACKGROUND Type=In Position=5 DataType=STRING Value=NULL 9617 [BUILTIN.END,3] Timestamp=53160, StartEvent=9616, Duration=0 9618 [Local_PU.END,2] Timestamp=53160, StartEvent=9613, Duration=0 9619 [TRIGGER.END,1] Timestamp=53160, StartEvent=9612, Duration=09620 [NETWORK.WRITE] Timestamp=53160, StartEvent=9601, Duration=0, Packets=1, Bytes=742 9621 [NETWORK.READ] Timestamp=53220, EndEvent=9625, Duration=0, Packets=1, Bytes=270
9622 [ECID] Timestamp=53220, Value=XXXXXXXXXX #9623 [Key] Timestamp=53220, FormName=XXX__YYY(3), KeyPressed=Up 9624 [ERROR] Timestamp=53220, Msg= Error Message: FRM-40100: At first record. 9625 [NETWORK.WRITE] Timestamp=53220, StartEvent=9621, Duration=0, Packets=1, Bytes=94
Fun with traces
- I extracted the NETWORK events timestamps.
[oracle@oraserver1 karun]$grep "NETWORK" trace_my_session_20180506.txt | tail -200
13378 [NETWORK.READ] Timestamp=185260, EndEvent=13397, Duration=0, Packets=1, Bytes=312 13397 [NETWORK.WRITE] Timestamp=185260, StartEvent=13378, Duration=0, Packets=1, Bytes=650 13398 [NETWORK.READ] Timestamp=185390, EndEvent=13417, Duration=0, Packets=1, Bytes=315 13417 [NETWORK.WRITE] Timestamp=185390, StartEvent=13398, Duration=0, Packets=1, Bytes=704 13418 [NETWORK.READ] Timestamp=185520, EndEvent=13437, Duration=0, Packets=1, Bytes=318 ... ...
- I graphed the timestamp data. I am interested in the timestamp intervals.