A while back, we had a seasonal technology stack update for one of our client's R12.1.3 Oracle E-Business Suite instance. One of the changes introduced was Oracle JDK 7 Update 241, which has a lot of known SSL/TLS protocol and cipher restrictions included. When following the "Enabling TLS in Oracle E-Business Suite Release 12.1 (Doc ID 376700.1)" document, everything operates fine, OHS or OPMN communication happens, and the system is, in general, stable. Later, we had to perform one post step for the 10.1.2.3.2 Forms overlay merge patch - forms.ear re-deployment that is still a manual task with R12.1.3. And that had issues.
$ $FND_TOP/bin/txkrun.pl -script=CfgOC4JApp ... Enter Application name for re-deployment ? forms Enter Oc4j Instance password for re-deployment ? Run Autoconfig <Yes/No> ? No ... Deplolying Application : "forms" onto OC4J instance: "forms" Errors encountered running /u01/app/vis121/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txkCfgOC4JApp.pl *******FATAL ERROR******* PROGRAM : /u01/app/vis121/apps/apps_st/appl/fnd/12.0.0/patch/115/bin/txkCfgOC4JApp.pl(/u01/app/vis121/apps/apps_st/appl/fnd/12.0.0/bin/txkrun.pl) TIME : Tue Oct 8 07:00:57 2019 FUNCTION: TXK::Process::run [ Level 3 ] MESSAGES: Command error: = 256, = /u01/app/vis121/apps/tech_st/10.1.3/jdk/bin/java -jar /u01/app/vis121/apps/tech_st/10.1.3/j2ee/home/admin_client.jar deployer:oc4j:opmn://ebsapp1.domain.com:6507/forms oc4jadmin ****** -deploy -file /u01/app/vis121/apps/tech_st/10.1.2/forms/j2ee/formsapp.ear -deploymentName forms ...Error 256 means the communication fails to establish. Usually, it is a sign of problems with the SSL communication establishment. After a long troubleshooting session, even updating the $OA_JRE_TOP/lib/security/java.security and setting jdk.certpath.disabledAlgorithms/jdk.jar.disabledAlgorithms/jdk.tls.disabledAlgorithms to null to allow everything, we still had nothing. Finally, one note was found - OC4J Service Management May Fail After Update To Java JDK1.6.0_181 Or JDK1.7.0_171 With Oracle E-Business Suite Release 12.1.3 (Doc ID 2353710.1). It described the same problem we had and pointed to a new parameter required to be added and used with R12.1.3 running recent JDK updates - -Djdk.crypto.KeyAgreement.legacyKDF=true. And that note also mentions issues with admin_client.jar. We tried to run the deployer manually and it worked.
$ /u01/app/vis121/apps/tech_st/10.1.3/jdk/bin/java -Djdk.crypto.KeyAgreement.legacyKDF=true -jar /u01/app/vis121/apps/tech_st/10.1.3/j2ee/home/admin_client.jar deployer:oc4j:opmn://ebsapp1.domain.com:6507/forms oc4jadmin ****** -deploy -file /u01/app/vis121/apps/tech_st/10.1.2/forms/j2ee/formsapp.ear -deploymentName forms 19/10/09 08:47:20 Notification ==>Application Deployer for forms STARTS. 19/10/09 08:47:20 Notification ==>Undeploy previous deployment 19/10/09 08:47:21 Notification ==>Initialize /u01/app/vis121/apps/tech_st/10.1.3/j2ee/forms/applications/forms.ear begins... 19/10/09 08:47:21 Notification ==>Initialize /u01/app/vis121/apps/tech_st/10.1.3/j2ee/forms/applications/forms.ear ends... 19/10/09 08:47:21 Notification ==>Starting application : forms 19/10/09 08:47:21 Notification ==>Initializing ClassLoader(s) 19/10/09 08:47:21 Notification ==>Initializing EJB container 19/10/09 08:47:21 Notification ==>Loading connector(s) 19/10/09 08:47:21 Notification ==>Starting up resource adapters 19/10/09 08:47:21 Notification ==>Initializing EJB sessions 19/10/09 08:47:21 Notification ==>Committing ClassLoader(s) 19/10/09 08:47:21 Notification ==>Initialize formsweb begins... 19/10/09 08:47:21 Notification ==>Initialize formsweb ends... 19/10/09 08:47:21 Notification ==>Started application : forms 19/10/09 08:47:21 Notification ==>Application Deployer for forms COMPLETES. Operation time: 558 msecs $
Note 2353710.1 outlines a solution to add this new Java parameter to all "*_jvm_start_options" context variables (oacore, oafm, forms, etc). We did that, ran AutoConfig, called "txkrun.pl -script=CfgOC4JApp" and it failed again. admin_client.jar Java call happened without -Djdk.crypto.KeyAgreement.legacyKDF=true. Long story short, that Note 2353710.1 has a documentation issue, and it's not outlining the solution to fix the admin_client.jar calls. All the arguments are hard in $FND_TOP/patch/115/bin/txkCfgOC4JApp.pl and do not include the new parameter. To fix this, we had to go with customization. It's not difficult to apply it, but you have to shift all "arg" indexes one by one because that new "-Djdk.crypto.KeyAgreement.legacyKDF=true" parameter has to go before the "-jar" specification. Please find the diff example below.
$ diff $FND_TOP/patch/115/bin/txkCfgOC4JApp.pl.before_crypto_manual_fix.09102019 $FND_TOP/patch/115/bin/txkCfgOC4JApp.pl
1216,1227c1216,1228
< arg1 => "-jar",
< arg2 => { text => "$oh/j2ee/home/admin_client.jar",
< type => TXK::Process::TRANSLATE_PATH },
< arg3 => "deployer:oc4j:opmn://$hostname.$domain:$request_port/". $args->{"ocj4instancename"},
< arg4 => "oc4jadmin",
< arg5 => { text => $args->{"oc4jadminpass"},
< secure => TXK::Util::TRUE },
< arg6 => "-deploy",
< arg7 => "-file",
< arg8 => $earFileToBeDeployed,
< arg9 => "-deploymentName",
< arg10 => $args->{"deploymentname"},
---
> arg1 => "-Djdk.crypto.KeyAgreement.legacyKDF=true",
> arg2 => "-jar",
> arg3 => { text => "$oh/j2ee/home/admin_client.jar",
> type => TXK::Process::TRANSLATE_PATH },
> arg4 => "deployer:oc4j:opmn://$hostname.$domain:$request_port/". $args->{"ocj4instancename"},
> arg5 => "oc4jadmin",
> arg6 => { text => $args->{"oc4jadminpass"},
> secure => TXK::Util::TRUE },
> arg7 => "-deploy",
> arg8 => "-file",
> arg9 => $earFileToBeDeployed,
> arg10 => "-deploymentName",
> arg11 => $args->{"deploymentname"},
1298,1299c1299,1301
< arg1 => "-jar",
< arg2 => { text => "$oh/j2ee/home/admin_client.jar",
---
> arg1 => "-Djdk.crypto.KeyAgreement.legacyKDF=true",
> arg2 => "-jar",
> arg3 => { text => "$oh/j2ee/home/admin_client.jar",
1301,1303c1303,1305
< arg3 => "deployer:oc4j:opmn://$hostname.$domain:$request_port/". $args->{"ocj4instancename"},
< arg4 => "oc4jadmin",
< arg5 => { text => $args->{"oc4jadminpass"},
---
> arg4 => "deployer:oc4j:opmn://$hostname.$domain:$request_port/". $args->{"ocj4instancename"},
> arg5 => "oc4jadmin",
> arg6 => { text => $args->{"oc4jadminpass"},
1305,1309c1307,1311
< arg6 => "-bindWebApp",
< arg7 => "-appName",
< arg8 => $args->{"deploymentname"},
< arg9 => "-webModuleName",
< arg10 => $args->{"webmodulename"},
---
> arg7 => "-bindWebApp",
> arg8 => "-appName",
> arg9 => $args->{"deploymentname"},
> arg10 => "-webModuleName",
> arg11 => $args->{"webmodulename"},
1785,1786c1787,1789
< arg1 => "-jar",
< arg2 => { text => "$oh/j2ee/home/admin_client.jar",
---
> arg1 => "-Djdk.crypto.KeyAgreement.legacyKDF=true",
> arg2 => "-jar",
> arg3 => { text => "$oh/j2ee/home/admin_client.jar",
1789,1791c1792,1794
< arg3 => "deployer:oc4j:opmn://$hostname.$domain:$request_port/" . $args->{"ocj4instancename"},
< arg4 => "oc4jadmin",
< arg5 => { text => $args->{"oc4jadminpass"},
---
> arg4 => "deployer:oc4j:opmn://$hostname.$domain:$request_port/" . $args->{"ocj4instancename"},
> arg5 => "oc4jadmin",
> arg6 => { text => $args->{"oc4jadminpass"},
1794,1799c1797,1802
< arg6 => "-deploy",
< arg7 => "-file",
< arg8 => $rarFileToBeDeployed,
< arg9 => "-deploymentName",
< arg10 => $args->{"deploymentname"},
< arg11 => "-grantAllPermissions",
---
> arg7 => "-deploy",
> arg8 => "-file",
> arg9 => $rarFileToBeDeployed,
> arg10 => "-deploymentName",
> arg11 => $args->{"deploymentname"},
> arg12 => "-grantAllPermissions",
1898,1899c1901,1903
< arg1 => "-jar",
< arg2 => { text => "$oh/j2ee/home/admin_client.jar",
---
> arg1 => "-Djdk.crypto.KeyAgreement.legacyKDF=true",
> arg2 => "-jar",
> arg3 => { text => "$oh/j2ee/home/admin_client.jar",
1902,1904c1906,1908
< arg3 => "deployer:oc4j:opmn://$hostname.$domain:$request_port/" . $args->{"ocj4instancename"},
< arg4 => "oc4jadmin",
< arg5 => { text => $args->{"oc4jadminpass"},
---
> arg4 => "deployer:oc4j:opmn://$hostname.$domain:$request_port/" . $args->{"ocj4instancename"},
> arg5 => "oc4jadmin",
> arg6 => { text => $args->{"oc4jadminpass"},
1907,1913c1911,1917
< arg6 => "-publishSharedLibrary",
< arg7 => "-name",
< arg8 => $args->{"libraryname"},
< arg9 => "-version",
< arg10 => $args->{"libraryversion"},
< arg11 => "-addSources",
< arg12 => $library_location,
---
> arg7 => "-publishSharedLibrary",
> arg8 => "-name",
> arg9 => $args->{"libraryname"},
> arg10 => "-version",
> arg11 => $args->{"libraryversion"},
> arg12 => "-addSources",
> arg13 => $library_location,
1991,1992c1995,1997
< arg1 => "-jar",
< arg2 => { text => "$oh/j2ee/home/admin_client.jar",
---
> arg1 => "-Djdk.crypto.KeyAgreement.legacyKDF=true",
> arg2 => "-jar",
> arg3 => { text => "$oh/j2ee/home/admin_client.jar",
1995,1997c2000,2002
< arg3 => "deployer:oc4j:opmn://$hostname.$domain:$request_port/" . $args->{"ocj4instancename"},
< arg4 => "oc4jadmin",
< arg5 => { text => $args->{"oc4jadminpass"},
---
> arg4 => "deployer:oc4j:opmn://$hostname.$domain:$request_port/" . $args->{"ocj4instancename"},
> arg5 => "oc4jadmin",
> arg6 => { text => $args->{"oc4jadminpass"},
2000c2005
< arg6 => "-undeploy",
---
> arg7 => "-undeploy",
$
And that solves the problem with the OC4J re-deployment having recent JDK in place. Maybe, at some point, Oracle is going to introduce an official fix for this, but for now, it's customization. P.S. Please don't forget to flag the customization.
UPDATE ad_files SET is_flagged_file = 'Y', flagged_file_comments = 'Customization to add -Djdk.crypto.KeyAgreement.legacyKDF=true' WHERE app_short_name = 'FND' AND subdir = 'patch/115/bin' AND filename = 'txkCfgOC4JApp.pl'; COMMIT;
Ready to optimize your Oracle Database for the future?