In looking at this subject, most of you will say: Hey Yury! What’s new that you can tell us about? The following note describes it all:
11gR2 Grid Infrastructure Does not Use ULIMIT Setting Appropriately [ID 983715.1]
These were exactly my thoughts when I was called to troubleshoot a case where a DB instance, starting with the “srvctl start instance” command failed to use Huge Pages in Linux.
It appears that it isn’t that obvious what exactly you should configure to make sure that your Oracle Instances uses Huge Pages memory regions for shared memory. The tricky area is the /etc/security/limits.conf file.
- grid – 11G Grid infrastructure Linux owner
- oracle – DB Instances owner
Huge pages are configured in Linux already as shown in the following example:
>grep -i huge /proc/meminfo HugePages_Total: 28416 HugePages_Free: 19678 HugePages_Rsvd: 992 Hugepagesize: 2048 kB >cat /proc/sys/vm/nr_hugepages 28416
Q:=A= What exactly do I need to configure to make sure that Oracle Cluster uses Huge Pages?
A: You must configure the following:
grep pam_limits.so /etc/pam.d/login session required pam_limits.so
-2- Adjust /etc/init.d/ohasd
REF: ID 983715.1
cat -n /etc/init.d/ohasd | grep ulimit 71 ulimit -n 65536 72 ulimit -l 58395968
-3- /etc/security/limits.conf for grid user
NOTE: This is what has been missed in the case I was called to troubleshoot
Add the following lines to the /etc/security/limits.conf file
grid soft memlock 15728640 grid hard memlock 15728640 grid soft nproc 2047 grid hard nproc 16384
The important bit to mention is that you must configure limits for “grid” user rather than for “oracle” user as this seems to be logical from the first look.
After all 3 components are set I am sure you will find that Huge Pages are in use if you start your instances using “srvctl” utility.
Q:=B= What exactly do I need to configure to make sure that Huge Pages get used if I use sqlplus to start an Oracle Instance?
A: Add the following lines to the /etc/security/limits.conf file
oracle soft memlock 15728640 oracle hard memlock 15728640 oracle soft nproc 2047 oracle hard nproc 16384
In that case you must configure limits for the user you use to start an Instance.
Q:=C= What else I should be aware of in that context?
If you need to restart Oracle Cluster 11GR2 without restarting the whole server (on OS level) you must do one of the following:
-1- I would suggest to use an adjusted /etc/init.d/ohasd script to restart the Cluster (see question =A=).
-2- If you want to go the other way to restart Cluster processes (like “crsctl start has”) set the following limits before you run the command in your shell environment.
ulimit -n 65536 ulimit -l 50000000
PS I am sure that the bug 9084067 is going to be fixed in the future (if not fixed already).
All the best,
3 Responses to “Oracle DBA: Huge Pages and Oracle Cluster 11GR2”
Leave a Reply