Oracle DBA: Huge Pages and Oracle Cluster 11GR2

Posted in: Technical Track

In looking at this subject, most of you will say: Hey Yury! What’s new that you can tell us about? The following note describes it all:
11gR2 Grid Infrastructure Does not Use ULIMIT Setting Appropriately [ID 983715.1]

These were exactly my thoughts when I was called to troubleshoot a case where a DB instance, starting with the “srvctl start instance” command failed to use Huge Pages in Linux.

It appears that it isn’t that obvious what exactly you should configure to make sure that your Oracle Instances uses Huge Pages memory regions for shared memory. The tricky area is the /etc/security/limits.conf file.

– grid – 11G Grid infrastructure Linux owner
– oracle – DB Instances owner
Huge pages are configured in Linux already as shown in the following example:

>grep -i huge /proc/meminfo
HugePages_Total: 28416
HugePages_Free:  19678
HugePages_Rsvd:    992
Hugepagesize:     2048 kB
>cat /proc/sys/vm/nr_hugepages

Q:=A= What exactly do I need to configure to make sure that Oracle Cluster uses Huge Pages?
A: You must configure the following:

grep /etc/pam.d/login
session    required

-2- Adjust /etc/init.d/ohasd
REF: ID 983715.1

cat -n /etc/init.d/ohasd  | grep ulimit
    71    ulimit -n 65536
    72    ulimit -l 58395968

-3- /etc/security/limits.conf for grid user
NOTE: This is what has been missed in the case I was called to troubleshoot
Add the following lines to the /etc/security/limits.conf file

grid soft memlock 15728640
grid hard memlock 15728640
grid soft nproc 2047
grid hard nproc 16384

The important bit to mention is that you must configure limits for “grid” user rather than for “oracle” user as this seems to be logical from the first look.

After all 3 components are set I am sure you will find that Huge Pages are in use if you start your instances using “srvctl” utility.

Q:=B= What exactly do I need to configure to make sure that Huge Pages get used if I use sqlplus to start an Oracle Instance?
A: Add the following lines to the /etc/security/limits.conf file

oracle soft memlock 15728640
oracle hard memlock 15728640
oracle soft nproc 2047
oracle hard nproc 16384

In that case you must configure limits for the user you use to start an Instance.

Q:=C= What else I should be aware of in that context?

If you need to restart Oracle Cluster 11GR2 without restarting the whole server (on OS level) you must do one of the following:
-1- I would suggest to use an adjusted /etc/init.d/ohasd script to restart the Cluster (see question =A=).
-2- If you want to go the other way to restart Cluster processes (like “crsctl start has”) set the following limits before you run the command in your shell environment.

ulimit -n 65536
ulimit -l 50000000

PS I am sure that the bug 9084067 is going to be fixed in the future (if not fixed already).

All the best,

Interested in working with Yury? Schedule a tech call.

About the Author

Yury is a nice person who enjoys meeting and working with people on challenging projects in the Oracle space. He started to work as an Oracle DBA 14 years ago (1997). For the past 10 years (since 2001) his main area of expertise is Oracle e-Business Suite. Yury is an OCP 7,8,9,10g and OCM 9i,10g. He is a frequent presenter at Oracle related conferences such as Hotsos, UKOUG and AUOUG. Yury is a socially active person. Apart from Social Media (Twitter, Bloging, Facebook) he is the primary organizer of Sydney Oracle Meetup group (250 people). So if you happen to be in Sydney (Australia) drop Yury a message and stop by at one of his Meetups.

3 Comments. Leave new

Tweets that mention Oracle DBA: Huge Pages and Oracle Cluster 11GR2 | The Pythian Blog --
February 8, 2011 10:22 pm

[…] This post was mentioned on Twitter by, John Seaman. John Seaman said: RT @orana: Oracle DBA: Huge Pages and Oracle Cluster 11GR2 […]


Hi Yury,

In we have the parameter use_large_pages which prevents the instance from opening without huge pages,


Hey Yasin!

Thank you for following and providing an additional reference.
If I understood it correctly the things about limits.conf settings is still true even in The new parameter most probably covers the ohasd file issue.
IMHO: We still need to set the limits in an appropriate way for version.

Thank you once again. I am sure readers will appreciate your input.



Leave a Reply

Your email address will not be published. Required fields are marked *