Does Oracle allocate PGA in HugePages?
The short questionWhile preparing a presentation I am giving this year about HugePages, a colleague of mine at Pythian posed an interesting question: Does Oracle use HugePages for PGA?
The short answerNo, it doesn't.
The long questionWe DBAs, not only Oracle but any DBA managing a database that may use large amounts of RAM, should be familiar with the HugePages feature. If you are not, don't miss my presentation at Technical SPOUG day 2018. The basic idea is that memory is allocated in 4KB chunks by default and these chunks are accessed by means of a memory structure, the PageTable, that keeps track of them. For large amounts of RAM, this memory structure can grow very quickly and become slow and cumbersome. If the memory chunks are set to a larger size, say 2MB, the PageTable size is proportionally reduced and the performance is good again. When we think of using this highly recommended feature for Oracle databases, we consider only the SGA to fit into HugePages but not the PGA. Why is that? Because the documentation, MOS notes and blog posts one finds everywhere mention only the SGA. It is hard to find a mention of the PGA, except for a reminder here and there that the total amount RAM allocated for HugePages should leave enough free memory on the system for the OS and the PGA. This alone may serve as a weak argument to say that Oracle does not allocate PGA in HugePages, but the question still stands: Is this true? It occurs to me, and probably other DBAs too, that when I think about PGA I think of it as a single entity, similar to the SGA. While this is helpful as a simplification, the truth is in the definition: The PGA is " the single process private memory structure that contains the data and control information of a server process". So, when I (we) think about PGA, I am actually thinking about the PGA_AGGREGATE_TARGET parameter of the database. While this simplification makes it easier to work with initial memory allocation, it also introduces a bias in reasoning when it comes to PGA and, related to the subject of this blog post, it triggers the question: Does Oracle allocate PGA in HugePages?
The long answer (including a small question)How could the PGA get into HugePages? As explained in this dated but accurate blog post, the HugePages implementation includes interfaces that allow private memory segments to be allocated to HugePages so, theoretically, PGA memory may be allocated here. Then it depends entirely on how Oracle has implemented the memory allocation for PGA. In MOS note " Bigpages vs. Hugetlb on RedHat Linux (Doc ID 261889.1)," the following is written:
A program that wants to allocate shared memory has to add a flag, SHM_HUGETLB, to the shmget() flags. This approach ensures that the Oracle shared memory segments will be allocated out of this pool. [...] Oracle Database 10g will do this by default;While this mentions quite old Oracle versions, I daresay that this still holds true for the newest ones, meaning that the interface used to allocate shared memory (SGA), and hence HugePages, is the shmget() call. Now, according to Fritz Hoogland's findings in his post " Oracle database operating system memory allocation management for PGA – part 4: Oracle 18.104.22.168 and AMM," Oracle allocates private memory using mmap() calls as follows:
mmap(0x7f0194f7a000, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 6, 0) = 0x7f0194f7a000The above call does not include the MAP_HUGETLB flag, which is required for the mmap() call to request memory in the HugePages area, so the PGA is not allocating any HugePages and this is the Oracle developer's chosen way.