Oracle's Exadata platform is a great way to run production Oracle database workloads. All of the newer releases of Exadata support elastic configurations, meaning that you could add compute and storage capacity to the rack per your requirements. In this two-part blog series, we will be looking at how to expand an Exadata X9M rack to provision additional storage servers.
Part 1: Reimage new servers
When you purchase new Exadata storage or compute servers from Oracle, the servers would most likely arrive with a factory default Exadata software version different from the version you are running in your existing Exadata rack.
For example, our production Exadata was running software version 21.2.11.
[root@exacel01 ~]# imageinfo
Kernel version: 4.14.35-2047.511.5.5.1.el7uek.x86_64 #2 SMP Wed Apr 6 10:05:39 PDT 2022 x86_64
Cell version: OSS_21.2.11.0.0_LINUX.X64_220414.1
Cell rpm version: cell-21.2.11.0.0_LINUX.X64_220414.1-1.x86_64
Active image version: 21.2.11.0.0.220414.1
Active image kernel version: 4.14.35-2047.511.5.5.1.el7uek
Active image activated: 2022-10-24 09:19:21 -0700
Active image status: success
Active node type: STORAGE
Active system partition on device: /dev/md24p6
Active software partition on device: /dev/md24p8
Cell boot usb partition: /dev/md25p1
Cell boot usb version: 21.2.11.0.0.220414.1
The new storage servers delivered to us were running version 22.1.9.
[root@node12 ~]# imageinfo
Kernel version: 4.14.35-2047.518.4.2.el7uek.x86_64 #2 SMP Thu Nov 3 14:28:31 PDT 2022 x86_64
Uptrack kernel version: 4.14.35-2047.522.3.el7uek.x86_64 #2 SMP Fri Jan 20 16:05:02 PST 2023 x86_64
Cell version: OSS_22.1.9.0.0_LINUX.X64_230302
Cell rpm version: cell-22.1.9.0.0_LINUX.X64_230302-1.x86_64
Active image version: 22.1.9.0.0.230302
Active image kernel version: 4.14.35-2047.518.4.2.el7uek
Active image activated: 2023-06-05 18:23:12 -0700
Active image status: success
Active node type: STORAGE
Active system partition on device: /dev/md24p5
Active software partition on device: /dev/md24p7
Cell boot usb partition: /dev/md25p1
Cell boot usb version: 22.1.9.0.0.230302
Although the kernel release on both servers is very similar, the Exadata software release is at a different level. Oracle recommends that the new servers be reimaged with the version of the existing servers to ensure they can be integrated into the rack without adverse effects. You can log in to the hosts before reimage and note down the hostname and IP addresses assigned to the interfaces. The same names and IP addresses can be used later after installation.
To perform the reimage, we must obtain the ISO to complete the OS image boot. This ISO can be obtained from edelivery.oracle.com, as shown here:
Extract the zip file downloaded from edelivery and use the iso image contained in the zip file.
There are several ways to reimage an Exadata host using an ISO image. I prefer the web ILOM method as it is the most convenient. Ensure you have Java Runtime Environment installed on your Mac/PC before starting the boot process.
On the ILOM console, go to Remote Control -> Redirection -> Launch Remote Console.
Go to KVMS -> Storage on the console to attach the ISO image. Once attached, select the image, click Connect to make it available to the host, and select OK to close the menu.
Back on the ILOM console, go to Host Management -> Host Control and select CDROM as the next boot device.
Finally, restart the host to kick off the boot process from Host Management -> Power Control.
It would take about an hour for the installation process to complete. Once the installation is complete, you will be asked to provide the hostname and network configuration details. Provide the same hostname and network details you saved before the installation process started. The rest of the settings, like DNS, NTP or ILOM settings, can be skipped or left as default.
Once the installation is complete, you can verify the Exadata software image installed with the ISO image.
[root@node12 ~]# imageinfo
Kernel version: 4.14.35-2047.511.5.5.1.el7uek.x86_64 #2 SMP Wed Apr 6 10:05:39 PDT 2022 x86_64
Cell version: OSS_21.2.11.0.0_LINUX.X64_220414.1
Cell rpm version: cell-21.2.11.0.0_LINUX.X64_220414.1-1.x86_64
Active image version: 21.2.11.0.0.220414.1
Active image kernel version: 4.14.35-2047.511.5.5.1.el7uek
Active image activated: 2023-10-18 11:27:55 -0700
Active image status: success
Active node type: STORAGE
Active system partition on device: /dev/md24p5
Active software partition on device: /dev/md24p7
Cell boot usb partition: /dev/md25p1
Cell boot usb version: 21.2.11.0.0.220414.1\
As a final step, create an empty file .elasticConfig under the / filesystem. This file is needed for the configuration's next steps, which we will discuss in Part 2 of this blog series.
