Physical Disk, Cell Disk, Grid Disk, and ASM Disk in Exadata
Exadata is the Ferrari of the technology world. It’s considered the technology for the select few, but everyone dreams about it. There is very little information available on the Internet right now about the architecture and the administration of Exadata, though it’s a huge subject in itself, which demand numerous high quality books, dedicated blogs, websites and special interest groups.
This blog post just mentions some of very important components of Exadata Storage Server, which are physical disks, cell disks, grid disks and ASM disks and their correspondence.
The Exadata Storage Server contains 12 physical disks.
There is one to one relationship between a physical disk and the cell disk. One physical disk corresponds to a single cell disk.
Then a Grid disk comprises of many cell disks.
And a Grid disk corresponds to one ASM disk.
An ASM diskgroup comprises of many ASM disks.
On the Exadata Storage Server, We can use cellcli command line utility in Exadata to see the information about physical disks, cell disks, grid disks and the cell.
The cellcli utility works from the root, celladmin and cellmonitor (read-only) users. The best practice is actually to run it from the last two less-privileged users, and not from the root user.
Now let’s have look at some of the disk management commands using cellcli utility.
[root@mycell-net0 ~]# cellcli CellCLI: Release 11.2.1.3.1 - Production on Fri Oct 29 07:47:26 GMT 2010 Copyright (c) 2007, 2009, Oracle. All rights reserved. Cell Efficiency Ratio: 140
Just to give you an idea about what cellcli has on offer, look at the output of the help command:
CellCLI> help HELP [topic] Available Topics: ALTER ALTER ALERTHISTORY ALTER CELL ALTER CELLDISK ALTER GRIDDISK ALTER IORMPLAN ALTER LUN ALTER THRESHOLD ASSIGN KEY CALIBRATE CREATE CREATE CELL CREATE CELLDISK CREATE FLASHCACHE CREATE GRIDDISK CREATE KEY CREATE THRESHOLD DESCRIBE DROP DROP ALERTHISTORY DROP CELL DROP CELLDISK DROP FLASHCACHE DROP GRIDDISK DROP THRESHOLD EXPORT CELLDISK IMPORT CELLDISK LIST LIST ACTIVEREQUEST LIST ALERTDEFINITION LIST ALERTHISTORY LIST CELL LIST CELLDISK LIST FLASHCACHE LIST FLASHCACHECONTENT LIST GRIDDISK LIST IORMPLAN LIST KEY LIST LUN LIST METRICCURRENT LIST METRICDEFINITION LIST METRICHISTORY LIST PHYSICALDISK LIST THRESHOLD SET SPOOL START
Let’s see the output of some of the commands listed above:
CellCLI> list physicaldisk detail name: [5:3:2:0] diskType: FlashDisk id: 00000200000000000000 luns: 1_2 makeModel: "MARVELL SD88SA02" physicalFirmware: D20R physicalInsertTime: 2010-10-04T21:05:46+00:00 physicalInterface: sas physicalSerial: 0000000000000000000 physicalSize: 22.8880615234375G slotNumber: "PCI Slot: 1; FDOM: 2" status: normal <pre>
CellCLI> list cell detail name: mycellnet0 bmcType: IPMI cellVersion: OSS_11.2.0.1.0_LINUX.X64_100818.1 cpuCount: 16 fanCount: 12/12 fanStatus: normal id: 0000X00000 interconnectCount: 3 interconnect1: bond0 iormBoost: 0.0 ipaddress1: kernelVersion: 2.6.18-194.3.1.0.2.el5 makeModel: SUN MICROSYSTEMS SUN FIRE X4275 SERVER SAS metricHistoryDays: 7 notificationMethod: snmp notificationPolicy: critical,warning,clear offloadEfficiency: 47,485.1 powerCount: 2/2 powerStatus: normal smtpFrom: smtpFromAddr: smtpPort: smtpServer: smtpToAddr: smtpUseSSL: snmpSubscriber: status: online temperatureReading: 27.0 temperatureStatus: normal upTime: 16 days, 14:01 cellsrvStatus: running msStatus: running rsStatus: running <pre>
CellCLI> list celldisk detail name: FD_15_mycellnet0 comment: creationTime: 2010-07-14T02:05:06+00:00 deviceName: /dev/sdy devicePartition: /dev/sdy diskType: FlashDisk errorCount: 0 freeSpace: 0 id: 00000000-0000-0000-0000-000000000000 interleaving: none lun: 5_3 size: 22.875G status: normal <pre>
CellCLI> list griddisk detail name: CD_15_mycellnet0 availableTo: cellDisk: CD_15_mycellnet0 comment: creationTime: 2010-07-14T02:12:55+00:00 diskType: FlashDisk errorCount: 0 id: 00000000-0000-000-0000-000000000000 offset: 528.734375G size: 29.125G status: active
In the future posts, I will be touching the administration of celldisks, as how to perform operations like (import/export/create/drop/alter) on the celldisks.
