Oracle ASM 11g: Does the ASMCMD cp Command Really Work?
Since the introduction of ASM in Oracle 10g Release 1, every ASM administrator has been dreaming of a simple command line tool to copy files between ASM diskgroups and other filesystems. Oracle ASM 10g Release 2 added the handy asmcmd utility, but even though everyone expected a copy command there, it had not been implemented.
The only way to copy files to or from an ASM diskgroup was either to use RMAN, to configure XDB for FTP access, or use the DBMS_FILE_TRANSFER package. No wonder that the cp command is the most popular addition to asmcmd tool in Oracle ASM 11g: the hardest barrier to convincing my customers to use ASM has been the inability to access the files and copy them to the OS filesystem using the command-line copy command. Customers wanted to “feel” the files and be able to easily manipulate them.
While working on a Collaborate 08 presentation on Oracle 11g new features out-of-the-box, I was verifying new commands in Oracle ASM 11g’s asmcmd utility. It turned our that copying files from or to ASM is still a problem.
First, I tried to copy a single text file to an ASM diskgroup:
ASMCMD> cp /home/oracle/.bash_profile +dg2/test.file source /home/oracle/.bash_profile target +dg2/test.file ASMCMD-08012: can not determine file type for file->’/home/oracle/.bash_profile’ ORA-15056: additional error message ORA-17503: ksfdopn:DGGetFileAttr15 Failed to open file /home/oracle/.bash_profile ORA-27046: file size is not a multiple of logical block size Additional information: 1 ORA-06512: at “SYS.X$DBMS_DISKGROUP”, line 207 ORA-06512: at line 3 (DBD ERROR: OCIStmtExecute)
Hmm . . . Okay. Let’s try to do it in multiples of diskgroup blocks:
ASMCMD> lsdg State Type Rebal Sector Block AU Total_MB… MOUNTED EXTERN N 512 4096 1048576 2048… MOUNTED NORMAL N 512 4096 2097152 200… ASMCMD> exit [oracle@lh8 ~]$ dd if=/dev/zero of=/home/oracle/test2.file bs=4k count=10 10+0 records in 10+0 records out [oracle@lh8 ~]$ asmcmd ASMCMD> cp /home/oracle/test2.file +DG2 source /home/oracle/test2.file target +DG2/test2.file ASMCMD-08012: can not determine file type for file->’/home/oracle/test2.file’ ORA-15056: additional error message ORA-19762: invalid file type DGGetFileAttr20 ORA-06512: at “SYS.X$DBMS_DISKGROUP”, line 207 ORA-06512: at line 3 (DBD ERROR: OCIStmtExecute)
Oops. It seems I can’t put any file to ASM. Not that I’m very surprised — I expected that ASM would “automagically” try to place all files based on OMF standards and templates. There are, however, only templates and rules for database files in 11g.
Right, let me try to backup a controlfile to a filesystem and copy it to ASM:
RMAN> backup format ‘/tmp/backup.ctl’ current controlfile;
…
channel ORA_DISK_1: finished piece 1 at 06-APR-08
piece handle=/tmp/backup.ctl tag=TAG20080406T202034 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:03
…
ASMCMD> cp /tmp/backup.ctl +dg2/backup.ctl
source /tmp/backup.ctl
target +dg2/backup.ctl
copying file(s)…
file, +DG2/backup.ctl, copy committed.
ASMCMD> ls -l +dg2/backup.ctl
Type Redund Striped Time Sys Name
N backup.ctl => +DG2/ASMTESTING/BACKUPSET/TESTING.256.651356493
Alright. That seems to work, except that ASM chose a bizarre location. For some reason, I’m not surprised again — I kind of expected it to place it somewhere into DB_UNKNOWN like RMAN does when recovering SPFILE from autobackup. In the best Oracle traditions of keeping things consistent, ASMTESTING seems to be reasonable. Was it a hard-coded leftover from the test implementation of the cp command? I wouldn’t be surprised, it’s deja-vu.
We can be patient and forgive this for the first release. Let’s try to copy it to a filesystem and back:
ASMCMD> cp +dg2/backup.ctl /tmp/backup.ctl2 source +dg2/backup.ctl target /tmp/backup.ctl2 copying file(s)… file, /tmp/backup.ctl2, copy committed. ASMCMD> cp /tmp/backup.ctl2 +dg2/backup.ctl2 source /tmp/backup.ctl2 target +dg2/backup.ctl2 ASMCMD-08012: can not determine file type for file->’/tmp/backup.ctl2′ ORA-15056: additional error message ORA-19762: invalid file type DGGetFileAttr20 ORA-06512: at “SYS.X$DBMS_DISKGROUP”, line 207 ORA-06512: at line 3 (DBD ERROR: OCIStmtExecute)
Now, that is odd. Perhaps I need to register this file with the database (i.e. in the controlfile)? Let’s ask RMAN to catalog this file:
[oracle@lh8 ~]$ ls -l /tmp/backup.ctl* -rw-r—– 1 oracle dba 9797632 Apr 6 20:20 /tmp/backup.ctl -rw-r—– 1 oracle dba 9797632 Apr 6 20:22 /tmp/backup.ctl2 … RMAN> catalog start with ‘/tmp/backup.ctl2′; … List of Files Which Where Not Cataloged ======================================= File Name: /tmp/backup.ctl2 RMAN-07517: Reason: The file header is corrupted
So the file got corrupted while copying from ASM to a filesystem? Okay. Let’s try an ASM -> ASM copy:
ASMCMD> cp +dg2/backup.ctl +dg2/backup.ctl3 source +dg2/backup.ctl target +dg2/backup.ctl3 copying file(s)… file, +DG2/backup.ctl3, copy committed. … RMAN> catalog start with ‘+dg2′; … List of Files Which Where Not Cataloged ======================================= File Name: +dg2/backup.ctl RMAN-07517: Reason: The file header is corrupted File Name: +dg2/backup.ctl3 RMAN-07517: Reason: The file header is corrupted
Hey, both files are actually corrupted, so the corruption occurred in the first copy, from the filesystem to ASM. Alright. Let’s try to simply copy a current controlfile within ASM:
ASMCMD> cp +dg1/db11g/controlfile/Current.256.651275203 +dg2/asm_copy
source +dg1/db11g/controlfile/Current.256.651275203
target +dg2/asm_copy
copying file(s)…
file, +DG2/asm_copy, copy committed.
ASMCMD> ls -l +dg2/asm_copy
Type Redund Striped Time Sys Name
N asm_copy => +DG2/ASMTESTING/CONTROLFILE/TESTING.258.651358725
…
RMAN> catalog start with ‘+dg2′;
…
List of Files Which Where Not Cataloged
=======================================
File Name: +dg2/asm_copy
RMAN-07517: Reason: The file header is corrupted
These results are disappointing — I couldn’t make the cp command work even a single time.
I should note that I did ask my old good friends, Metalink and Google, about ASMCMD-08012 and the like, but they came back empty.
If anyone has been able to test the ASMCMD cp command in 11g, please share your experience.
April 8th, 2008 at 2:44 pm
Hi Alex,
Well spotted, I have reproduced your testcase in my DB too. I wouldn’t sat that 11g asmcmd cp command doesn’t work in all cases though! For example, it seems to work fine when copying datafiles.
BTW, it shouldn’t be too difficult to backport asmcmd cp to 10g using the X$DBMS_DISKGROUP package and the perl scripts that one can find in 11g. It’s been in my ‘list of things to try out when I have some time’ for a while..
Cheers,
Luca
April 8th, 2008 at 3:58 pm
Thanks Luca. Interesting, it looks like the problem is specific to certain file types then.
After I saw your first research on ASM internals, the idea of writing a copy utility didn’t leave me but you have better chances to find some time than me. :-)
Actually, it might be relatively easy to port it from 11g to 10g unless there is specific support in the database itself required. I would assume that XDB FTP in 10g is based on the same ideas so back port should be feasible. On the other hand, upgrading ASM to 11g could be an easier option.
Btw, did you use standard 1 MB AU size? I just noticed than me DG2 diskgroup was created with 2 MB allocation unit following my tests for variable AU size.
April 8th, 2008 at 11:44 pm
Definitely something to discuss with the (Oracle, and other) gurus next week at Collaborate. I haven’t tried it myself, but agree that this is one of the most anticipated new features in ASMCMD, possibly ASM in general. For it to be so highly anticipated and not function as expected is certainly disappointing.
See you next week!
April 9th, 2008 at 2:55 am
Really i’ve not yet testet ASM 11g new cp command, but i can say that i’ve made test similar to your first test about a copy on a non-oracle file on ASM with XDB FTP with 10g and i’ve got same result. Oracle check the header of the file and allows only copy of Oracle files on ASM.
My question is: your database was MOUNTED?, OPENED? On my test XDB FTP works only with database OPEN and i’ve tested with backup files o archived log files. I think that it does not work with CONTROL FILE because it is opened and locked by the instance. With datafiles may be that ORacle put them automatically on backup mode?
Regards,
Cristian
April 9th, 2008 at 3:30 am
I have tested cp using a DG with 1MB AU size.
April 9th, 2008 at 7:53 am
Christian,
My DB was but I was working on the backup copy (except one last case - was just too lazy). XDB has to have database open because it implements FTP protocol inside the database (if I can put it this way… Marco will correct me if I’m wrong).
I’m pretty sure that Oracle doesn’t automagically put datafiles into backup mode. It should have nothing to do with ASM copy.
I tested it with database instance down and cp works for datafile only still. It also works for datafile when DB is up:
spfile seems to be copied fine as well:
Let’s test with md5 checksum:
So cp definitely doesn’t work for controlfiles. It seems to work for datafiles and spfiles at least. However, my trust to the ASMCMD cp command is hurt.
April 10th, 2008 at 3:09 pm
I know that this feature is important and it seems that others have also experienced the same behavior (so it isn’t something “wrong” with what you did). I’m hoping to get to test this myself as well. In the meantime, I’ve confirmed that ASM development is investigating and hopefully we’ll hear something from them (here) soon.
April 10th, 2008 at 4:33 pm
Thanks Dan.
April 11th, 2008 at 11:44 am
[…] into Oracle stuff, our Alex Gorbachev also pointed out something that doesn’t quite work: the ASMCMD cp command in ASM 11g. He sure gives it a try, but finally concludes: “I couldn’t make the cp command […]
July 2nd, 2008 at 6:52 pm
ASM only support native Oracle Files: Datafiles, Online Logs, Control Files, RMAN backups, block change tracking files, exports, Flashback Logs, Archive Logs, etc. ASM does not support other types of files (Oracle makes a header verification. If the files does not match with an Oracle File, the copy process doesn’t work). Sorry, my english.
July 3rd, 2008 at 10:31 am
Axel, thanks for emphasizing that. I noticed it - It seems I can’t put any file to ASM. It’s probably documented somewhere. The rest of the blog is focused on supported file types.
July 3rd, 2008 at 8:34 pm
What Types of Files Does ASM Support?
This chapter describes how to administer files when you use the Automatic Storage Management (ASM) storage option…
http://download.oracle.com/docs/cd/B28359_01/server.111/b31107/asmfiles.htm
Regards,
Axel.