ZFS – VM for Storage?

Jun 8, 2006 / By Christo Kutrovsky

Tags: ,

James Foronda sent a link to his blog entry about silent data corruption in an Oracle-l discussion. He points out a new file system called ZFS that has self-healing features. I really liked this quote

“Does for storage what VM did for memory”

I read all the slides and it really looks like something cool to try out. It has some features similar to Oracle’s ASM, such as dynamic adding of disks, self balancing etc.

It’s fully transactional, supports snapshots, online everything. Auto striping (not sure how this would work). Very feature rich.

Also extremely simple (from looking at the demo1 and demo2.)

I am really excited. Finally a file system that offers something really special.

I am looking forward to try it out. Currently for SUN only. I wonder when the linux port will show up.

P.S.
I think it’s called “zfs” (with z – the last letter in the alphabet) because it’s “THE LAST WORD IN FILE SYSTEMS”.

16 Responses to “ZFS – VM for Storage?”

  • Doug Burns says:

    zfs was one of the main reasons I went for the Solaris 10 installation on my ISP4400 – http://oracledoug.blogspot.com/2006/03/solaris-10-and-oracle-10g.html. Sadly, I’m still waiting for Oracle release the software to run on it (but that’s because I’m on Linux x86). The thing to note with zfs is that, the last time I checked, you need the Solaris Express edition which is a few steps ahead of the production version in the release schedule.

    I think the other big step forward in Solaris 10 is dtrace.

    Both make it well worth a try.

  • Christo Kutrovsky says:

    I wonder if Oracle will rush to imrove ASM, given ZFS. And I also wonder if ZFS will be (or already is?) cluster compatible?

    Which one would you choose ?
    - The Oracle provided ASM
    - The all new ZFS file system

    They both offer a lot of flexibility, except it seems that ZFS is far more tested in failure scenarios.

  • Doug Burns says:

    Christo,

    First, I meant to say Intel x86 before, not Linux

    I have one sensible reason for using something like ZFS rather than ASM

    1) Even on a database server, not all files are going to be controlled by Oracle. If you’re going to use a technology to perform intelligent management of data location, you might as well pick one that’s not limited to Oracle.

    and a few more fuzzy, opinionated suggestions :-

    2) In large corporations, it’s not usually the DBA team who make the decisions about storage technology. This seems trivial, but is actually key. I suppose we can just request raw volumes be presented for ASM use, but the reality is that system and storage administrators will have often decide this long before we’re involved. We don’t like in an Oracle-centric world (sadly ;-))

    3) Instinctively (with all the possible negatives that word implies) I would rather leave management of this to someone with a bit more experience in filesystem development. That doesn’t necessarily mean Sun, either, but I’m sure it’s not Oracle.

    I’m not anti-ASM at all, I just feel that these things are best implemented at a lower level and not specific to one application (which ASM is, albeit a very generic application)

  • Howard J. Rogers says:

    I couldn’t disagree with Doug more on this one!

    ‘Not all files are going to be controlled by Oracle, so you might as well pick a [storage] technology that’s not limited to Oracle’.Part one of that statement is true, but part 2 is a non sequiteur. Not all transport is controlled by airlines, so you might as well pretend that airplanes don’t exist?!

    His point 2 is misguided too. If it is the case that in large organisations it is not the DBA team that decides about DATABASE storage technology, then something is wrong, and bad habits need to change. I’ve always said that resistance to ASM is largely a matter of office politics, so I recognise where Doug is coming from… but I don’t agree that’s a convincing case for then just giving in to the situation. It is rather occasion to call to management to change the situation.

    Point 3 is superficially plausible: I never trusted OCFS because of who it came from, either. But OCFS is in the Linux kernel these days and that’s got to count for something. Similarly, ASM is, no doubt, an evolving technology and can only get better -as OCFS has morphed into OCFS2. It is, in short, an argument for perhaps being wary of ASM *now*, but not about ASM in general or in principle. As for ‘someone with a bit more experience in filesystem development’… how about EMC? They worked very closely with Oracle to make ASM happen, and I think they know a thing or two about storage!

  • Christo Kutrovsky says:

    Personally, I like ASM because of it’s simplicity. I dont even consider it a file system, but more like a layer to connect multiple raw devices with the database.

    As far as experience and filesystems, the moment you change the category of ASM from filesystem, to raw device manager this argument becomes un-needed.

    Maybe the fact that Oracle released ASM will help DBA-s to regain control over the storage of a database.

  • Doug Burns says:

    That’s funny – I *almost* dropped Howard an email to say ‘look, I’ve commented on ASM in a blog, so you might want to follow up to provide a bit of balance’. I should have known he’d turn up here – he has an ASM radar.

    To clarify what I mean in points 1) and 2) above …

    My experience is that large companies want to standardise on as few software components as they can get away with – to reduce support and licensing costs and make it easier for managers to understand ;-). If there is an storage software option that covers easy provisioning, intelligent data relocation etc. that will work for non-Oracle servers as well as the database servers, that’s what they’ll pick unless there’s a particularly good reason to add a second product (ASM). So if ZFS can give us even *most* of the functionality that ASM does, they’ll go with ZFS because they can use it across the server estate.

    As for control over storage management being out of DBA’s hands, I agree that’s a deeply unsatisfactory state of affairs and needs to be challenged. However, challenging it (again, in my personal experience) has been a pretty thankless task with disappointing results. Sadly, technology is often not the driver in IT any more :-( Of course, as DBAs become more distanced from areas of the technology stack that are not ‘our responsibility’, it de-skills some of us and adds weight to the argument of those who would keep us out of the loop. There was a time when I could claim to understand most areas of Unix sys admin, but that’s dropped over the years as everything gets more complicated and there are dedicated resources to take care of such things. Sad but often (not always) true.

    As for point 3) and not necessarily trusting ASM, I should be clear that I wouldn’t necessarily trust ZFS at this stage, either. All technologies take time to bed in and I agree that ASM might be the future – if the management issues don’t get in the way.

  • Steve says:

    As ZFS never writes a block to the same physical location it read it from (a feature to aid in data recovery following media corruption), surely you’d need to be declared clinically insane, before running Oracle with ZFS? The fragmentation would be horrendous, surely?

  • Christo Kutrovsky says:

    Agreed. But I am sure they will have some kind of solution for this, as it’s not just Oracle that will have problems with this.

    I am definetelly not considering ZFS for Oracle at this point. The same way we allocate a separate memory pool (hugepages) for Oracle’s SGA in linux VM, the same way you do not use ZFS for Oracle’s datafiles. It just doesnt make sense, Oracle already has most of this functionality integrated.

  • James Foronda says:

    Doug,

    > Sadly, I’m still waiting for Oracle release the software to run on it
    > (but that’s because I’m on Linux x86)

    > First, I meant to say Intel x86 before, not Linux

    Can you please elaborate on what you mean by this? Do you mean Solaris x86? If yes, then there is an Oracle version for Solaris x86.

    > The thing to note with zfs is that, the last time I checked,
    > you need the Solaris Express edition which is a few steps
    > ahead of the production version in the release schedule.

    That’s correct at the time Doug wrote that. You could also do it with OpenSolaris. Please note, though, that Solaris10 update 2 (S10U2) has been officially released a few weeks ago. S10U2 has ZFS built-in and is officially supported.

    And yes, DTrace is *very* cool. :)

    Christo,

    Thanks for linking to my blog. I think your post is driving some traffic to my blogger.com account which is in turn redirected to my new blog at http://jforonda.vintarinian.com. It would have been cool if I can configure blogger to redirect to specific post but I don’t know of an easy way to do that.

    So… if it is not too much trouble for you, can you please change the link above from:

    http://jforonda.blogspot.com/2006/06/silent-data-corruption.html

    to:

    http://jforonda.vintarinian.com/?p=4

    I hope you don’t mind me putting this request in the comment area of your post. I wanted to email you personally about this but I could not find your email address.

    Thanks.

  • Paul Vallee says:

    Hi James,

    Thanks for your comment; I updated the URL. Cheers,

    Paul

  • Doug Burns says:

    James,

    “Can you please elaborate on what you mean by this? Do you mean Solaris x86? If yes, then there is an Oracle version for Solaris x86.”

    When I wanted to install Oracle 10.2 on Solaris 10 x86 on my old 32-bit server, the Oracle part wasn’t available in a 32-bit version. I’ve just had a look again and there still isn’t a 32-bit version. Of course, I could user -ignoresysprereqs (hope I’ve remembered that right) to by pass version checking issues, but it would be nice to use a properly supported platform.

    Thanks for the OpenSolaris mention, too. My sys admin mate at work had mentioned that to me and I’ve been planning to look into it. I’m about to start a new paper and I’m hoping it’ll be an opportunity to mess around with DTrace, which really floats my boat ;-)

  • James Foronda says:

    Doug,

    My laptop is a Toshiba Tecra M2 with Intel Centrino running 32-bit Solaris x86 (some old Nevada build, when ZFS was just released to the OpenSolaris community). I dont remember exactly where I got the Oracle bits (probably OTN) but I have 32-bit Oracle 10gR1 running on it:

    ———————————
    o10R1/O10gR1@rapanui> sqlplus “/ as sysdba”

    SQL*Plus: Release 10.1.0.3.0 – Production on Fri Jul 28 21:57:06 2006

    Copyright (c) 1982, 2004, Oracle. All rights reserved.

    Connected to:
    Oracle Database 10g Enterprise Edition Release 10.1.0.3.0 – Production
    With the Partitioning, OLAP and Data Mining options

    sys@O10GR1>
    ———————————

    IIRC, 10gR2 is when Oracle started supporting only the 64-bit version for Solaris.

    As for Solaris, it can run on both for 32-bit or 64-bit hardware.

    Since you are just about to try, I think going for Solaris 10 U2 is a good starting point. OpenSolaris is bleeding edge which is not necessarily bad, depending on what you are trying to do. :)

    James

  • Doug Burns says:

    Thanks James. Very much appreciated. Particularly as my 2 week’s vacation started today ;-)

    Then again, I was hoping to go with vmware too (on Howard’s recommendation) so it sounds like I’ll have plenty to get my teeth into!

  • outside the box says:

    ZFS – Another way to recover from a screw up… :)…

    Doug Burns wrote in his blog how he accidentally messed up some of his Oracle database development environments. Doug, I remember that a little while back, we had a short chat over at Christo Kutrovsky’s follow-up blog entry to my ZFS post.
    If y…

  • Paul Vallee says:

    Tip of the hat to Doug Burns again, http://oracledoug.com/serendipity/index.php?/archives/1102-10gR2-on-Solaris-10.html for this very interesting link detailing ZFS performance issues handling an OLTP workload: http://blogs.sun.com/roch/date/20060922

    Paul

Leave a Reply

  • (will not be published)

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>