Unveiling the OLTP Oracle Database Machine & Exadata v2

Posted in: Technical Track

Update, July 9, 2010: Pythian has now announced our range of services for Oracle Exadata, along with successful implementations and reference customers.

Now that I, apparently successfully, predicted OLTP Database Machine on Sun hardware, I had to wake up before 6AM in Sydney to tune into Larry’s joined with Sun Microsystems webcast (just to learn that he is late, by the way – 8 minutes so far…). As the follow up post’s comments show, people are interested in the role of SPARC platform in the new OLTP Oracle Database Machine (turns our there is no role for SPARC as of now).

Waiting… Waiting… ah here it comes — yachts, BMW (yeah love it as well) and Larry walks in — he starts by mentioning lo-o-o-ong partnership with Sun and announced —
“Oracle Exadata version 2 – hardware by Sun and software by Oracle.” Funny, I heard exactly the same sitting at the Oracle Open World last year but with HP. He then proceeds — “It is the *First* Database Machine that does OLTP. All the other machines, Teradata, Netezza, etc. are designed just for data warehousing.”

Interesting that Larry’s speech was very harsh on competition and where it comes to data warehousing, it’s Netezza and Teradata, while in hardware it’s IBM. I need to count how many times Larry said “better/cheaper/faster than IBM” during his announcement.

Ellison also seems to freely use Exadata term for the Database Machine itself which is a bit confusing but the more confusion, the more hype and interest so I guess he must have been asked to do it.

Let’s go into the nitty-gritty… ODBM (Oracle Database Machine) v2 is twice as fast as version 1 when it comes to data warehousing. Compare to version 1, ODBM v2 runs OLTP traffic — “something that Netezza and Teradata can’t do at all… but we can do both [i.e. data warehousing] and we do both very well” said Ellison.

So what’s inside?

Database Servers

Sun Fire X4170 with two Xeon Nehalem processors and this is the state of the art. Nehalem processors are the fastest CPU’s for database workload in case you didn’t know. Each server also has 40 Gb connectivity to InfiniBand switch.

Memory is 3 times faster in the new database server and there more the two times memory now — 72GB per database server. For those of you who still use 5 GB for their SGA’s when the server has 128GB — wake up! The prohibitive overhead of larger SGA is long time gone and all modern releases of Oracle handle large SGA really well.

What’s important is that each server has 4 10Gbit Ethernet ports. OLTP traffic now has bigger pipes to connect application tiers which what was one the main problems why ODBM v1 wasn’t suitable for OLTP traffic.

Exadata Storage Servers

Storage Servers are based on Sun Fire X4275. Again top of the shelf Intel Xeon 5500 Nehalem processors.

Capacity increased — SAS disks are 600 GB (+33%) and SATA disks are 2 TB which doubles what ODBM v1 offered. We all know that big disks are bad but that’s where Sun FlashWire technology comes into play with flash cache on the controllers (5+TB for the full ODBM).

This is what Ellison said about Flash cache — “These are not Flash disks. Make no mistake — these are no Flash disks. This is a memory hierarchy made up of DRAM in our database servers and Flash in our storage servers with very sophisticated algorithms, not simple LRU — we know for doing a sequential scan, we don’t blow out the cache.”

Database Machine — all together

8 Compute Servers — note that they are not Database Servers anymore. That means, customers are expected to be able to run their application and web tiers on those compute servers just as well as Oracle databases. Of course, it’s called the server grid. You have in total 64 Nehalem cores and 400 GB of memory.

14 Storage Servers feature 300+TB of disks (using SATA disks) and more than 5TB of flash cache. The flash cache is built into the disk controllers. Each disk controller has 96 GB of flash. Each Exadata Storage server has 4 of those controllers so you do math (we must be at least as good as Larry in math — “1 million IO’s per second (IOPS) from a single cabinet. Obviously, 2 cabinets is 2 million IOPS. 8 Cabinets is… I can do the multiplications in my head — it’ll be 8 million random IOPS.”). So it’s 5+ TB of disk controllers flash cache — Sun’s FlashWire technology.

Ellison mentioned 880 Gb/sec aggregate throughput for InfiniBand. In reality, 8 compute servers can only do 320 Gb/sec which is a lot and if you are going to use this machine for OLTP — you don’t care as all you interested is random IO’s and the machine is sized to deliver one million random IO’s.

It’s not clear whether customers have complete flexibility on configuration of the database machine but based on Larry’s scale-out ideas, it’s possible. He said — “just plug in another compute server or another storage server or just plug in another InfiniBand networking switch”. The latter can let you expand beyond one single database machine and rack together 8, 16, 32 machine and according to Ellison “you can just go on and on and on”. Petabyte era comes indeed, we are in the Petabyte territory with 3 database machine racks.

1 million random IO’s per second — this is what you can achieve if your whole database fits into Flash Cache. The bandwidth of Flash cache is only 2.5 times faster than disks (50GB/sec vs 21 GB/sec) but don’t forget that we need Flash Cache only for random IO’s. Don’t forget that this is random reads performance. Right now, it’s not clear how write IO’s impact the number.

Ellison also mentioned compression and then flash can fit something like 50TB of the raw data after compression (10x compression for data warehouse). It’s not free but the results are impressive and based on the current adoption, customers are very happy with the new compression that Oracle database 11g provides.

11gR2 with Oracle Database Machine is running in-memory-only faster than all new start-ups specialized on in-memory databases. This is an interesting claim — I’m keen to verify it. According to Ellison, “Exadata is faster than the best of the specialized in-memory-only database query processing systems”. Oracle ran TPC-H data warehousing benchmark for 1TB data in memory and… won, according to Ellison.

IBM

Building the similar system on IBM Power6 Model 595 and 8 DS8300 Turbo storage, will cost you US$10 million.

Two racks of database machine cost $2.3 million only which is less than a quarter of what you would pay for IBM hardware. Putting it another way around — you can spend the same dollars and get 4 times faster hardware. Interesting note from Ellison to this IBM set — “we are fault tolerant, they are not”. Hm… I was quite sure IBM hardware is damn reliable but OK, if Larry says so…

One slide said that Oracle Database Machine is 20 times faster than IBM fastest computer. Not sure how that was calculated?

Ellison also claimed that Massively Parallel Architecture is what Oracle has been pursuing over the last 20 years. Interesting that many keep blaming Oracle for focusing on shared-everything while other vendors using shared-nothing with massively parallel processing. Well, if nobody else can do shared-everything — why not to blame this technology and stamp it as non-scalable. Here is where Ellison mentioned IBM DB2 and SQL Server and that they, obviously, can’t do what Oracle database can.

Pricing

  • 1 Exadata Server + 1 Compute Server — $110K
  • Quarter Rack — $350K
  • Half Rack — $650K
  • Full Rack — $1.15M

Obviously, that doesn’t include licensing but you can take over you existing Oracle database licenses. Update 16-Sep-09: Beware of Exadata Software licensing – it’s $10K per disk! Official price list has been updated.

How do you migrate

There is no changes to application that need to be done to migrate to Oracle Database Machine — all your existing applications will just work. The migration time-frame can be very aggressive — up to days really.

And if you need help migrating your application to the new Oracle Database Machine — Pythian, of course, can help you in doing so: migration with minimal downtime, optimal layout and capacity planning, help you transfer your licenses, evaluate potential compression benefits, upgrade your database version up to the latest 11g Release 2 (we were part of the beta program as usual) and all the tasks involved including backups and business continuity as well as monitoring.

Learn more about Pythian’s services for evaluation, migration to and operational support for Oracle Exadata.

Interested in working with Alex? Schedule a tech call.

About the Author

What does it take to be chief technology officer at a company of technology experts? Experience. Imagination. Passion. Alex Gorbachev has all three. He’s played a key role in taking the company global, having set up Pythian’s Asia Pacific operations. Today, the CTO office is an incubator of new services and technologies – a mini-startup inside Pythian. Most recently, Alex built a Big Data Engineering services team and established a Data Science practice. Highly sought after for his deep expertise and interest in emerging trends, Alex routinely speaks at industry events as a member of the OakTable.

12 Comments. Leave new

AMIS Technology blog » Blog Archive » Oracle Database 11g Release 2: Flash Cache and Exadata V2
September 16, 2009 3:34 am

[…] Ellison’s launch event, but reading Alex Gorbatchev’s blog post on the Pythian web site Unveiling the OLTP Oracle Database Machine & Exadata v2 is enough explanation in the "why" and "how". Flash cache is one of the pillars […]

Reply

Hi Alex,

Just 1 question.

Are you sure about :
“11g Release 2 (we were part of the beta program as usual)” ?

If I recall Pythian Group was not part of 11GR2 beta testing.
Please correct me in case of an error on my part.

Thanks.

Waseem

Reply

@Waseem:

If I recall…

Can’t be more sure and it’s an error on your part. Not sure where you recall that from.

We haven’t been as extensively involved as the previous release and you probably noticed that our blog was quiet about that.

Reply

I remember thinking Pythian weren’t part of 11gR2 beta as well, so was wondering where I got that idea from. I think it might have been Christo’s post here, which seemed to imply you weren’t (at that time, anyway)

http://www.pythian.com/news/1246/oracles-secret-new-feature-educated-guesses

“Oracle has been keeping the new feature a secret. Even the 11gR2 beta program had very few participants to prevent information leaking out. It’s, “Something’s coming, but I am not telling what.””

At which point, Christo started to speculate, which I’m sure he wouldn’t have done if you were part of that beta?

But it was a long time ago, I just remember thinking the same. Maybe it was even memtioned in other posts?

Reply

@Doug: Yes sir. We’ve joined much later than that as most of the beta participants I think.

@Waseem: Now I see where you implied that from. You are reading our blog much more attentive than me it seems. :)

We also didn’t take part in on-site beta camps this time (for a number of boring reasons).

Reply
Oracle Database Machine on a Budget: Standard Edition (SE)? | Pythian Group Blog
September 17, 2009 8:33 am

[…] Recall that compute servers are dual socket Sun Fire X4170. This means that Oracle SE license could cover up to 2 nodes RAC (included in SE) with 4 sockets in total. For two compute servers in a quarter rack database machine, licensing Oracle SE costs just US$70,000. […]

Reply
Christo Kutrovsky
September 17, 2009 10:08 am

Ah … finally, Oracle found a way to get money from the size of your database.

I wonder when RAM will come into play ;D

I don’t know about you, but 10K a disk looks rather high to me.

That’s 120K per cell in addition to hardware cost. I wonder why they don’t just bundle it all up.

Just as storage was becoming cheaper and cheaper, suddenly there’s a big price tag attached to it. That is, if you use the database machine.

Reply

@Christo: yeah… I planned another post on $$ per disk.

Reply
Log Buffer #162: a Carnival of the Vanities for DBAs | Pythian Group Blog
September 18, 2009 1:09 pm

[…] big news this week came was Oracle’s unveiling the OLTP Oracle Database Machine & Exadata v2, as reported by Alex […]

Reply
Christo Kutrovsky
September 18, 2009 1:10 pm

How about $$ per record?

Or even better, $$ per online user. Given 10 000 per disk, and on avg 100 IOs per disk, that’s 100$ per PIO per Second.

You get the idea.

That got me thinking … that’s 10 000 for exadata software, per disk.

10 000 $ can buy you 20 additional disks. SO are Exadata’s IOs 20 times better? For OLTP that is…

Reply
Blogroll Report 11/09/2009 – 18/09/2009 « Coskan’s Approach to Oracle
September 19, 2009 2:02 pm

[…] Alex Gorbachev -Unveiling the OLTP Oracle Database Machine & Exadata v2 […]

Reply
Christo Kutrovsky
September 21, 2009 3:59 pm

I was reading:

http://structureddata.org/2008/10/24/oracle-exadata-in-response-to-chuck-hollis/

That changed my mind a little bit.

For the amount of political pressure, marketing pressure, opinion pressure, $10 000 per disk as expensive as it is, solve a lot of problems.

Balanced systems out of the box – think of how much money will be saved from endless meetings on choosing the “right” configuration and raid level …

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *