A couple of points. Not pointing out compression available in the new InnoDB plug-in was a serious oversight on my part. I am going to say I was having a senior moment. Shame on me!!

In retrospect, the idea of comparing Falcon and InnoDB was probably a mistake. My original idea was to have a post about each of the more well known transactional engines pointing out their features/benefits and disadvantages. I am not sure at what point this post turned into Falcon vs InnoDB.

Thanks everyone for the able defense(s) of InnoDB. InnoDB does have a long track record. And there is something to be said for that. I don’t think people will be rushing out to “ALTER TABLE” all their databases after they do get around to upgrading to 6.0. As a database administrator I would not recommend that anyone do that.

I might be optimistic, but I do think that Falcon is going to have a great run and InnoDB is going to be around as a very viable option for a long long time. As I said, competition brings out the best. So, here’s to competition!!

1) start transaction
2) do a bunch of updates, but don’t lock anything, because that wouldn’t be True MVCC
3) commit — whoops, someone else updated some of that data and committed before I did. Roll back.

I think a lot of the bullet points and phrases you copied from the documentation are only design goals at this point, and have never been substantiated. What would be interesting is to examine Falcon’s true behavior and see if they can be substantiated and really explained. Right now they just sound like “Falcon will magically do everything right.” But even if they’re true, I don’t see the technical explanations of how this is supposed to happen and what the consequences are. The “True MVCC” is just one example of that.

And do these things really work in the real world? That’s another issue. For example — the row cache. It’s supposed to increase cache efficiency, but the people I know who have benchmarked have had to tune it to the point of essentially disabling it to get better performance. In other words, making it behave more like InnoDB.

The architecture of InnoDB is incredibly sound, and has been proven through years of experience at 1000s and 1000s of customer sites. It will naturally take years for Falcon to prove itself in this way. No doubt we will see more comparisons like this as Falcon (and MySQL 6.0) approach GA. But it is important that such comparisons be complete.

Others have blogged in-depth on some other aspects of the Falcon architecture and differences between Falcon and InnoDB (see for example the article by PEter Zaitsev: http://www.mysqlperformanceblog.com/2007/01/12/falcon-storage-engine-design-review/).

So, here I’ll just briefly mention a few important things you didn’t mention (in addition to the fact that InnoDB supports both statement-based and row-based replication, both of which have their place).

InnoDB supports Referential Integrity; Falcon won’t do so until such support is implemented “above” the storage engine in a future release of MySQL.

InnoDB uses “clustered indexes”, where the data is stored in the leaves of the (primary key or clustered) B-Tree index. (This may be what you are alluding to when you say that the data is stored on disk in sorted order, though this is not quite correct, as the index pages themselves may be stored “randomly” on disk.) This data structure supports an important optimization called a “covering index”. InnoDB can often retrieve all of the required data in a single i/o request (if it is not already in memory), since the non-key column values follow the value of the primary key in the clustered index. The novel Falcon “row cache” may or may not hold the required data in memory, thus potentially requiring an extra i/o after the index lookup is performed.

The InnoDB architecture is very closely modeled on the foundational work of Jim Gray and Andreas Reuters, as described in “Transaction Processing: Concepts and Techniques”. Heikki Tuuri added some ideas taken from systems like Oracle (”undo or rollback segments” stored in the database) and created some additional very clever optimizations like the adaptive hash index and the insert buffer, which (as far as I understand) have no analogs in Falcon.

Regarding compression, the recently released InnoDB Plugin (see the website: http://www.innodb.com/innodb_plugin/) supports very effective data compression (2:1 or better), and it uniquely compresses both data and indexes. Some early users of compression in the InnoDB Plugin have reported outstanding results: significantly smaller databases and reduced i/o. The InnoDB Plugin introduces several other significant new features, and it works in the context of MySQL 5.1.

As you mention, the multi-core scalability issue may be (close to) being fixed. In general, scalability limitations in well-designed software are often comparable to peeling an onion. Remove one limitation, and another surfaces just below. It’s really not an exaggeration to say that (many of) the limitations of InnoDB’s multi-core scalability are “bugs” or quite simple changes in a few low-level primitives for synchronization, as Mark Callaghan has demonstrated. This is not to say that removing these limitations is simple or easy. Achieving superior scalability is the trickiest part of any system and it is easy to introduce subtle bugs when you are working at that layer of the code.

Hope this additional information is useful.