Mark Callaghan of Facebook: “What Comes Next for MySQL”
- focus on large, sharded deployments
- interesting numbers from their deployment (MySQL with Innodb): 60M QPS and 1.5B rows read/second in production
MySQL with InnoDB is “web scale”
- scaled to 10x more data on the same servers:
- Start with MySQL 5.1, flashcache, find and fix stalls, use multi-threaded purge from Percona, ask the db-ops team to deploy a lot of changes, use OSC (Online Schema Change) to add many covering indexes, use Faker from Percona+Facebook to fix replication lag, and make InnoDB compression good for OLTP.
“MySQL has made amazing progress”
- InnoDB multi-core performance is impressive. (Yes, it’s finally overcome that early limitation!)
- Replication is robust (global transaction IDs, multi-threaded, crash-proof, group-commits).
- But a lot of work remains (of course).
And now, on to the future of databases in general. They brought up some very valid points around Flash vs. HDDs and how databases need to work differently for them (minimize iops vs. minimize size). Update in place vs. Write-optimized:
- Backups account of 50% of their IOPS when run – this needs work (for sure).
- Innodb Fragmentation causes lot of space usage vs. AltDB (write-optimized store).
- Pooling is an issue – 50% of i/o capacity is used on average, but too many outliers are out of capacity. Solutions are hard (Reshard, migrate shards, deploy a SAN).
There is consistency between shards within a region, between master and slave for the same shard. The problem isn’t unique to MySQL, but solutions are hard and current solutions are too constrained.
MySQL still cannot max out modern hardware.
And more specific issues as related to Facebook and the Facebook architecture.
Interested in working with Raj? Schedule a tech call.