MySQL Conference 2012 – keynotes on day 2 (2)

Posted in: Technical Track

Mark Callaghan of Facebook: “What Comes Next for MySQL”

  • focus on large, sharded deployments
  • interesting numbers from their deployment (MySQL with Innodb): 60M QPS and 1.5B rows read/second in production

MySQL with InnoDB is “web scale”

  • scaled to 10x more data on the same servers:
    • Start with MySQL 5.1, flashcache, find and fix stalls, use multi-threaded purge from Percona, ask the db-ops team to deploy a lot of changes, use OSC (Online Schema Change) to add many covering indexes, use Faker from Percona+Facebook to fix replication lag, and make InnoDB compression good for OLTP.

“MySQL has made amazing progress”

  • InnoDB multi-core performance is impressive. (Yes, it’s finally overcome that early limitation!)
  • Replication is robust (global transaction IDs, multi-threaded, crash-proof, group-commits).
  • But a lot of work remains (of course).

And now, on to the future of databases in general. They brought up some very valid points around Flash vs. HDDs and how databases need to work differently for them (minimize iops vs. minimize size). Update in place vs. Write-optimized:

  • Backups account of 50% of their IOPS when run – this needs work (for sure).
  • Innodb Fragmentation causes lot of space usage vs. AltDB (write-optimized store).
  • Pooling is an issue – 50% of i/o capacity is used on average, but too many outliers are out of capacity. Solutions are hard (Reshard, migrate shards, deploy a SAN).

There is consistency between shards within a region, between master and slave for the same shard. The problem isn’t unique to MySQL, but solutions are hard and current solutions are too constrained.

MySQL still cannot max out modern hardware.

And more specific issues as related to Facebook and the Facebook architecture.

Interested in working with Raj? Schedule a tech call.

About the Author

Raj has been managing and guiding highly effective teams for over 20 years. Raj combines strong leadership skills with a deeply technical background in IT consulting to bring tailored and efficient technology solutions to Pythian. Additionally, his focus on data security has helped Pythian meet and often exceed client and regulatory requirements. Prior to joining Pythian in 2003, Raj held various leadership positions with a proven record of success. He has a degree in Industrial Engineering and has worked in multiple IT and engineering fields.

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *