Scaling MySpace: Treading Water in a Tsunami

Posted in: Technical Track

Those of you that that enjoyed my post on Stability, Agility, Scalability and Cost: The eBay example are sure to like this technical interview of the desperate efforts to stay only a bit behind the traffic demands at MySpace, written by David Carr at Baseline Magazine

Some take-aways:

“The thing is, it provides so much of a benefit to people that the errors and glitches we find are forgivable.” If the site is down or malfunctioning one day, he simply comes back the next and picks up where he left off, Tanner says.

Hmm, that sounds a tad different to the eBay database use case, no?!

Standard database server configuration consists of Hewlett-Packard HP 585 servers with 4 AMD Opteron dual-core, 64-bit processors with 64 gigabytes of memory (recently upgraded from 32). The operating system is Windows 2003, Service Pack 1; the database software is Microsoft SQL Server 2005, Service Pack 1. There’s a 10-gigabit-per-second Ethernet network card, plus two host bus adapters for storage area network communications. The infrastructure for the core user profiles application includes 65 of these database servers with a total capacity of more than 2 terabytes of memory, 520 processors and 130 gigabytes of network throughput. Source: user conference presentations

A worthy read for those of you interested in the lofty heights of database scaling, where we pack oxygen and have to make some choices and compromises along the way. And often being forced to discard niceties such as

“[…] business logic in database”

[…] ” – Referential integrity”
” – Joins”
” – Sorting”


… along the way! :-)

Disclaimer: Fox Interactive Media is a customer of Pythian’s, however we manage the IGN properties and Fox Sports environments only at this juncture. [Check out the Pythian attribution footers on the other IGN properties Rotten Tomatoes, Ask Men, etc.] As much as I’d love to have a crack at the MySpace database challenges, so far they have not been sent our way.

Interested in working with Paul? Schedule a tech call.

About the Author

As Pythian’s Chief Executive Officer, Paul leads this center of excellence for expert, outsourced technical services for companies whose systems are directly tied to revenue growth and business success. His passion and foresight for using data and technology to drive business success has helped Pythian become a high-growth global company with over 400 employees and offices in North America, Europe, and Asia. Paul, who started his career as a data scientist, founded Pythian when he was 25 years old. In addition to driving the business, Paul is a vocal proponent of diversity in the workplace, human rights, and economic empowerment. He supports his commitment through Pythian’s hiring and retention practices, his role as board member for the Basic Income Canada Network, and as a supporter of women in technology.

1 Comment. Leave new

Good post, Paul.


Leave a Reply

Your email address will not be published. Required fields are marked *