Scaling up vs. scaling out

Interesting post on scaling up vs. scaling out from Jay Pipes.

Worth reading if you plan to build the next killer site using MySQL.

There in an interesting comment in the post:

Anyone who wants to design their database architecture so that it’ll allow them to inexpensively grow from one box rank nothing to the top ten or hundred sites on the net should start out by designing it to handle slightly out of date data from replication slaves, know how to load balance to slaves for all read queries and if at all possible to design it so that chunks of data (batches of users, accounts, whatever) can go on different servers. You can do this from day one using virtualisation, proving the architecture when you’re small. It’s a LOT easier than doing it while load is doubling every few months!

I used to think that using replication slaves as read servers to take the load off the master server was a good idea. I am not so sure now given past experience. The issue is that replication slaves will fall behind if there is any kind of load on them, which will happen because they have to support both the same write load as the master and very likely a great read load too. Things are further stacked against you because slaves are usually underpowered compared to the master (who wants to use their best machines for slaves, right?) and the replication write thread runs as a single thread, whereas there are multiple write threads on the master.

I now think it is much better to use the replication slaves as hot backups for the master should the master fail (using similar machines for the master and the slaves), and shard your data to scale out.

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: