I just came across this updated post on scaling by Greg Linden.
First he has a reference to an old article “Don’t Scale”. The article makes the (flawed) argument that scaling does not matter. It also confuses uptime and scaling, the two are very different.
There is a hardware aspect and a software aspect to uptime. The hardware aspect relates to the redundancy in your hardware architecture and the software aspect relates to the redundancy in your software architecture. Getting perfect uptime is really hard, especially when many things out of your control can go wrong. Getting all those nines after the decimal point eventually mean that you can only be down for minutes per years, clearly difficult to achieve.
Scaling has to do with how your system responds to increased usage, again there is a hardware aspect and a software aspect to this. A system will scale well from a hardware aspect if all you need to do is bring more boxes online when usage increases. Similarly, a system will scale well from a software aspect if you don’t need to go through wrenching software changes as your usage increases. Scaling hardware is relatively easy to achieve because the bandwidth and latency of hardware is well understood and easy to measure. Scaling software is more difficult to achieve since it can be difficult to figure out ahead of time where bottlenecks will occur and software behavior is more difficult to predict than hardware behavior. This article by Dan Pritchett is a good primer in scaling software.
Secondly he made two comments on the article:
Stepping back for a second, a toned down version of David’s argument is clearly correct. A company should focus on users first and infrastructure second. The architecture, the software, the hardware cluster, these are just tools. They serve a purpose, to help users, and have little value on their own.
But this extreme argument that scaling and performance don’t matter is clearly wrong. People don’t like to wait and they don’t like outages. Getting what people need quickly and reliably is an important part of the user experience. Scaling does matter.
I agree that you initially need to focus on getting the application working and building up your user base, but scaling and performance really does matter and you need to think hard about those issues from the start. In my experience it is usually much easier to scale a system which was conceived with scaling in mind than a system which was just thrown together without real thought. I’ve been there, trust me, nothing makes your heart sink faster than code that has not been properly tiered, or a crappy data model, or a schema that was just thrown together, or seeing IP addresses in the code.
If you don’t build with scaling in mind from the start, you will find that you will have to throw everything away much sooner than you thought.
Greg then follows up with an except from an interview with a Twitter developer Alex Payne who says:
Twitter is the biggest Rails site on the net right now. Running on Rails has forced us to deal with scaling issues – issues that any growing site eventually contends with – far sooner than I think we would on another framework.
The common wisdom in the Rails community at this time is that scaling Rails is a matter of cost: just throw more CPUs at it. The problem is that more instances of Rails (running as part of a Mongrel cluster, in our case) means more requests to your database. At this point in time there’s no facility in Rails to talk to more than one database at a time.
The solutions to this are caching the hell out of everything and setting up multiple read-only slave databases, neither of which are quick fixes to implement. So it’s not just cost, it’s time, and time is that much more precious when people can[‘t] reach your site.
None of these scaling approaches are as fun and easy as developing for Rails. All the convenience methods and syntactical sugar that makes Rails such a pleasure for coders ends up being absolutely punishing, performance-wise. Once you hit a certain threshold of traffic, either you need to strip out all the costly neat stuff that Rails does for you (RJS, ActiveRecord, ActiveSupport, etc.) or move the slow parts of your application out of Rails, or both.
I am no expert in Ruby or Ruby on Rail, but a few things jumped out at me in this section, and this moves away from the higher level concepts and more into implementation issues.
The first was the comment about caching. Cache is king so the (corrupted) saying goes, and the more data you can cache, the less frequently you will have to go back the source for data. Data can be cached on the local file system, either in files or in a small database (Berkeleydb or SQLite being good options.) A good place for those on Linux is to use the shared memory partition (/dev/shm) which functions much like a RAM disc. Data there will usually be stored in RAM but will be swapped out if there is a memory shortage, so is only good for small amounts of data. Data on the shared memory partition will be wiped if the machine is rebooted.
There are also memory based caching systems, such as memcached, which allows for caching either on the local machine or a dedicated machine (or a batch of dedicated machines). I have used memcached before and the best configuration is to set up a batch of machines dedicated to caching, preferably with lots of RAM in each machine. The two caveats is that you much make sure that the memcached process never, ever, gets swapped out otherwise your cache performance will crash, and never rely on memcached to keep an object in cache indefinitely, objects will get aged out from the cache if space is needed for younger objects.
The second was about using multiple read-only slave databases. I used to a fan of that until Peter Zaitsev pointed out to me that this really does not work well in an environment where there is a lot of write activity because the slaves will have support both the write load and the read load, while the master will only have to support the write load (assuming you don’t want anyone reading from the master unless they have to.) This gets nastier because the temptation is make the “baddest” machine the master and to use less powerful machines for the slaves. This gets even nastier if you use MySQL (very likely) because there is only a single replication thread running on the slave which impacts the write performance. Once your slaves start to fall behind, they will never catch up.
What I am a fan of now is to dice data so that it can be spread across multiple machines. You can also slice your data into partitions too to make your individual databases and indices smaller, which will help performance. The impact of this is what you will need to move some amount of integrity checking out of your database (like triggers and constraints) into a higher layer. This is not all bad as it means that your database will be doing less work. Of course how this slices and dices depends on your application. This presentation on how eBay scales provides a very good illustration on how to do this.