Slice and Dice

I really enjoyed reading this post entitled “You Scaled Your What?” which talks about scaling, and stresses the point that there are multiple dimensions to scaling.

My rule of thumb with scaling is that if you can’t slice and dice it, it just won’t scale. What I mean by this is that if you cant slice your data, spreading it across servers and/or partitioning it, your system just won’t scale.

Slicing and dicing brings transactional scalability because you now have many more machines to process your data and to respond to searches. Of course this is dependent on the data actually being sliceable, but I would contend that very few real world data sets cant be sliced in one way or another, with lots of data sets being embarrassingly sliceable.

Slicing and dicing brings redundancy because you have mirrors hosting the same data, enough said.

Slicing and dicing allows you to deploy additional machines relatively easily and ramp up your processing capacity. This assumes that you made deployment easy, you did make deployment easy, right? In my experience, it is usually much simpler for operations to deploy a large number of smaller machines and great big machines which require lots of lead time to set up and stabilize. Operations also benefits because they have simple, cookie cutter machines to deploy and maintain as opposed to big monoliths where all hell usually breaks loose if something goes wrong.

What slicing and dicing does not speak to is productivity and feature ttm, but I would contend that those stem from good architecture and development process, which are hard to achieve but with big payoffs.


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: