Scaling MySQL

Natim Shalom talks about scaling MySQL.

He reviews replication and sharding as a means of scaling and dismisses both pretty quickly in favor of a system called In-memory data grids (IMDG). From what I can tell this is a piece of middleware that sits between the application and the database. He describes it thus:

In-memory data grids (IMDG) provide object-based database capabilities in memory, and support core database functionality, such as advanced indexing and querying, transactional semantics and locking. IMDGs also abstract data topology from application code. With this approach, the database is not completely eliminated, but put it in the *right* place. I refer to this model as Persistence as a Service (PaaS).

I am not sure I get the point of this. From what I can tell it is just another layer of abstraction between the application and the database, a layer which sits in memory, is distributed across nodes and optionally offers a different data model to the application. I wonder if this is better than Hibernate which seems to offer the same features.

I also think he dismissed sharding too hastily, a lot of his arguments can be countered fairly effectively. For example he argues that disk I/O is a problem with sharding, but I would argue that sharding is done to deal with disk I/O, assuming of course that you are sharding across machines (which also increases your aggregate memory pool). Resharding can take place without taking the application offline, admittedly this is complicated. As for changes to the data model and application code, my feeling is that most data models can be sharded (some better than others), and you can abstract the shard selection code relatively easily so the application does not ’see’ the shards.

Still I think this is a thought provoking article well worth reading.


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: