MySQL replication

On of the less fun tasks I had at Feedster was dealing with MySQL replication. This post by Kristian Köhntopp highlights some of the issues we ran into.

The main issue we had to deal with was that the slave would always be running behind the master because the log was being applied by the single thread rather than multiple threads. The YouTube foks ran into that and dealt with it by writing a small hack which pre-fetched (thus caching) affected rows before they were updated by the replication thread, which I think is a very neat way of handling this issue. The way I dealt with this was to write my own replication system outside MySQL. If you don’t mind forgoing referential integrity, you can peel apart the binary log and set up multiple replication threads, each thread handling a table. This was a real hack and the clean way to do this would have been to do this inside MySQL, but I just did not have the time to delve into that. Besides which MySQL support would always balk when we would run our own MySQL build rather than the official ones, so we just ran the official build to remove the “we can’t help you because you are running your own (buggy) build” answer we would invariably get.

The second issue we had to deal with was integrity. Even MySQL’s official replication would mess up from time to time and we would need to sync the data.

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: