Swaping and filesystem caching

Kevin Burton discovered that Linux can sometimes swap out processes if it needs to cache the filesystem.

The premise is quite simple, a system where all the processes fit comfortably in memory should not need to swap, and yet he shows that it does sometimes swap to accomodate filesystem caching.

In his post, it is not clear which process is being swapped out. Indeed if there is a process that has been doing nothing for a while, like a daemon, then it should be swapped out to make room for other things like the filesystem cache if it makes sense. What you don’t want to have happen is for the operating system to swap out active processes, only to swap them back in when they are needed. As an aside, this was a bug in Solaris 2.5 (or 2.6, I can’t recall exactly as this was over 10 years ago), the operating system would swap out active processes when there was high demand for filesystem caching, so if you had a large process reading lots of file data, there was a good chance it would get swapped out, causing your machine to trash. The fix was pretty simple really, introduce a filesystem cache high water mark, i.e. a maximum amount of RAM that could be used for filesystem caching.

Kevin tried using O_DIRECT but that it seems to work for reads and not writes.

I checked this pages on Tuning Linux for Oracle and while it was very informative, there did not seem to be any way to cap the amount of physical memory allocated to the filesystem cache, though the document is packed with very good information. If you are really interested in the linux virtual memory design, you should read“Understanding Virtual Memory”.

I would suggest trying two things at this point. First I would reduce the amount of memory allocated to InnoDB to see if this reduces swapping, checking that it does not reduce MySQL performance. And then I would just run the machine without swap enabled.

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: