“Disk is the new RAM”, or is it the other way around

By way of Greg Linden, I just finished watching a tech talk given by Professor Gene Cooperman on “Disk-Based Parallel Computation, Rubik’s Cube, and Checkpointing“.

The part that interested me was Cooperman’s assertion that “disk is the new RAM” if you have enough machines in your cluster, reaching a point where the aggregate bandwidth of the disks reach that of RAM.

The first obvious thing that jumps is that this does not make any sense, while the bandwidth may be the same, the latency is quite different, and will be a performance killer.

Cooperman recognizes that and makes the point that you need to organize your disk accesses to minimize latency, basically avoiding piecemeal reading and going for batch reading. Effectively what you are doing is shuttling data to and from memory in very large batches. (As an aside this is nothing new, the Connection Machine 5 had a similar disk system and I am sure there are other such systems out there).

Apple’s Time Capsule

I was happy to read the review of Apple’s Time Capsule by Robert Mohns for Macintouch.

What really interested me was not so much Time Capsule but the fact that disk I/O was very much improved on the original Apple Extreme Base Station. I had run some disk I/O benchmarks on it and was pretty slow.

ps – Macintouch is a must read for anyone who owns a Mac, it is updated daily, there is a wealth of review and help information as well as

Mobile Monday and the iPhone SDK

I attended the Mobile Monday iPhone SDK Party on monday 24th. The event was fun but packed. I am not sure the Apple folks knew what they were letting themselves in for when they allowed this event to be held in their store.

The Mobile Monday web site has pointers to various recaps of the event along with pictures.

A good friend pointed me to an article about the event on “the Onda by Antonio Rodriguez“, one thing in the article jumped out at me:

…it turns out that the using the radio to send/receive IP packets is relatively cheap from a power perspective, especially when compared to running the CPU at Safari-induced speeds or even keeping the display on for prolonged periods of time.

My experience with power consumption on the iPhone point to a few things which are power hungry. Bluetooth should go to the to of the list, for some reason leaving that on and having a headset paired with it really suck the electrons out of the battery. Second on that list should go Safari and the YouTube application, both are CPU intensive. And third on that list is Wifi which seems to be always on even when the phone is sleeping.

ps – the first presenter at the event was someone called Jonathan A. Zdziarski (also known as “NerveGas”), who recently published an article entitled “The iPhone SDK: APIs Apple Didn’t Want You to Know About“.

Rome API

Recently I have been working on parsing RSS and ATOM feeds, as part of a larger project, and found ROME to be a pretty good toolkit to do that.

JavaWorld has a good article on how to use ROME.

ROME does a good job of hiding the little idiosyncrasies that differentiate different feed types and it also provides a mechanism to access foreign markup in the feeds like the Media RSS elements.

Arthur C. Clarke dies at 90

I was very sorry to read that Arthur C. Clarke died today. The NY Times has a very good account of his life.

I have read all his novels, some more than once. I also remember coming across geostationary orbits (which he discovered) when I was studying physics at school and being surprised at their implications after I worked out the maths for myself.

Smart Sleeping

I have been looking for a utility like this for a while.

Mac laptops have a strange way of sleeping. Unless the battery is close to being completely depleted they will just go to sleep as opposed to hibernate (by hibernate I mean that the contents of the RAM are saved to disk and the machine is actually shut down.) While this may see like a good idea, it does not work well if you want to change the battery without having access to power as the Mac does not go into hibernate mode in a predictable fashion.

So what I have been looking for is a way to make my Mac laptop go into hibernation predictably and I think I may have found it. SmartSleep is a control panel which allows you to control if and how you Mac laptop goes to sleep and/or into hibernation.

State of Perl

Tim Bunce has posted an interesting presentation on Perl 5 and Perl myths.

I have to confess that I have a special affinity for Perl as it was the first ‘scripting’ language I learned, and I have used it extensively over the past 12 years. For example all the index administration code at Feedster was written in Perl, along with lots of parsing and utility libraries, and I always turn to it for all my ‘quick and dirty’ scripts, especially any text processing scripts.

File systems

ArsTechnica published an interesting article about the history of file systems which is a fun and interesting read.

Some interesting anecdotes I can add to this article:

A long time ago I worked in tech support for a company selling PC clones, I think we were delivering DOS 3.x at the time. I came across an interesting feature of FAT-32, which was that while you were limited to 512 files per directory the OS did not complain if you exceeded that limit. It would just extend the table into adjoining sectors and eventually either files would be corrupted if the table got too long, or files would be lost if the end of the table was overwritten by a new file.

More recently I tried JFS as an alternative to EXT3 at Feedster and was bitten by a bug of some sort which caused the file system to stop responding for 5 seconds at periodic intervals. This suspension was accompanied by a spike in CPU usage. Odd stuff, and I did not have time (or the inclination) to look for the cause so we just ditched it in favor of EXT3. We did look at XFS briefly but felt that there was too much risk involved since it would lose metadata is there was a crash or a power failure (we were using Gentoo at the time which did not help.)

Evolution of Apple design

This is worth a quick gander, a poster of the evolution of Apple design over the past 30 years.

Apple design process

Some interesting insight on the Apple design process outlined in a presentation at SXSW by Michael Lopp (by way of Apple Insider).

What is really interesting is this part of the process:

Paired Design Meetings
This was really interesting. Every week, the teams have two meetings. One in which to brainstorm, to forget about constraints and think freely. As Lopp put it: to “go crazy”. Then they also hold a production meeting, an entirely separate but equally regular meeting which is the other’s antithesis. Here, the designers and engineers are required to nail everything down, to work out how this crazy idea might actually work. This process and organization continues throughout the development of any app, though of course the balance shifts as the app progresses. But keeping an option for creative thought even at a late stage is really smart.

Where you open up people’s thinking (going crazy) and let their imaginations run wild, and then focus on what can actually be delivered. It is hard to see each process not feeding into the other.


Get every new post delivered to your Inbox.