A week with Hadoop

I have been digging into Hadoop over the past week. For those who don’t know, it is a java based distributed storage and compute cluster.

I originally set it up to encompass both a Centos based PC and a Mac running Mac OS 10.4.10, it took a while to set up, there were a few non-obvious things that were not adequately covered in the documentation, but things worked well once it was set up. I did run into a few problems with the the compute cluster part on the Mac, for some reason the reduce process in the WordCount example would get stuck in a loop.

Anyway, I wrote my first Hadoop program earlier this week, which turned out to be very simple. I will post more about it as I delve into in more.


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: