October 6, 2008 Leave a comment
Sometimes throwing out the database with the bathwater is not a bad thing:
We’re entering the age of cloud computing, remember? And clouds, it turns out, don’t like databases, at least not as they have traditionally been used.
This fact came out in my EmTech panel and all the experts onstage with me nodded sagely as my mind reeled. No database?
To scale the Google search service, then, they figured that many large problems did not intrinsically require doing actions one at a time. But Google first had to free itself of the false dependencies. So they coined the term MapReduce and created both a set of operations and a way to store the data for those operations natively, all while preserving the natural independence that is inherent in each problem, building the whole mess atop the remarkable Google File System, which I’ll cover some other day.
And more about MapReduce going mainstream.