Large-scale Incremental Processing Using Distributed Transactions and Notifications

This is definitely worth reading:

Large-scale Incremental Processing Using Distributed Transactions and Notifications: “Updating an index of the web as documents are crawled requires continuously transforming a large repository of existing documents as new documents arrive. This task is one example of a class of data processing tasks that transform a large repository of data via small, independent mutations. These tasks lie in a gap between the capabilities of existing infrastructure. Databases do not meet the storage or throughput requirements of these tasks: Google’s indexing system stores tens of petabytes of data and processes billions of updates per day on thousands of machines. MapReduce and other batch-processing systems cannot process small updates individually as they rely on creating large batches for efficiency.

We have built Percolator, a system for incrementally processing updates to a large data set, and deployed it to create the Google web search index. By replacing a batch-based indexing system with an indexing system based on incremental processing using Percolator, we process the same number of documents per day, while reducing the average age of documents in Google search results by 50%. Links: [abstract] [pdf] [search]

(Via Recent Google Publications (Atom).)

I built an incremental indexer while at Feedster, albeit on a much smaller scale, we had a 10 minute turn around time for newly crawled stuff which wasn’t too shabby I think.



Three Little Things

Earlier this week I was listening to Paul Thurrott on Windows Weekly talking about Android and how it was overtaking the iPhone (note that I said iPhone and not iOS). He also mentioned it on his SuperSite Blog Daily Update.

You can’t really deny that Apple is at a competitive disadvantage here, the iPhone is on one carrier only while Android is on multiple devices and on multiple carriers.

So here is my (humble) list of three items Apple needs to address to stay competitive:

  • It’s ok to release new hardware once a year, but there should be a major software release every 6 months, something that brings new features in mid-cycle to keep the platform fresh.
  • It needs to open up to other carriers, notably Verizon, to expand the customer base.
  • Finally, iTunes needs to be fixed. And free the phone from having to rely on iTunes.

I doubt SteveO reads this blog (I would hope he has better things to do), but I would see these as a minimum.



iPod Nano

Well I was going to buy the new IPod Nano, but Robert Mons’ review on MacInTouch put me off. The really small number of songs displayed on the screen (see the photo about halfway down the review) is a non-starter for me as I have long lists of songs in my playlists.

Too bad, I really like the clip and the pedometer.


Nokia and the Road Ahead

Very interesting read about the road that Nokia faces. I believe that Nokia is still the largest sellers of phones in the world in terms of units, from that point of view they are already a commodity product. However they are getting pressure from the high end, and their high end phones have not been a hit in the US for whatever reason. Gruber’s article implies that they need to pick a software platform and that it should be applied across the board. I am not sure I agree. At the high end sure, but at the low end they just need something simple, a phone rather than a platform. Nothing wrong with a dual operating system strategy.


★ What’s Next for Nokia?: “None of Nokia’s options seem appealing, though. Symbian is crufty and has no legs. MeeGo isn’t ready, and has the whiff of vaporware. But going with Android or Windows Phone 7 reduces Nokia to the role of a commodity OEM, a peer not to Apple or RIM but rather merely to companies like HTC and LG. Nokia needs a platform that makes people want to buy a ‘Nokia phone’, not an ‘Android phone’ or ‘Windows phone’.

(Via Daring Fireball.)

Adobe – Security Advisories: APSA10-03 – Security Advisory for Flash Player

It seems the only reason I upgrade Flash is to deal with security issues, I am sure this upgrade will have a security issue as well, so why bother upgrading:

Adobe – Security Advisories: APSA10-03 – Security Advisory for Flash Player: “Accessibility Search Your account Your cart ContactUnited States (Change) Solutions Products Support Communities Company Downloads Store HomeSupportSecurity advisoriesSecurity bulletin Security Advisory for Flash Player Release date: September 13, 2010

Vulnerability identifier: APSA10-03

CVE number: CVE-2010-2884

Platform: All


A critical vulnerability exists in Adobe Flash Player and earlier versions for Windows, Macintosh, Linux, Solaris, and Adobe Flash Player for Android. This vulnerability also affects Adobe Reader 9.3.4 for Windows, Macintosh and UNIX, and Adobe Acrobat 9.3.4 and earlier versions for Windows and Macintosh. This vulnerability (CVE-2010-2884) could cause a crash and potentially allow an attacker to take control of the affected system. There are reports that this vulnerability is being actively exploited in the wild against Adobe Flash Player on Windows. Adobe is not aware of any attacks exploiting this vulnerability against Adobe R”



Great Hardware, Poor Software

I like the new iPods. The new Shuffle goes back to the previous design (I own one), I did not think much of the previous version which took minimalism a little too far I think, people like buttons it seems. The new Touch is a very nice evolution, basically catching up with the current iPhone (except for the camera). And the new Nano is really nice, I think I will be getting one. I am not worried that it lost its camera, and I could not care less about radio (unlike the RIAA which thinks I really need one).

The new Apple TV is very nice, I keep running out of space on the current one I have which means that I need to juggle what goes on there (I had upgraded to 160GB, was considering upgrading it to 320GB, but that would cost the same as a new Apple TV).

On the other hand iTunes keeps on disappointing and is in dire need of a complete rethink (see Ars Technica and Paul Thurrott on this). I was expecting Apple to having something new this time because they reached version 10, now it looks like we are going to have to wait until version 11 which may not come before September next year.

Old Computer Ads

The sad thing is that I used some of them.