Buy vs. Build

Very interesting post by Nati Shalom about Buy vs. Build:

“The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is as difficult as establishing the detailed technical requirements, including all the interfaces to people, to machines, and to other software systems. No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later.”

Go check out the paper “No Silver Bullet” by Fred Brooks as recommended by Nati Shalom.

Nice Java Tools Roundup

Nice Java tools roundup on JavaWorld.

Sources for latency – is it the network or the software?

Nati Shalom has a very interesting take on latency.

One section struck me as interesting:

When discussing latency most people fall into one of two main camps: the “networking” camp and the “software architecture” camp. The former tends to think that the impact of software on latency is negligible, especially when it comes to Web applications.

Marc Abrams says “The bulk of this time is the round trip delay, and only a tiny portion is delay at the server. This implies that the bottleneck in accessing pages over the Internet is due to the Internet itself, and not the server speed.”

The “software architecture” camp tends to believe that network latency is a given and there is little we can do about it. The bulk of latency that we can control lies within the software/application architecture. Dan Pritchett’s Lessons for Managing Latency (located here) provides guidelines for an application architecture that addresses latency requirements using loosely-coupled components, asynchronousinterfaces, horizontal scale from the start, active/active architecture and by avoiding ACID and pessimistic transactions.

I think that splitting into two camps is a little silly and misses the picture, I view hardware and software as part of a continuum and you need to look at your system as a whole when tracking down latency issues.

I have an interesting anecdote here. About 13-14 years ago I was talking to a computer scientist at a large (photocopier company) who told me that they created their own version of 100BaseT (this was before 100BaseT existed) to speed up communications between an image rasterizer and a printer because they had identified that as te bottleneck. Unfortunately they found that the communication speed had only marginally improved because the bottleneck had moved from the hardware to the software that ran the communication.

Further down:

Focus on application architecture and leave hardware and OS optimizations as a last resort. The performance provided by commodity hardware should be good enough for 80% of cases. In addition, the effort of optimizing hardware and Internet routers might involve a huge investment, and therefore, should be used sparingly.

I completely agree with this. Running on commodity hardware and generic versions of an operating system makes perfect sense because you know ahead of time the base you will be deploying too, and it makes administration easier. No sys-admin is going to want to customize machines when running a site with hundreds of machine. On the other hand, if you are running sites with thousands of machines (or more) then it would probably make sense to create your own OS distribution with whatever tools you want installed on each and every machine.

One thing I would add to the article:

Measure everything – it is really important to get as many measurements as possible on the time that operations take at every level of the stack. I have also found it useful to take measurements of the time needed to perform external operations, for example how long a piece of middleware had to wait for a external operation (such as a REST call) to complete. Additionally gathering these metrics will allow you to compare current timings against historical data so you will be able to tell very quickly when things are not operating normally.

Early on in Feedster’s life my co-founder and I had a major disagreement as to why searches were slow (at the user end of things.) The search engine was blamed and I added logging as to how long various operations took, and generated daily reports on the search times and was able to show that the slowness was not in the search engine.

Interarchy

Those out there who use Macs will likely have hear of Interarchy, a really excellent FTP application for the Mac. Actually calling an FTP application is a vast under-statement, it does a lot, lot more. It handles FTP. SFTP, SSH, WebDAV, iDisk, Amazon S3, HTTP and file access, along with bookmarks, auto and scheduled uploads, net disks. If you have to access file on remote servers at all you really should check it out.

Computers Emit as Much Carbon as Aviation Does

From the Economist:

The overall volume of emissions is comparable with that from aviation. But the IT industry, unlike aviation, has not provoked the wrath of environmental campaigners. Perhaps that is because computers are less visibly polluting, or their use is not deemed, like air travel, to be frivolous and unnecessary.

Carl Safina – Pop!Tech 2008

I just watched a really excellent presentation that Carl Safina did at Pop!Tech 2008:

Ecologist, author and lifelong fisherman Carl Safina is fighting to wake people up to the fact that fish are going the way of the buffalo. In this eye-opening Pop!Cast, Safina paints a grim portrait of the impact we’re having on the Earth’s oceans and what we need to do to turn the tide.

Well worth watching, and you should check out the Blue Ocean Institute and his personal web site.

iPhone Nano

There are rumors swirling around an iPhone Nano case that has hit the web as a possible indication of smaller iPhone.

First the skepticism. It is hard to see how the UI will work on a smaller device, especially the keyboard. It is also hard to see what kind of (smaller) battery could keep the device powered for a day, without removing things like GPS, Bluetooth and WiFi.

Then the optimism. I still have my original iPod which I take out from time to time to charge up and play around with. Next to the new iPods it really looks and feels like a brick, and it is heavier than I my iPhone, so it may not be too much of a stretch to imagine a progression for the iPhone which is similar to that of the iPod.

I think an iPhone, stripped of its GPS, Bluetooth, WiFi and most of its memory could be that small and could retail for $99 with a contract.

The Fallacy of the Two-Hour Lunch Break

Michael Arrington took a shot at european entrepreneurship at the last Le Web Conference in Paris and manage to infuriate just about every European there judging from the reactions on the web and the fact that a poll taken indicates that they don’t want him back.

First the disclaimers, I was born French, educated in England, worked both in France and in England before moved to the US in 1989, became America in 1999. My bio has a more information about my work history here.

I think Arrington has a valid point but delivered it in a cack-handed way. Europeans seldom take two hour lunches anymore, I think he was just extrapolating from his (brief and sumptuous) experience there as a conference guest.

I also think that you cannot generalize about Europeans, Europe is a collection of countries each with their own systems, laws and cultures. Viewed from the United States it is easy to assume that Europe is an homogenous block. Just as it is easy for Europeans to assume that United States is an homogenous block as well, which it isn’t as I frequently point out to Europeans I talk to. The United States is a collection of States each with their own governments, cultures, strengths and weaknesses. Europe is the same.

The things Arrington should have been pointing out to the Europeans are these:

France is somewhat hostile to entrepreneurs, as it Germany and some other countries. Employment is very rigidly controlled, making it difficult to fire people makes management think twice before hiring people. Employer taxes are high making it costly to hire people, Europeans automate where Americas hire. France is particularly backwards is some respects, limiting the workweek to 35 hours, having punitive tax rates, with a culture of ‘Dirigisme’ where control over the economy (and industry) emanates from the center of power (the government), something that is deeply rooted in the culture of the country, going back to the First Empire under Napoleon, and kept going by De Gaulle (with various revolutions and republics in between).

But when you look to the UK, or Ireland, or Spain, you find a different culture, one which is much more conducive to entrepreneurship. The reasons are diverse, ranging from large scale deregulation (in the UK) and a vast influx of European money (Ireland and Spain).

You need a set of preconditions for entrepreneurship to flourish, fluid hiring and firing practices, a willingness to take risks, an acceptance of failure as a normal part of business, the ability to keeps one’s gains relatively tax free so that they can be reinvested in future business, etc… All things which are hard to find in Europe, and all things which Europeans should press their leaders to deliver.

What Arrington should have been pointing are the differences between Europe and the US, specifically what makes Europe hostile to entrepreneurs and what makes the United States attractive to entrepreneurs. At least then a constructive dialog could be started, as it is there is no real dialog which is a loss to both sides.

Updated – Dec 24th, 2009, I forgot to add this quote from a very recent article in the Economist about a French entrepreneur called Malamine Koné:

Other than that, he has made his own way. “The French system is more rigid, more cautious and less risk-taking than the American one,” he says. “In France, the entrepreneur is afraid of failure.” Airness has succeeded despite the closed French elite, not because of it.

Aperture Photo Ordering

This morning I ordered some photo prints through Aperture. TWIP reported that the quality of the prints had improved since the 2.1.2 update.

The process was not without its hiccups though, my account was missing some information and the order would fail with a message that “the billing information was updated” (this is not a typo). Once that was sorted out the order went through without issue, and the order can be tracked on the Apple Photo Services Order History page.

I will post an update on the quality of the prints when I get them, which should be in a couple of weeks.

Having to Learn PHP on the Fly

Three days ago I took responsibility for a web site which runs on PHP so am having to learn PHP on the fly.

Interesting…