Finally Pinned Down my Java Issue

I finally pinned down the Java issue that I have been dealing with, documented in this post.

I finally hit on it when I was doing some work to lazy-load data in a ket object, and found that performance was good when I was part done and went to hell when I was fully done. Back-tracking and digging through the code, effectively cutting out pieces until I narrowed it down to this:

Set urlSet = new HashSet();
#Loop... {
 String href = "...";
 urlSet.add(new URL(href)); # This kills the JVM

What I am doing is extracting the urls from a piece of html and using the Java Set to de-duplicate them. For some reason the last statement causes the JVM to slow down to a point where it is just idling from the CPU’s point of view (2%-5% usage).

However this works fine as a work-around:

Set urlSet = new HashSet();
#Loop... {
 String href = "...";

List urlList = new ArrayList();
for ( String href : hrefSet ) {
 urlList.add(new URL(href));

I am using the latest version of the JVM on Linux Centos 5.0, Dual 2.4GHz Xeon, 4GB RAM:

java version “1.6.0_10″
Java(TM) SE Runtime Environment (build 1.6.0_10-b33)
Java HotSpot(TM) Server VM (build 11.0-b15, mixed mode)

Finally, and not least, thanks to a colleague for suggesting testing the code with strings as opposed to URLs.

Updated – my colleague pushed me to check the Java source code and it turns out that the class uses the method which does a DNS lookup on line 337 to work out the hash code:

InetAddress addr = getHostAddress(u);


MySQL 5.1.29 Released

MySQL 5.1.29 was just released, the final release candidate on the way to general availability. I have been running 5.1.28 for a while with no issues.

The main change I see here (for me at least) is the ability to change logging without having to restart the server.

Log on demand is one of the main features of MySQL 5.1. It means that you can enable and disable general logs and slow query logs without restarting the server. You can also change the log file names, again without a restart.

What was missing was the ability of setting a log file name in the options file, without actually starting the logging. The old option log, could set the general log file name, but it will also start the logging immediately. If you want to set the log file name without activating the logging, now you can use general_log_file=filename or slow_query_log_file=file_name in the options file. These features were already available as system variables, but not as startup options.

From the “software takes time department”, I first started playing around with the 5.1 release while at Feedster at the end of 2006, but we chose not to use it because it was just too unstable.

Accident Waiting to Happen

MacMerc published a set of disassembly photos of the faulty iPhone charger (the one that is getting recalled).

It turns out the prongs are just glued to the casing, though I have not tried it (and I don’t recommend that anyone does) it looks like you can rip the prongs out by just twisting the charger when it is in a wall socket.

Upgraded to BBEdit 9.0.2

I have finally upgraded up to BBEdit 9.0.2. I had had a few problems with the initial 9.0 and 9.0.1 releases (see here and here).

I provided BareBones with a sample Java file so they were able to fix the function popup issue there. The “Text file busy” issue turned out to be a little more complicated. BBEdit uses kevents to monitor for file changes and it looks like netatalk interprets that as a lock of some sort. I have seen this issue before on linux with volume mounted via NFS and lockd was not running.

I played around a little bit with the netatalk configuration and was not able to resolve the issue, but there are two workarounds.

One is to turn off file monitoring via the “Automatically refresh documents as they change on disk” preference.

Another is to precede the script with the interpreter name as follows:

linux32: ~ > perl ./

I will post an update if I find a cleaner solution as I like to have file monitoring turned on.

Hermit Crab

Apologies for skipping a week in the scuba posting, things have been very busy.

Time to post something small again (I alternate between something large and something small). So I picked a Hermit Crab, they are usually small right? Well usually except that this one is pretty BIG having made a home in an empty Conch shell.

Hermit Crabs are very shy and it does not take much for them to retreat back into their shells, so you will need to wait patiently if you want to a picture or two.

I suspect that this guy would be about the size of a baseball if you were to take it out of its shell, which I would recommend against trying because its claws are pretty big and could inflict a lot of damage.

Java Exceptions

Recently I asked a colleague what the current orthodoxy was regarding exception handling in Java. This is not covered at all well in any of the books I have read, besides which I suspected that this was shifting ground, so asking such a dumb question was in order.

I was pointed to three documents, the official Sun position, Bruce Eckel’s post and finally this (very good) article by Brian Goetz.

Having read all this, and based on my experience, I think this is probably the sanest approach (or least insane depending on your point of view), from the article by Brian Goetz:

Rod Johnson, author of J2EE Design and Development (see Resources), which is one of the best books I’ve read on Java development, J2EE or not, takes a less radical approach. He enumerates several categories of exceptions, and identifies a strategy for each. Some exceptions are basically secondary return codes (which generally signal violation of business rules), and some are of the “something went horribly wrong” variety (such as failure to make a database connection). Johnson advocates using checked exceptions for the first category (alternative return codes), and runtime exceptions for the latter category. In the “something went horribly wrong” category, the motivation is simply to recognize the fact that no caller is going to effectively handle this exception, so it might as well get propagated all the way up the stack with the minimum of impact on the intervening code (and minimize the chance of exception swallowing).

Johnson also enumerates a middle ground, for which he asks the question, “Will only a minority of callers want to handle the problem?” For these scenarios, he also suggests using unchecked exceptions. As an example for this category, he lists JDO exceptions — most of the time, JDO exceptions indicate a condition that the caller will not want to handle, but some cases occur where specific types of exceptions might usefully be caught and handled. Rather than make the rest of the JDO-consuming classes pay for this possibility in the form of catching and rethrowing these exceptions, he suggests using unchecked exceptions here instead.

Enceladus Up Close

Amazing pictures of Enceladus up close:

Saturn’s tiny, icy moon Enceladus has recently been visited by NASA’s Cassini orbiter on several very close approaches – once coming within a mere 25 kilometers (15 miles) of the surface. Scientists are learning a great deal about this curious little moon. Only about 500 kilometers wide (310 miles), it is very active, emitting internal heat, churning its surface, and – through cryovolcanism – ejecting masses of microscopic ice particles into Saturnian orbit. Cassini has been orbiting Saturn for over 4 years now, and has provided some amazing views of tiny Enceladus, some collected here. Another close flyby is scheduled for Halloween, October 31st.

MacBook Review

Macintouch published a really good review of the new Apple MacBook:

The radically redesigned aluminum MacBook is the first major retooling of Apple’s consumer-oriented laptops since the Intel-based MacBook replaced the PowerPC-based iBook in May 2006. A new “unibody” chassis made from a single piece of machined aluminum brings the look of the MacBook Pro line, while improving serviceability, and a striking black, glass display now unifies Apple’s MacBook, iMac and iPhone designs. A new, glass trackpad extends gestural control.

Brand new video hardware from Nvidia replaces the old Intel graphics, and the mini-DVI port has been replaced with mini-DisplayPort, so the MacBook now can drive the massive 30″ Apple Cinema Display.

But Apple has removed FireWire, leaving many users — especially IT professionals, musicians and video enthusiasts — stranded with unusable investments in important FireWire peripherals.

The new MacBook costs more than the old one — it retails for $1299 — but the old, white MacBook model, upgraded with a SuperDrive and complete with FireWire, is available for just $999. A $1599 aluminum MacBook model adds a faster CPU and a larger hard drive. Like all Macs, the aluminum MacBook includes the latest iLife suite (iPhoto, iMovie, GarageBand, iDVD, iWeb, and iTunes).

Well worth reading if you are thinking of getting one of these puppies, the lack of FireWire and the shiny, shiny screen is an issue though.

Lazy Linux

I have to admit that I am somewhat lazy, well lazy is not the best way to put it, I am careful with how I spend my time, so spending time checking on machines, processes, and what-have-you is not very productive. I much prefer to automate and have problems flagged to me when needed.

While the article “Lazy Linux: 11 secrets for lazy cluster admins” talks about cluster management, the lessons are equally applicable to JBOM (Just a Bunch Of Machines):

Cluster means different things to different people. In the context of this article, cluster is best defined as scale-out — scale-out clusters generally have a lot of the same type of components like Web farms, render farms, and high performance computing (HPC) systems. Administrators will tell you that with scale-out clusters any change, no matter how small, must be repeated up to hundreds of thousands of times; the laziest of admins have mastered techniques of scale-out management so that regardless of the number of nodes, the effort is the same. In this article, the authors peer into the minds of the laziest Linux® admins on Earth and divulge their secrets.

Java Performance, Lack Thereof, A Follow-Up

I had been having an issue with lack of performance in a Java app, this post tells all.

Over the weekend I decided to just shoot the problem between the eyes with extreme prejudice (bonus points if you get the movie reference), toss out the whole thing and move to a messaging architecture. I should be done in a couple of days, but early tests looks very promising.

Effectively what I decided to do was to do away with pulling data from MySQL entirely, instead having it sent to a queue for processing (hence the messaging.)


Get every new post delivered to your Inbox.