Old man of the sea


A white tip shark, about 6 feet long, resting on a ledge at about 80 feet at Alcyone in the Cocos Island. White tips hunt at night, sleep during the day and are one of the few sharks who are able to ‘breathe’ while not moving. By ‘breathing’ I mean that they are able to pump water across their gills.

I was able to get very close to this one, about 18 inches away and took six shots, four of which were at this angle and two were profile shots where I was able to get in closer.

I call this shot the old man of the sea because he was quite big for a white tip and you could see that he had been in a few fights.

MySQL engines, MyISAM vs. Innodb

I think Narayan Newton does a very good job of summarizing the pros and cons of MyISAM and Innodb in this post “MySQL engines, MyISAM vs. Innodb”.

I have seen a lot written about this before but I think his post neatly summarizes the arguments on both sides and as worth reading if you are having to make a decision about this.

My reflex is to always use Innodb unless there is a compelling reason for using MyISAM, and it has to be really, really compelling.

I did take issue with one point he makes which he illustrates with an experience:

On the other hand, InnoDB is a largely ACID (Atomicity, Consistency, Isolation, Durability) engine, built to guarantee consistency and durability. It does this through a transaction log (with the option of a two-phase commit if you have the binary log enabled), a double-write buffer and automatic checksumming and checksum validation of database pages. These safety measures not only prevent corruption on “hard” shutdowns, but can even detect hardware failure (such as memory failure/corruption) and prevent damage to your data.

Drupal.org has made use of this feature of InnoDB as well. The database in question contains a large amount of user contributed content, cvs messages, cvs history, forum messages, comments and, more critically, the issue queues for the entire Drupal project. This is not data where corruption is an option. In 2007, the master database server for the project went down. After examining the logs, it became clear that it hadn’t crashed as such, but InnoDB had read a checksum from disk that didn’t match the checksum it had in memory. In this case, the checksum miss-match was a clear sign of memory corruption. Not only did it detect this, but it killed the MySQL daemon to prevent data corruption. In fact, it wouldn’t let the MySQL daemon run for more than a half hour on that server without killing it after finding a checksum miss-match. When your data is of the utmost importance, this is very comforting behavior.

I have certainly had this happen to me, once or twice, and it is very satisfying to see Innodb recover and carry on on its merry way. However I did experience very nasty hardware failure where a RAID controller went nuts and sprayed bad data out to storage, Innodb won’t prevent this, the database had turned into a small pile of bit goo and Innodb was not able to recover it regardless of how high the ACP(*) innodb_force_recovery was set. We had to switch to backup and wipe the original system clean. It is probable that MyISAM would have been able to recover the database because of its simpler structure.

(*) ass-covering parameter.

We don’t support that in a Mac Pro!!

I have two graphics cards in my 1st generation MacPro which allow me to drive three monitors. I originally bought the machine two NVIDIA GeForce 7300 GT video cards because the two other choices were not really good choices (one was very expensive, and the other was a card which suffered from overheating). At the time the NVIDIA GeForce 7300 GT video card was considered respectable, which is another way of saying that it worked fine is a little pokey and was going to be EOLed momentarily.

Last week Apple released an NVIDIA GeForce 8800 GT video card upgrade for the 1st generation MacPro and I bought myself two to replace the two NVIDIA GeForce 7300 GT video cards. The card originally released only works in the 2nd generation MacPro (PCI express v.1 vs. v.2 issue.)

One of the NVIDIA GeForce 8800 GT video cards died after one day, the MacPro just did not “see” it anymore. The machine would boot fine but never detected the card, and the fan on the card was continually spinning at full speed, usually a good sign of a bad problem.

I called Apple and they told me that this was not a supported configuration. They first told me that this was a power draw issue which seems odd since the card is rated at 110W and Apple claims that the PCI express bus supports “300W for up to four PCI Express graphics cards”. Then they told me that it had to do with PCI express lanes, yeah right… what next, Aperture does not use the GPU to help with photo processing… oh wait, they did not know that either…

It came down to this, it just wasn’t a supported configuration. I had no choice but to replace the dead NVIDIA GeForce 8800 GT video card with one of the NVIDIA GeForce 7300 GT video cards.

One thing I learnt is that if you have two video cards in a 1st generation MacPro you should put them in slots 1 and 4 to get the best bandwidth out of the cards. You can also use a utility called “Expansion Slot Utility” which is located in “/System/Library/Core Services/”. This utility allows you some control on how the PCIe lanes are apportioned between the PCI cards in your MacPro. This does not apply to the 2nd generation MacPro though, that generation manages that automatically (what did you expect, they are 2nd generation!!)

Earth Day

Today is Earth Day.

There are dozens, if not hundreds, of issues one could pick to talk about in regards to conservation. I would like to draw attention to one I feel strongly about, shark fishing. Sharks are vastly overfished for their fins (to make shark fin soup.) The process is as horrific as it is simple, sharks are fished in very large numbers, their fins are cut off and they are tossed back (alive) into the ocean where they die by drowning (most sharks need to swim to ‘breathe’.) We also need to dispel the notion that a good shark is a dead shark, this is simply not the case, I have swum with sharks on many occasions and have never felt threatened by them even when they bumped into me (*).

This issue is pithy because the Shark Conservation Act of 2008 was recently introduced to Congress. Jim Toomey (who authors the Sherman’s Lagoon comic strip) explains why this is important:

April could turn out to be a pivotal month for shark conservation here in the U.S. Just a few days ago, the Shark Conservation Act of 2008 was introduced to our Congress. There are also lots of other activities this week and month that could go a long way toward much needed shark fishing regulations both here and in international waters. But before the political will must come the public interest. So, this week, it’s my goal to demonstrate to policy makers in Washington that public interest in shark conservation is quite strong, and the old notion that “the only good shark is a dead shark” is not a concept the public embraces anymore.

This Sunday, April 20, I have devoted my color Sunday “Sherman’s Lagoon” comic strip to creating awareness and public interest in saving sharks from extinction. Recent population studies done by numerous independent marine biologists confirm that many species of large sharks – from great whites to hammerheads to tiger sharks – are being overfished to the point that only 5% of their historic populations remain. This fall, international shark catch limits are going to be reevaluated by scientists, and the US National Marine Fisheries Service (NMFS) is going to establish much more stringent catch limits in US waters. The US regulations could set an international precedent, which is badly needed, since most of the overfishing is now happening in unregulated international waters. Having gained adequate limits in the US, the NMFS will have an opportunity – and the clout – to propose and fight for the first international catch limits for sharks in a key international meeting coming up this November.

We have a chance to show the NMFS that the public does indeed care about sharks – that they aren’t considered pests but a vital part of nature. When the Sunday April 20 cartoon gets published in your local paper, or here on this website, you will have an opportunity to participate in this public awareness effort. Dr. James Balsiger, director of NMFS, who will be the recipient of all of your mail-in cartoons, is aware of this campaign and is actually looking forward to a heavy response as a way to point out to the fishing communities and regulatory bodies that this is an issue the the public cares about. Please help him make that point, which could go a long way toward creating much-needed regulations, not only in US waters but around the world.

Thanks,

Jim Toomey

It could not be simpler, go to this page, print the image, draw your favorite shark on it and mail it to the address listed. I put mine in the mail yesterday.

(*) Interestingly more people are killed every year in the USA by pigs than by sharks. One would be forgiven thinking otherwise given the media coverage generated when the latter happens.

IT Consulting

Bob Cringely has a good article on IT Consultants, specifically what makes good consultants and bad consultants.

I was an IT consultant for 10 years, from 1993-2003, my clients ran the gamut from small startup to large multi-national corporation, and the contracts ran from short two week stints to much longer multi-year contracts.

This paragraph in his post really resonated with me:

The best consultant I ever knew was Christine Comaford-Lynch, who is now an author and a VC and no longer does IT consulting at all. A key part of her success was her requirements gathering process. She turned it into a very effective collaboration effort involving the key people who would use the software. The requirements would be tight, the project would be highly focused, and there would be little or no scope creep. When it came time to implement the project her project managers didn’t have to be Attila’s — there was cooperation and enthusiasm. The training and start up of the application was quicker and easier. There were few surprises that needed to be fixed.

Over time I learned that the key to a successful consulting job was setting and managing expectations, explaining to the client exactly what they were going to get, and making it clear where one’s expertise ran out. Clients really appreciates that last point, I had developed a very specific set of skills and would focus on those. While it does mean that money is left on the table, but it means that you don’t set yourself up for failure.

Caribbean Reef Octopus


This little guy is a juvenile caribbean reef octopus, smaller than my fist. I took quite a few shots, and I was able to get close enough to switch to macro mode on my camera. I was not more that 6 inches away. It got scared and dug itself into the sand under the rock right next to it.

For some reason the eye looks quite mesmerizing.

WordPress spam

For some reason I am starting to get a lot of comment spam from my blog, it started a few days ago and I am not sure why because usually I get no comment spam.

MySQL Proxy for sharding

I have been reading about various experiments using MySQL Proxy to handle sharding (and by extension scaling) for application by rewriting SQL queries as they come through and directing them to the appropriate shards.

The most visible project seems to be HScale, which is well worth looking at and reading about.

The premise is very compelling, which is to remove the issue of sharding from the application layer, moving it into the database layer. This makes the application less complex because it no longer needs to deal with sharding (though it could be argued that sharding, if correctly done, has very little ‘imprint’ on the application.)

I think this project has promise but there are some questions that needs to be addressed before it is really ready to be used in a production setting:

  • First is that the MySQL Proxy introduces a single point of failure. If it fails, the application stops. At the very least, there needs to be a number of proxies and the application needs to be able to detect when one has failed and switch over to another one. I suspect you could get around that issue with a load balancer.
  • Second sharding does not mean that your application automatically becomes fault tolerant. If you have more machines, the odds of one failing go up, so the proxy needs to be able to handle failing over from a failing server to a backup server.

Both of those are difficult problems to deal with, and like a lot of software projects it is the 20% that is going to take 80% of the time.

When in Rome…

I have been doing a lot of work parsing feeds (both RSS and ATOM) lately and have been using a tool called “Project ROME” for that. I know there is another tool called Abdera but that only handles ATOM feeds.

The ROME project page describes it as follows:

ROME is an set of open source Java tools for parsing, generating and publishing RSS and Atom feeds. The core ROME library depends only on the JDOM XML parser and supports parsing, generating and converting all of the popular RSS and Atom formats including RSS 0.90, RSS 0.91 Netscape, RSS 0.91 Userland, RSS 0.92, RSS 0.93, RSS 0.94, RSS 1.0, RSS 2.0, Atom 0.3, and Atom 1.0. You can parse to an RSS object model, an Atom object model or an abstract SyndFeed model that can model either family of formats.

Which is what it does and it does it very well. I have thrown any number of feeds at it and it has performed very well. What I particularly like is the fact that foreign markup is accessible so any special tags like iTunes and Media RSS.

No tool is perfect and there are a few ‘lackings’ in it.

  • For some reason it does not support comment urls in items, I am not sure why this is the case since I would have expected it.
  • Some feeds contain some XSL/CSS directives located just before the feed itself, those are used to direct a browser to “pretty print” the feed when it displays it rather than raw XML. ROME does not like that at all and this stuff needs to be stripped from the feed before it is handed over for parsing.
  • Some feeds (like the NY Times, ahem…), have lots of null characters past the end of the feed, but which are part of the document. I suspect what is happening somewhere is that the feed is deemed to be longer than it actually is and the empty space is filled with null characters (let us pass on the existential issue of filling empty space with nulls). Those also need to be stripped out.

Unfortunately the last release was made in December 2006 and the project does not seem to have any work done on it since. Hopefully someone will step up to the plate and take it on, I might when work lets up. The one obvious thing I would do is add Generics to it.

Java 5.0

Recently I asked a colleague if he could recommend some good books on Java, specifically covering the new features in Java 5.0. I have used a number of the 5.0 features by gleaning them from code and documentation I gathered from the internet, but I was looking for something which brought everything together. He did not have any ideas off the top of his head so I did a little digging and found two books which fit the bill:

The first one is “Java In A Nutshell, 5th Edition” (by O’Reilly Media). It has a chapter dedicated to the bigger additions to Java, namely generics, enumerations and annotations. Additionally it covers additional features like ‘for/in’ loops other control flow features in the chapter on Java syntax, making it clear what features were added in 5.0.

The second one is “Learning Java, 3rd Edition” (by O’Reilly Media). It has two chapters dedicated to Java 5.0 features.

While all the information in those books can be found on the internet, I find it easier to have it all collated in books.

Follow

Get every new post delivered to your Inbox.