Java Garbage Collection

Lucid Imagination has put together a good primer to garbage collection in Java:

Garbage collection in Java is the processes of freeing the dynamic memory used by objects that are no longer being used by an application. In languages such as or C or C++, the developer is often responsible for managing dynamic memory (using malloc and free or new and delete). However, in Java, this task is left up to something known as the garbage collector. A garbage collector automatically frees unused memory, freeing the developer from much of this thankless memory juggling.

Well worth reading.

Additionally here are some more links:

 

Simple Logging Facade for Java (SLF4J)

Recently came across Simple Logging Facade for Java (SLF4J), it is easier to use as logging front end to Apache log4j than Apache Commons Logging (I use log4j as my logging system.)

The Simple Logging Facade for Java or (SLF4J) serves as a simple facade or abstraction for various logging frameworks, e.g. java.util.logging, log4j and logback, allowing the end user to plug in the desired logging framework at deployment time.

Before you start using SLF4J, we highly recommend that you read the two-page SLF4J user manual.

In case you wish to migrate your Java source files to SLF4J, consider our migrator tool which can migrate your project into SLF4J in minutes.

In case an externally-maintained component you depend on uses a logging API other than SLF4J, such as commons logging, log4j or j.u.l, have a look at SLF4J’s binary-support for legacy APIs.

It did not take me too long to switch all my code over, a few hours and a couple of Perl scripts to do all the search-replace. The reason I switched is that it promises to have a much lighter CPU footprint compared to Commons Logging.
I did miss the Fatal logging level though:

Why doesn’t the org.slf4j.Logger interface have methods for the FATAL level?

From the stand point of a logging system, the distinction between a fatal error and an error is usually not very useful. Most programmers exit the application when a fatal error is encountered. However, a logging library cannot (and should not) decide on its own to terminate an application. The initiative to exit the application must be left to the developer.

Thus, the most the FATAL level can do is to highlight a given error as the cause for application to crash. However, errors are by definition exceptional events that merit attention. If a given situation causes errors to be logged, the causes should be attended to as soon as possible. However, if the “error” is actually a normal situation which cannot be prevented but merits being aware of, then it should be marked as WARN, not ERROR.

Assuming the ERROR level designates exceptional situations meriting close attention, we are inclined to believe that the FATAL level is superfluous.

I am not sure I buy their argument though, I had a number of fatal events logged in my code which would cause instant termination. Usually this because of missing configuration files, missing directories, failure to connect to a service to get vital information, stuff like that. I renamed them to errors and changed the message to be a little more urgent in tone.

Memory Leaks in Java

I came across three good articles on memory leaks in Java and how to go about tracking them:

Memory leaks et alii

One of the most common problems in building enterprise web applications are leaks. A leak is consumption of a resource by a program where the program is unable to release the resource. Leaks come in various types, such as

  • Memory leaks
  • Thread and ThreadLocal leaks
  • ClassLoader leaks
  • System resource leaks
  • Connection leaks

A day in the life of a memory leak hunter

… but my overall happiness level suffered a noticeable drop when I got a mail in my Inbox from a coworker saying that the logs of a particular service in our dev cluster are littered with OutOfMemoryError-s.

Taxonomy of class loader problems encountered when using Jakarta Commons Logging

This document attempts to systematically categorize the types of problems encountered when using Jakarta Commons Logging, hereafter abbreviated as JCL.

Migrating from iBatis 2.x to 3.0

I recently migrated a project from iBatis 2.x to 3.0 and thought I would share some notes on the differences between the two:

  • You will need to download asm-3.2 from http://asm.ow2.org/.
  • The iBatis xml configuration file format has been changed a little, the handling of different environments is much improved, allowing you to easily switch data sources in the xml file.
  • The mapper xml file format has been changed a lot, the naming has been cleaned up, and the dynamic SQL generation is a lot cleaner. One gotcha is that you need to set the jdbcType of columns which allow NULL values (this was not the case before, but is not a bit deal.)
  • SqlClientMap has been done away with and replaced by a SqlSession which has a much cleaner API and a much better life-cycle. Note that you can set a SqlSession to autocommit or you can call .commit(), and it is always a great idea to .close() a SqlSession.
  • On thing that did bite me is that SqlSession cache results so a second SELECT will return the same results even though the data was changed by another process (not sure if this applies to INSERT and UPDATE though,) so you should use .clearCache() if you want the results to come from the database as opposed to the cache.

A few other notes:

  • With iBatis 2.0 I would create a bunch of SqlClientMap and cache them in a hash so that I would not need to create new ones for each session. This helped performance a lot but was not good practice. You cannot do that with SqlSession, so what I did was to create and cache my SqlSessionFactory objects and create new a SqlSession from them when I needed. The reason I needed to do this is that the project in question has about 45 databases. SqlSessionFactory objects can be shared across threads.
  • I use C3P0 as my data sources and a little work was needed to make it work properly under iBatis 3.0. What I did was to create a new combo pooled data source for each driver/url/user name combination and cache those in a hash returning them as needed by iBatis.
  • iBatis 3.0 is much, much faster than 2.0 when it comes to SELECTs, this is a very welcome change, however INSERTs/UPDATEs/DELETEs are a little slower but not enough to have any impact.
  • Finally the beans used by iBatis are “enhanced by CGLIB”, so the result of ‘o.getClass()’ will return ‘O$$EnhancerByCGLIB$$2632338b’ which makes a clause like ‘getClass() != o.getClass()’ in the .equals() method quite useless and you need to use ‘!(o instanceofO)’ instead, otherwise objects which are the same will not be reported as such. Of course this assumes that you have an .equals() method in your beans.

You will also want to read the iBatis 3.0 documentation.

JDK 1.6.0_18 Bug

Found a bug in JDK 1.6.0_18.

This, abeit brain-dead, code causes some sort of deadlock condition where multiple threads get stuck in the replaceAll() method:

// Replace all double spaces with single spaces
while ( string.indexOf(" ") != -1 ) {
   string = string.replaceAll("\\s\\s", " ");
}

Modify it to this makes it go away:

// Replace all double spaces with single spaces
string = string.replaceAll("\\s+", " ");

Maybe the first version of the code was not supposed to work in the first place.

To equals() or not to equals()

“How to Write an Equality Method in Java” from Artima:

Class java.lang.Object defines an equals method, which subclasses may override. Unfortunately, it turns out that writing a correct equality method is surprisingly difficult in object-oriented languages. In fact, after studying a large body of Java code, the authors of a 2007 paper concluded that almost all implementations of equals methods are faulty.

The article goes into some depth on how to write a good equals method, and on the pitfalls that surround such a method. The article also makes a reference to “Effective Java Second Edition” by Josh Bloch which I am currently reading:

In Item 8 of Effective Java1, Josh Bloch describes the difficulty of preserving the equals contract when subclassing as a “fundamental problem of equivalence relations in object-oriented languages.” Bloch writes:

There is no way to extend an instantiable class and add a value component while preserving the equals contract, unless you are willing to forgo the benefits of object-oriented abstraction.

C3P0, MySQL and ‘Error Reading Communication Packets’

I have been using C3P0 as my connection pool for iBatis to connect to a MySQL database (via the MySQL connector for Java) and have been dogged by an annoying issue where the client (presumably iBatis) drops the connection without closing it first causing MySQL to complain with the error message “Got an error reading communication packets”. Not a major issue because it was not causing the application to fail, but it was generating about 2GB of unnecessary log every month.

Basically what is going on is that the connection times out at the MySQL server end because it is idle (more than 60 seconds in this case). I set the idle timeout in C3P0 to 50 seconds to ensure that the connection was closed there before the timeout triggered in MySQL, obviously that was not working. As I mentioned above what I think is going on is that iBatis is dropping the connection without closing it properly first:

I get pages of error logs with this message –
“Got an error reading communication packets”. And the aborted clients
are easily in hundreds.

You get that message in the log every time the server closes a client
connection because it was inactive too long. Too long is “longer than
wait_timeout”.

In fact C3P0’s author, Steve Waldman, mentions this:

Sometimes client applications are sloppy about close()ing all Connections they check out. Eventually, the pool grows to maxPoolSize, and then runs out of Connections, because of these bad clients.

I was not about to go grubbing around in the iBatis code (and whatever else) to figure out where the issue was, and found that ‘propertyCycle’ would probably help me:

propertyCycle

Default: 0
Maximum time in seconds before user configuration constraints are enforced. Determines how frequently maxConnectionAge, maxIdleTime, maxIdleTimeExcessConnections, unreturnedConnectionTimeout are enforced. c3p0 periodically checks the age of Connections to see whether they’ve timed out. This parameter determines the period. Zero means automatic: A suitable period will be determined by c3p0. [You can call getEffectivePropertyCycle…() methods on a c3p0 PooledDataSource to find the period automatically chosen.]

The ‘suitable period’ chosen by C3P0 here was 12.5 seconds, which means that it is very likely for the cycle to fall outside my window of 50 and 60 seconds (see above.) Changing this to 5 seconds fixed the issue (10 still generated errors once in a while.) Interestingly the ‘suitable period’ is stored internally as a float (as retrieved by getEffectivePropertyCycleDefaultUser()) but is set as an int (as set by setPropertyCycle()), go figure.