Java

Hibernate: could not read column value from result set?

I got this Error today after changing the primary key of my model objects from int to Serializable (as I wanted all of my objects, even the ones with a String primary key to use the same API, which was limited to int). But when I started the app I got the Error and the following Exception:

org.hibernate.type.SerializationException: could not deserialize
at org.hibernate.util.SerializationHelper.deserialize(SerializationHelper.java:217)
at org.hibernate.util.SerializationHelper.deserialize(SerializationHelper.java:240)
at org.hibernate.type.SerializableType.fromBytes(SerializableType.java:82)
at org.hibernate.type.SerializableType.get(SerializableType.java:39)
at org.hibernate.type.NullableType.nullSafeGet(NullableType.java:163)
at org.hibernate.type.NullableType.nullSafeGet(NullableType.java:154)
…..

I even got this one as cause:

Caused by: java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2279)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2748)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:780)
at java.io.ObjectInputStream.(ObjectInputStream.java:280)
….

EOF? wtf? Don’t get too confused with that EOF, the solution was simply to add the type of the id in the mapping file.

Before:

<id name=”id”>
<generator class=”native”>
</id>

After:

<id name=”id” type=”integer”> <!– or long or string of whatever your hibernate type is –>
<generator class=”native”>
</id>


Introducing iSport.eu

iSport.eu is a simple but hopefully useful website for people like me who spend several hours a week to get the latest football (American readers: feel free to replace the word ‘football’ with ‘soccer’) news from the major European leagues. iSport.eu is kind of an RSS reader, that adds some nice features (well, at least I like them):

  • no need to explain your less tech-savvy friends what the hell RSS is and why it is useful (RSS is only used for content retrieval, not for publishing)
  • auto-tagging of entries
  • grouping of entries into different categories (i.e. teams) – get only the news for your favorite teams.

While this tool currently is focused on football news, it could be used for everything that comes with a news feed.

Depending on the feedback, future enhancements could include:

  • Display only news from your favorite news portal
  • Create your personal set of categories and sources (not into football? create your personal hockey news site!)
  • Ratings and comments
  • Tag clouds – every web-2.0-ish site has to have those ;)
  • Mark entries as “read”
  • More languages (currently English, German, Italian, Spanish and French are supported)

iSport.eu is built on Java, Wicket, Spring, Hibernate, Derby, Jetty, Commons-Digester and about 10 days of manpower.


Overcautious or just paranoid?

Wicket‘s RepeatingView is a very handy component: You can add as many components as you want. As a result, the usage of this component differs from other, as you don’t add components using html tags with a wicket:id attribute. Instead, IDs are generated by incrementing a number and convert it to a string. The RepeatingView even offers a nice helper method to generate those IDs. Basically, this method does nothing else than

public String newChildId() {
  return Integer.toString(childIdCounter++);
}

But the folks behind Wicket seem to work more carefully than I do:


/**
 * Generates a unique id string. This makes it easy to add items to be rendered w/out having to
 * worry about generating unique id strings in your code.
 *
 * @return unique child id
 */
public String newChildId()
{
  childIdCounter++;
  if (childIdCounter == Long.MAX_VALUE)
  {
    // mmm yeah...like this will ever happen
    throw new RuntimeException("generateChildId() out of space.");
  }

  // We prepend the id's with the text 'id' so they will generate valid
  // markup id's if needed.
  return String.valueOf(childIdCounter);
}

But does it really make sense to check whether Long.MAX_VALUE (263-1 or 9,223,372,036,854,775,808) was reached? The developer doesn’t believe so, but seems to work carefully … or overcautious? … or just paranoid? Well, let’s do the maths:

As already an empty string object has 40 bytes (x86 processor, Sun JVM), creating this number of empty strings would require more than 325.000 petabytes of memory.

But even if the objects are left for garbage collection, calling this method Long.MAX_VALUE times takes ages. I tested the same method with Integer.MAX_VALUE (231-1 or 2,147,483,647) on my Pentium M 1.86GHz and it took 14 minutes to hit the RuntimeException. As It would take 232 times longer to reach Long.MAX_VALUE, it would approximately take 114,401 years. Quite a long time to construct a web page, eh? ;)


Using Wicket’s jmx-panel to view Hibernate’s Statistics

The Java Management Extensions (JMX) is perfect to build manageable applications without the need to create any administration frontend. Managed Beans (MBeans) are similar to regular Java Beans. Hibernate offers an MBean-wrapper around its Statistics API which is really interesting to optimize caching. Therefore, I was looking for a simple way to use JMX with Wicket. Fortunately, Wicket can also be managed with JMX. All it needs is to drop the wicket-jmx JAR on the classpath – no configuration needed! It is therefore sufficient to add wicket-jmx to the POM:


<dependency>
 <groupid>org.apache.wicket</groupid>
 <artifactid>wicket-jmx</artifactid>
 <version>1.3.0-SNAPSHOT</version>
</dependency>

But Wicket (more precisely Wicketstuff) also contains a simple JMX client called wicketstuff-jmx-panel. Using this client is as simple as adding a single component to whatever Wicket-page:

POM:

<repository>
 <id>wicket-stuff-repository</id>
 <name>Wicket-Stuff Repository</name>
 <url>http://www.wicketstuff.org/maven/repository</url>
</repository>
<!-- SNIP -->
<dependency>
 <groupid>org.wicketstuff</groupid>
 <artifactid>wicketstuff-jmx-panel</artifactid>
 <version>1.3.0-SNAPSHOT</version>
</dependency>

Java:


add(new JmxPanel("jmx"));

However, documentation on this JMX panel is sparse. It took me some time to figure out the following: This panel filters all MBeans that do not have the Wicket-application’s name in their domain. This behaviour may be changed using an annonymous inner class:


add(new JmxPanel("jmx") {
 protected IDomainFilter getDomainFilter() {
   return DOMAIN_FILTER;
}
});

private static final IDomainFilter DOMAIN_FILTER = new IDomainFilter() {
 private static final long serialVersionUID = 1L;

 public boolean accept(String domain) {
   // your filter code
   return true;
}
};

It does make sense to filter out some MBeans as there are already some by default, e.g. for java.util.logging which you probably don’t use at all.

Now it’s time to add Hibernate’s statistics MBean. This MBean is can be used per SessionFactory. I therefore decided to do registration and unregistration together with SessionFactory creation. As I am also using Spring, I went for my own SessionFactoryBean:


public class MBeanRegisteringLocalSessionFactoryBean extends
   org.springframework.orm.hibernate3.LocalSessionFactoryBean {

 private String _sessionFactoryName;
 private ObjectName _objectName;
 private MBeanServer _mbeanServer;

 @Override
 protected void afterSessionFactoryCreation() throws Exception {
   super.afterSessionFactoryCreation();
   try {
     // register JMX MBean
     _objectName = new ObjectName("Hibernate:type=statistics,application="
       + getSessionFactoryName());
     _mbeanServer = getMBeanServer();

     final StatisticsService mBean = new StatisticsService();
     mBean.setSessionFactory(getSessionFactory());
     _mbeanServer.registerMBean(mBean, _objectName);

   } catch (final Throwable t) {
     log.warn("failed to register MBean for SessionFactory: " + _objectName, t);
     _objectName = null;
     _mbeanServer = null;
   }
   // if _objectName != null --> MBean was registered
 }

 @Override
 protected void beforeSessionFactoryDestruction() {
   if (_objectName != null) {
     try {
       _mbeanServer.unregisterMBean(_objectName);
     } catch (final Throwable t) {
       log.warn("failed to unregister MBean for SessionFactory: "
         + _objectName.getDomain());
     }
   }
 }

 /* SNIP: getMBeanServer(), getSessionFactoryName(), setSessionFactoryName(String) */
}

I copied (sorry, there is no nicer way yet) the code for getMBeanServer() from wicket-jmx’s Initializer class (sorry, can’t find an online version – you can get the source using Maven) in order to get the exact same behaviour. Now you only need to replace LocalSessionFactoryBean with MBeanRegisteringLocalSessionFactoryBean in your beans.xml and you are set. In order to enable statistics by default, add these two lines to your hibernate properties:


hibernate.generate_statistics=true
hibernate.cache.use_structured_entries=true

That’s it! Hibernate statistics directly from your webapp:


Wicket ChoiceRenderer for Enums

I now got to the point where I wanted to put some i18n into my DropDownChoices. What I found out pretty soon was that you need an IChoiceRenderer to customise your DropDownChoices. If you don’t use one, you’ll get by default short values and toString display names. E.g. the enum


enum { JOHN, PAUL, GEORGE, RINGO }

produces an output like that


<select name="beatles">
<option value="0">JOHN</option>
<option value="1">PAUL</option>
<option value="2">GEORGE</option>
<option value="3">RINGO</option>
</select>

But what I want is “John” instead of JOHN or “John Lennon, Rhythm Guitar” or anything but JOHN. On the mailinglist and in the wiki I discovered some complicated, overhead solutions using helper objects, so I decided to implement my own reusable enum-renderer that reads the display values from a properties file. All you have to do is to pass the Component that is responsible for the properties file.


public class EnumChoiceRenderer implements IChoiceRenderer {

private static final long serialVersionUID = 1L;

private final Component _resourceProvider;

public EnumChoiceRenderer(final Component resourceProvider) {
_resourceProvider = resourceProvider;
}

public Object getDisplayValue(final Object object) {
final Enum<?> v = (Enum<?>) object;
return _resourceProvider.getString(v.name());
}

public String getIdValue(final Object object, final int index) {
return Integer.toString(index);
}

}

Yeah that was quite easy, but maybe saves some of you the burdon of using helper-classes ;-)

Oh yes, the properties file looks something like that:

JOHN=John Lennon, Rhythm Guitar
PAUL=Paul McCartney, Bass
GEORGE=George Harrison, Lead Guitar
RINGO=Ringo Starr, Drums

what then produces the desired output:

<select name="beatles">
<option value="0">
John Lennon, Rhythm Guitar</option>
<option value="1">
Paul McCartney, Bass</option>
<option value="2">
George Harrison, Lead Guitar</option>
<option value="3">
Ringo Starr, Drums</option>
</select>


Capacity planning

Today, I discovered a nice little gotcha in the Spring source, namely in org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor, while I was implementing a priority task queue. The mentioned class uses a template method to create the backing queue for pending tasks:

protected BlockingQueue createQueue(int queueCapacity) {...}

I’ve overridden this method like so:

protected BlockingQueue createQueue(int queueCapacity) {
return new PriorityBlockingQueue(queueCapacity, new Comparator() {...});
}

However, this didn’t work as I got:

java.lang.OutOfMemoryError: Requested array size exceeds VM limit

Why? By default, the ThreadPoolTaskExecutor uses

new LinkedBlockingQueue(queueCapacity);

With a default queueCapacity of Integer.MAX_VALUE … This is ok, as the capacity of a LinkedBlockingQueue is its maximum capacity while the capacity of the PriorityBlockingQueue is its initial capacity (and used to create a array of the given size). Unfortunately, it isnt’t even possible to limit the size of PriorityBlockingQueue at all. Therefore, the parameter should be ignored for PriorityBlockingQueues.


wicketstuff-push, bayeux and the memory leak

I’ve been fiddling around with reverse-ajax (or comet or pushlets) these days/weeks. You know, that stuff where the webserver keeps an open connection to the client and pushes new events to the client with just the transportation-delay – but no client interference. As described in my recent posts, I decided to use Apache Wicket for my webapp, so I looked around if there’s something in the wicketstuff project that fits my needs. And yes, there is the wicketstuff-push project that offers different implementations of server pushes. Xavier Hanin, one of the men behind Ivy, and Vincent Demay did great effort there to implement a timerbased PushImplementation and also a Cometd-based implementation of a event subscribe/publish mechanism. The latter uses an implementation of the Bayeux-spec and the Dojo-library together with wicketstuff-dojo to subscribe to channels and to process the pushes on client side.

Yesterday I took the time to have a closer look on how the Java implementation of the Bayeux protocol works and found out that each new client that subscribes to a channel gets stored in the Bayeux’s client-HashMap … and stays there. So apparently it is up to the user to remove the clients from there. Otherwise the HashMap would grow towards an OutOfMemoryError, which needed some 400.000 clients on my machine.
So how to get them out? Well it ain’t just that easy to say “on session unbinding I simply remove the client with the corresponding session id” as the client id the Bayeux protocol uses isn’t the http-session id. The client id also changes more frequently, a simple reload of the page is enough to have a new client id. So for my application, I subclassed CometdServlet (over which all request and thus all subscriptions are processed) and made a http-session – bayeux-client-session mapping. This means that I always know which session has which client id, and if the client-id differs, it simply gets removed from the Bayeux-map.

Problem solved

P.S.: Bayeux does the same with channels, so be careful with the amount of channels in your webapp
P.P.S.: Wicket’s first release candidate for version 1.3 is out *thumbs-up*


Maven: Unable to find resources in test cases?

Just right now, I was once again fighting the beast called Maven. This time, my JUnit tests failed with Maven although they used to work within in Eclipse. The test cases failed due to missing resources. I searched for these resources using the ClassLoader.getResource() mechanism which always worked just fine so far.

After some swearing and investigation I finally found the problem: In my code, I used ClassLoader.getSystemResource(...) to find a resource, which internally calls ClassLoader.getSystemClassloader(). However, with Maven’s Surefire plugin, the system classloader does neither contain target/classes nor target/test-classes in its classpath. Therefore, you have to make sure, that the classloader that loaded the test cases (and the tested classes) is used to get the resources.

Therefore, instead of

ClassLoader.getSystemResource(...)

always use

this.getClass().getClassLoader().getResource(...)

Problem solved!


Maven – Curse or Blessing?

Today I spent several hours fighting a beast called Maven. I started using Maven some days ago for a new project. But each and every day I was asking myself the question: “Maven – curse or blessing?”. If you google around a litte bit, you can find lots of developers swearing in their blogs – and so was I today.

Everything was working nicely until I carelessly added several dependencies this morning. Suddenly, my Wicket-based webapp stopped working with some strange exception. A class in Jetty called a method that was not found in the servlet API (javax.servlet). Well, rather strange, I thought. After a little online-research I found that this method was introduced in version 2.5 of the servlet API. I doublechecked with the Jetty docs and yes, Jetty 6.1 implements exactly this version of the servlet API. Well … so what’s wrong?

But then I noticed that Maven added two different versions of the servlet API to my dependencies: 2.3 and 2.5. But isn’t Maven meant for resolving such conflicts? In deed it is and it actually wasn’t Maven’s fault. While Jetty comes with its own version of javax.servlet (called org.mortbay.jetty:servlet-api-2.5 in Maven), Maven added javax.servlet:servlet-api (version 2.3) as a transient dependency.

So how to get rid of unwanted dependencies? In Maven it is possible to exclude transient dependencies. The problem is that you have to do that for every package that depends on this unwanted package. You have two choices to find out which package depends on it: you either look at the POM of every dependency and the POMs of their dependencies … or you look at the dependency tree that can be generated with Maven’s dependency plugin (mvn dependency:tree). Well, the tree-task didn’t work for me, for whatever reason. I remembered that the dependency tree was also part of the project website that can be generated with Maven’s site plugin. I did generate the project’s website (mvn site) and there was also a dependency tree … but javax.servlet:servlet-api wan’t in that tree … but in the list of transient dependencies. If I remember correctly that was were I started swearing. So it was time to manually inspect all the POMs. (For the sake of completeness: I think that the servlet API wasn’t in the dependency tree as it only contains three levels of dependencies.)

I started analysing the POMs one-by-one and found out several interesting things:

  • Some POMs added junit as dependency without specifying the test-scope (it would therefore be added to the generated WAR file)
  • Optional dependencies and alternatives are not defined consistently: sometimes all optional dependencies are normal dependencies, sometimes none of the alternatives is a dependency, … altogether it’s quite messy

I finally found the dependency that added version 2.3 of the servlet-api to my dependencies: commons-logging. Yes, exactly that commons-logging that is basically meant to write some lines into a file. Why does it depend on the servlet API? Because it has its own logger that is supported by commons-logging and the commons-logging POM has dependencies on all supported logging implementations:

So you have to explicitly exclude the dependencies you don’t need … or you will deploy your desktop application with the servlet API. And the best thing of all: you have to do that for each and every dependency that introduces commons-logging to your project dependencies.

I then decided not only to exclude all unnecessary logging-frameworks but commons-logging all together in favour of slf4j. Unfortunately, it’s not possible to globally exclude a package all together – as Maven developers want to “protect” you from doing that – excluding commons-logging can get quite complicated. Erik van Oosten came up with a nice workaround for this topic. He announced a faked Maven repository that offers an empty jar file with version 99.0-does-not-exist for all artefacts you ask it for. So if you explicitly depend on this version, no lower versions will be added to your dependencies. And if you mark this dependency as provided, it also won’t be included in your WAR files. Yes, it’s a hack, but it’s a nice one ;)

Besides a small problem with empty jar files that Erik is going to fix (see my comment in Erik’s blog) the whole thing works like a charm – no commons-logging, no version conflicts with the servlet API, everything is fine.

I think I still can’t answer the question whether Maven is a curse or a blessing. The only thing I can say is that Maven has its problems, but also great power. I’d say it’s very important that you really go into the topic before you use it. Maven is big, *really* big. It’s not like Ant that you can start using after reading a short tutorial. If you don’t know how Maven works it will certainly kick your ass. But if you know how to handle that beast, it really kicks ass ;)

However, I am still not happy with the official Maven repositories. They are completely messy and Maven is not really helpful in coping with this mess. I would really love to see something similar to Debian package management: with provides, depends, suggests, recommends, meta-packages, nice updates … and a repository that is under tight control. Maybe even the new Ant subproject Ivy is a step into the right direction (I would love to see it in action together with Maven). But for the moment the only thing you can do is to have a close look on each and every POM in your tree of dependencies. Yes, it is extensive work, but it’s the only way to reduce your dependencies to a minimum … And dear Maven developers: please don’t try to prevent us from doing evil things, as we might do them anyway (like using version 99.0-does-not exist). Just give as the freedom to do what we want to do.


Some wicket impressions

I already mentioned Apache Wicket in my first post and immediately received advice to not gloss “over a great framework too quickly”. This advice came from a guy named jonathan … coincidentally one of Wicket’s most ambitioned developers is Jonathan Locke, who’s always present in discussions about Wicket … so there may be a connection to that guy who left a comment to my first post :-) .

So I gave it a try. As the Wicket website suggests to use version 1.3.0beta, the first I did was to download it as maven dependency.

<groupId>org.apache.wicket</groupId>
<artifactId>wicket</artifactId>
<version>${wicket.version}</version>
</dependency>

<dependency>
<groupId>org.apache.wicket</groupId>
<artifactId>wicket-extensions</artifactId>
<version>${wicket.version}</version>
</dependency>

Then I took a look around to find some code snippets or examples that could give me a smooth start. The guestbook example on the website was the first a had a look at (well after the hello world, which doesn’t count :-) ). The guestbook was easy to understand and ran without problems …
Wicket’s AJAX support was the next thing I had a look at. I found some nice examples in the Wicket Library and on wicketstuff.org (in fact the same on both websites). And yes, it was the guestbook with added ajax behaviour which I tried first. I adapted it a bit to my needs and tried to run it. It ended up by throwing a MarkupNotFoundException which suggested to

Enable debug messages for org.apache.wicket.util.resource to get a list of all filenames tried

as Guestbook.html could not be found. I enabled debug messages by adding the following line to my log4j.properties file.

log4j.category.org.apache.wicket.util.resource=debug

The debug message I got was

[DEBUG] ResourceStreamLocator Attempting to locate resource 'org/example/wicket/Guestbook.html' on path [folders = [], webapppaths: []]

I thought “uh, there aren’t any webapppaths or folders in wicket’s path, gotta add some”. I then entered the path in the web.xml file as property of the Wicket filter. Well, the path was then inside the brackets, but the exception still got thrown.
Eventually I saw the (my) mistake: there was a <wicket:extend> tag in the markup html, which was completely new to me. Having read the wiki article about markup inheritance and having removed the tag, the Guestbook ran without problems. Yepp it was my mistake to copy-paste the thing and to overlook the tag, but couldn’t the error message be a bit better/concise?

Alastair Maw’s BeanEditPanel reminded me a lot of Tapestry’s BeanEditForm, but of course doesn’t have all the BeanEditForm’s functionality (yet). But it was a great starting point for me to get a bit more into wicket … I already extended the BeanEditPanel a bit … yes, to be honest, the more I work with Wicket, the more I like it. The use of plain html and wicket IDs is a great concept and I also like the ease of adding AJAX behaviour. Just have a look at the ajax examples.

A nice summary of wicket’s features can be found in Peter Thomas’s Blog. I also recommend to read Karthik Gurumurthy’s introduction.

The only complaints I have so far are that I could not find the source code, which would be quite comfortable to have attached to the jar, as I sometimes like to see what’s going on behind the curtain… and no I don’t want to check it out from the SVN :-) .
There’s also no link to the 1.3.0beta javadocs on the official website. I found the link in a newsgroup post.

What I’m going to find out next is how the Validators work in Wicket. In a example I found, a Validator was added to a text field for instance like that

TextField name = new TextField("name");
number.add(StringValidator.minimumLength(5));

But contrary to the javadocs, a method FormComponent.add(IValidator) is not available … which I found a bit strange …

That’s all for now, and comments, especially to the validator stuff, are always welcome ;-)