Changes from 2.0 to 2.1 (Wicket 1.3.x):

• Annotation-based Mounting of Resources
• [WMR-7] Failure to load resources if this library is provided in a parent classloader
• [WMR-9] Make it possible to define some sort of order in the way files are included in the overall file
• [WMR-10] wicketstuff-merged-resources does not compile; missing jetty plugin artifact
• and a few minor bug fixes and performance improvements: See full FishEye Activity report

(Note: [WMR-12] and [WMR-14] already went into 2.2-SNAPSHOT)

Changes for 3.0 (Wicket 1.4.x):

• Annotation-based Mounting of Resources
• [WMR-6] Compatibility to wicket 1.4
• [WMR-7] Failure to load resources if this library is provided in a parent classloader
• [WMR-9] Make it possible to define some sort of order in the way files are included in the overall file
• [WMR-10] wicketstuff-merged-resources does not compile; missing jetty plugin artifact
• [WMR-12] Allow CSS media to be specified in ResourceMount, and honored by build()
• [WMR-14] Error detection for adding CSS and JavaScript resources to ResourceMount
• and a few minor bug fixes and performance improvements: See full FishEye Activity report

Downloads:

• wicketstuff-merged-resources 2.1: Jar Sources
• wicketstuff-merged-resources 3.0: Jar Sources

public interface IRole {
public List getImpliedRoles();
 
public boolean hasRole(final IRole role);
}

we’re using an implementation that’s quite close to this:

public enum SessionRole implements IRole {
 
ANONYMOUS(),
USER(),
MODERATOR(USER),
ADMIN(MODERATOR);
 
private final List _impliedRoles;
 
private SessionRole() {
   _impliedRoles = Collections.emptyList();
}
 
private SessionRole(final IRole... impliedRoles) {
    _name = createName();
 
   final HashSet set = new HashSet();
   for (final IRole impliedRole : impliedRoles) {
   set.add(impliedRole);
   set.addAll(impliedRole.getImpliedRoles());
}
 
_impliedRoles = Collections.unmodifiableList(new ArrayList(set));
}
 
public List getImpliedRoles() {
return _impliedRoles;
}
 
public boolean hasRole(final IRole role) {
return this == role || _impliedRoles.contains(role);
}
 
}

And for the sake of completeness, here the indexed entity (only showing the relevant fields):

public class ForumEntry {
 
private String _headline;
 
private String _text;
 
private IRole _readRole;
 
}

The visibility of the forum wasn’t a problem at all – where it got complicated was when performing a search. Of course the lucene index contains all posts and thus the search runs over all entries and also returns all entries – leading to results showing up for regular users that should be for moderators’ eyes only.

The first ideas that came to my mind (and also the first I abandoned) were:

1. Performing a regular search without considering any roles at all, then sort out the ones the user isn’t allowed to see
2. Save all roles that are allowed to see the entry to the ForumEntry
3. Add a hook into the indexing process and save the moderator entries in an other index (probably the worst idea)

1. is quite bad as far as design is concerned and not really handy when it comes to paging through the results, 2. would bloat the index a bit and each IRole would not only have to know its implied roles, but also its inferring roles (the roles that are higher in hierarchy). E.g. USER would have to know that MODERATOR and ADMIN have at least the same rights. This might seem easy for this case but gets complicated with a more complex role structure. 3. no that’s really bad … we want to use the same index (and also the same UI) for all searches.

The solution is pretty simple to implement but was very hard to find: CompassQueryFilters. As the name suggests, a CompassQueryFilter is

A filter used to filter out query results.

For filtering my ForumEntry entities based on the user role, I use a boolean query builder and add a should-rule for each implied role and for the role itself. A should role for a boolean query applies when at least one of the rules applies to the matching hits. The code goes something like that:

CompassSession session = ...;
IRole userRole = ...;
 
final CompassQueryBuilder build = session.queryBuilder();
final CompassQueryStringBuilder queryStringBuilder = build.queryString(qry);
final CompassQuery q = queryStringBuilder.toQuery();
final CompassQueryFilterBuilder queryFilterBuilder = session.queryFilterBuilder();
final CompassQueryBuilder.CompassBooleanQueryBuilder booleanQueryBuilder = build.bool();
 
if (userRole.getImpliedRoles() != null) {
for (final IRole r : userRole.getImpliedRoles()) {
booleanQueryBuilder.addShould(build.term("readRole", r.toString().toLowerCase()));
}
}
booleanQueryBuilder.addShould(build.term("readRole", userRole.toString().toLowerCase()));
 
final CompassQueryFilter queryFilter = filterBuilder.query(booleanQueryBuilder.toQuery());
q.setFilter(queryFilter);

One last caveat: the indexed IRole field of ForumEntry cannot be empty or null, the whole filtering process would then yield wrong results. A simple solution is to add a base role to the existing roles which every role implies, something like that:

public enum SessionRole implements IRole {
 
DEFAULT(),
ANONYMOUS(DEFAULT),
USER(DEFAULT),
MODERATOR(USER),
ADMIN(MODERATOR);
 
}

Note: the impatient may skip down to the list of requirements – but please let me give some background first

Yesterday, I was trying to make all our JUnit tests independent of the current content of a system’s temporary directory (“java.io.tmpdir”). It’s pretty simple to add “-Djava.io.tmpdir=`mktemp -d /tmp/junit.XXXX`” as a JVM parameter – at least as long as you don’t have to do it more than a few times. As I didn’t like the idea of adding this parameter to each and every invocation of our Maven build, I’ve abandoned this idea rather quickly. Instead, I was looking to set java.io.tmpdir within Maven. To do this, all it takes is adding the appropriate configuration to the maven-surefire-plugin:

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.4.3</version>
      <configuration>
        <systemProperties>
          <property>
            <name>java.io.tmpdir</name>
            <value>${java.io.tmpdir}${file.separator}surefire-${user.name}-${project.artifactId}</value>
          </property>
        </systemProperties>
      </configuration>
    </plugin>
  </plugins>
</build>

In my case, the resulting value of java.io.tmpdir was “/tmp/surefire-build-molindo-util-r4711″ (btw, “build.vcs.number” is set by TeamCity). Having this done, I’ve found out another thing: while the JVM does not guarantee that java.io.tmpdir exists quite a bunch of our tests assumed that java.io.tmpdir points to an existing directory. So the next step was refactoring some code:

// existing
File tmp = new File(System.getProperty("java.io.tmpdir"), "output.txt");
 
// new
File tmpDir = new File(System.getProperty("java.io.tmpdir"));
tmpDir.mkdirs();
File tmp = new File(tmpDir, "output.txt");

As I don’t like to repeat myself – who does? – I thought about creating a new utility method:

public static File getTmpDir() {
  File tmpDir = new File(System.getProperty("java.io.tmpdir"));
  tmpDir.mkdirs();
  return tmpDir;
}

Well, nothing spectacular here, most likely I wouldn’t be the first one to write this method. As a second though, I remembered that I had to lookup the property names for my Maven configuration: “java.io.tmpdir”, “file.separator”, “user.name”. Some of you might be able to name all of them instantly, but I’m admittedly not very good in remembering such things.  As a result, I decided to create a simple utility that should provide four things:

Requirements:

• simple and consistent access to Java system properties
• full documentation of available properties and their possible values – within my IDE (i.e. auto-completion, inline Javadoc)
• fix inconsistencies in returned values and/or naming
• make sure that java.io.tmpdir exists – acutally that’s the main reason why I’m doing all this

That’s why I came up with the following class: SystemProperty.java

It’s not complete yet, especially the documentation. Yet, I’ve only tested it with Sun Java 1.6 on Linux and Mac. To get full documentation of available properties and values, I’d like to ask you to run the included main method and send me the output (simply paste it to your favourite pastebin and post the link in the comments) along with your environment (OS, JVM version). In return, I’ll promise to update the class with all the gathered input and keep it available at the above URL.

Some highlights:

// Use toString() to get values:
// "/tmp/output.txt"
System.out.println(new File(SystemProperty.JAVA_IO_TMPDIR + "output.txt"));
// "You are running Linux, version 2.6.24-25-generic"
System.out.println(String.format("You are running %s, version %s", SystemProperty.OS_NAME, SystemProperty.OS_VERSION));
 
// get and set values
SystemProperty.JAVA_IO_TMPDIR.set("/dev/null");
 
// avoid setting of properties that shouldn't be changed
// throws UnsupportedOperationException
SystemProperty.JAVA_CLASS_PATH.set(newClassPath);

Additionally, access to properties is wrapped into a PrivilegedAction which is suggested as a best practice. If you know of any derived properties that would be cool to include, please let me now (e.g. Maven is providing non-standard os.family which I might include). Next, if you know of any properties that shouldn’t be read-only, let me know. Finally, if you know of any inconsistencies across different environments (e.g. like java.io.tmpdir that might or might not endswith a file seperator) – make a guess – let me know!

EDIT: I’ve also asked the people on Stack Overflow for collaboration. You may also add your output there.

I’ve spent quite some time looking for existing implementations of percentile calculations (e.g.  1, 2). All of them required storing response times for each and every request and calculate the percentile on demand or adding new response times in order. This was not what I wanted. I was hoping for a solution that would allow memory and CPU efficient live statistics for hundreds of thousands of requests. Storing response times for hundreds of thousands of requests and calculating the percentile on demand does neither sound CPU nor memory efficient.

Such a solution as I was hoping for simply seems not to exist. On second thought, I came up with another idea: For the type of performance evaluation I was looking for, it’s not necessary to get the exact percentile. An approximate answer like “the 95th percentile is between 850ms and 900ms” would totally suffice. Lowering the requirements this way makes an implementation extremely easy, especially if upper and lower borders for the possible results are known. For example, I’m not interested in response times higher than several seconds – they are extremely bad anyway, regardless of being 10 seconds or 15 seconds.

So here is the idea behind the implementation:

1. Define any random number of response time buckets (e.g. 0ms – 100ms, 100ms – 200ms, 200ms – 400ms, 400ms – 800ms,800ms – 1200ms, …)
2. Count number of responses and number of response each bucket (For a response time of 360ms, increment the counter for the 200ms – 400ms bucket)
3. Estimate the n-th percentile by summing counter for buckets until the sum exceeds n percent of the total

It’s that simple. And here is the code:  PercentileCounter.java and Percentile.java

Some highlights:

public void increment(final int millis) {
    final int i = index(millis);
    if (i < _limits.length) {
        _counts[i]++;
    }
    _total++;
}
 
public int estimatePercentile(final double percentile) {
    if (percentile < 0.0 || percentile > 100.0) {
        throw new IllegalArgumentException("percentile must be between 0.0 and 100.0, was " + percentile);
    }
 
    for (final Percentile p : this) {
        if (percentile - p.getPercentage() <= 0.0001) {
            return p.getLimit();
        }
    }
    return Integer.MAX_VALUE;
}

This approach only requires two int values (= 8 byte) per bucket, allowing to track 128 buckets with 1K of memory. More than sufficient for analysing response times of a web application using a granularity of 50ms). Additionally, for the sake of performance, I’ve intentionally implemented this without any synchronization(e.g. using AtomicIntegers), knowing that some increments might get lost.

By the way, using Google Charts and 60 percentile counters, I was able to create a nice graph out of one hour of collected response times:

@JsContribution
@CssContribution(media = "print")
@ResourceContribution(value = "accept.png", path = "/img/accept.png")
public class PanelOne extends Panel {
 
    public PanelOne(String id) {
        super(id);
        // ...
    }
}

The example above mounts and merges PanelOne.js (default is simple name of class + “.js”) into all.js (another default) and PanelOne-print.css into print.css which will be included with scope print. Additionally, accept.png will be mounted to /img/accept.png (as path starts with a ‘/’) which makes it easy to use from css files.

There’s not much left for this to work. All it takes is a single method call in your Application’s init code:

ResourceMount.mountAnnotatedPackageResources("/files", "com.example.components", this, mount);

In this example, all components in package com.example.components (including sub-packages) will be scanned for annotations. If not defined otherwise, all resources will be merged into single files for JS (/files/all.js) and all CSS media types (/files/all.css, /files/screen.css, /files/print.css, …).

As an added benefit, you’ll get all the other features of wicketstuff-merged-resources:

• merging of multiple files into one for less HTTP requests
• adding of versions to resource paths for aggressive caching
• pre-processing of resources (e.g. replacing colors in CSS files)
• optionally uploading them to Amazon Cloudfront (well, at least you can expect this feature soon – we are using it already)

So you will speed up rendering of your pages while simplifying and reducing your code (there’s no need to merge, mount or add HeaderContributors manually anymore)!

Further instructions are available on wicketstfuff-merged-resources’ page on Wicket Stuff Wiki.

(This new feature is mainly based on the idea and code of Jörn Zaefferer – Tanks!)

I’ve managed to get something working right now. It’s not completely tested but it should work quite well. What I’m doing is the following: I parse the charset files of MySQL (on an Ubuntu system, you can find them in /usr/share/mysql/charsets/) and do the collation based on those files myself rather than using Java’s built-in collations.

You may download the full source code.

The main class now is ResourceMount which replaces the deprecated ResourceMountHelper (already removed in trunk version). Mounting is now much simpler. Just see this example:

ResourceMount mount = new ResourceMount()
.setResourceVersionProvider(new RevisionVersionProvider())
.setPath("/style/all.css")
.addResourceSpecsMatchingSuffix(PanelOne.class, ComponentB.class, MyForm.class)
.mount(this);

Note that you still use the resources as you are used to, e.g.

add(HeaderContributor.forCss(MyForm.class, "MyForm.css"));
add(HeaderContributor.forJavaScript(MyForm.class, "MyForm.js"));

This example simply mounts PanelOne.css, ComponentB.css and MyForm.css into a single file at /style/all.css (the suffix “.css” is determined from the path – “/style/all.js” would mount all “.js” files). Additionally, the RevisionVersionProvider is used to determine which version to append (you may implement your own IResourceVersionProvider that best fits your environment). For example, the merged file may be mounted at /style/all-42.css (while still being available at /style/all.css through a permanent redirect).

While ResourceMount tries to autodetect most properties of the merged resource (e.g. style, locale, compression) it comes with a ton of customization options. For instance, it’s possible to use a custom IRequestTargetUrlCodingStrategy. Doesn’t that sound interesting to you? Well, we used this option to upload all resources to Amazon Simple Storage Service/CloudFront in order to transparently leverage Amazon’s CDN by simply mounting them in our applications (Isn’t that awesome? Stay tuned for another blog post. Donating might speed things up though).

Jörn Zaefferer suggested adding wicketstuff-annotations-like mounting of (merged) resources using annotations, which will hopefully be added in a later version. In the meantime, I may provide Jörn’s solution for those interested (Jörn, you don’t mind, do you?) – just leave a message.

Additional feature include CSS compression using Yahoo’s YUI Compressor and resource preprocessing. The later is particularly interesting, as you might implement things like replacing colors in CSS files or add internationalization from .properties files to JS files – just implement the IResourcePreProcessor interface (or use StringResourcePreProcessor if your only dealing with Strings).

I think I’ve mentioned the most important features now. Just see wicketstuff-merged-resources in WicketStuff’s Wiki for more (and possibly up-to-date) documentation. If you have any questions, don’t hesitate to post a comment! It you’re grateful, don’t hesitate to donate!

I suddenly had the feeling that ther’s something fishy with the maven-eclipse-plugin. And in deed, a look into the ~/.m2/repository/org/apache/maven/plugins/maven-eclipse-plugin proved me right. There is a brand new version 2.6 of the plugin.

This new version automagically adds projects from the workspace rather than JARs from the local repository (which is cool, basically). However, that caused Eclipse to load the wrong JARs: One project depended on sl4j 1.3, another one on 1.5 and the other project. Maven perfectly resolves the version of this transient depenedency, while Eclipse picks one by chance. Gotcha#1!

The first warning was caused by maven-compiler-plugin beeing configured to compile for Java 1.5 while only a 1.6 JDK was available. Gotcha#2!

Hopefully this post will save other people from being as clueless as I’ve been

public final Object writeReplace() throws ObjectStreamException {
  if (isTransient(this)) {
    return this;
  } else {
    return new ReplaceHolder(getClass(), getId());
  }
}

The ReplaceHolder – isn’t it a lovely name? – is responsible for replacing itself with the persistent object we didn’t want to serialize before:

public static final class ReplaceHolder implements Serializable {
 
  private static final long serialVersionUID = 1L;
 
  private final String _entityName;
  private final Serializable _id;
 
  public ReplaceHolder(final Class entity, final Serializable id) {
    _entityName = entity.getName();
    _id = id;
  }
 
  public Serializable getId() {
    return _id;
  }
 
  @SuppressWarnings("unchecked")
  public Class getEntity() throws ClassNotFoundException {
    return (Class) Class.forName(_entityName);
  }
 
  public final Object readResolve() throws ObjectStreamException {
    try {
      return loadEntity(getEntity(), getId());
    } catch (final ClassNotFoundException e) {
      throw new InvalidClassException(_entityName, e.getClass().getSimpleName() + ": " + e.getMessage());
    }
  }
}

And that’s it. Well, not completely, but see the attached .java file for details.

There is only one thing Wicket developers have to be aware of. Objects aren’t serialized between requests – don’t confuse it with model detaching. If I remember correctly, there is always one page kept detached but un-serialized. So if one would like to use the serialization, one has to make sure, that the objects are already serialized as soon as the models are detached. For this purpose, I implemented a special Model, that serializes its object on detach:

public void detach() {
  if (_object != null) {
    try {
      final ByteArrayOutputStream bs = new ByteArrayOutputStream();
      new ObjectOutputStream(bs).writeObject(_object);
      _bytes = bs.toByteArray();
      _object = null;
    } catch (final IOException e) {
      throw new WicketRuntimeException(e);
    }
  }
}

You might say that this feels like a hack – at least that’s what I think of it But it works perfectly, especially for complex forms where you add persistent objects to transient ones. Furthermore, you make sure that you’ll never serialize a huge persistent object graph into your Wicket session, which definetely is a Bad Thing ™.

EDIT: This blogger is thirsty! Please buy him a beer:

If it comes to new posts, We have a lot of ideas but not enough time to elaborate them all. Some ideas are:

• An easy to use Wicket component for advanced Google Analytics (Outgoing link tracking, search tracking, AJAX events tracking, user defined variables, …)
• Faster batch indexing for Compass
• Managing Subversion user access privileges with Drupal – and probably how to move to Wordpress afterwards
• Wicket Blueprint integration
• How to apply some on-site SEO with a little help of Wicket
• Unleashing the full power of Wicket with Cometd using wicketstuff-dojo-1.1 and loose coupling of components.

If you particulary like one of the ideas post a wish in the comments – and think about enforcing your wish with a donation: