Molindo Techblog » Michael Sparer

Compass: Role based searching using CompassQueryFilter

Michael Sparer — Fri, 18 Dec 2009 15:14:37 +0000

While implementing a forum with wicket, spring, hibernate and compass for search, I recently ran into a problem: there are topics and posts that should only visible for some users. Say there’s a moderator forum where all content should only be visible for … well moderators .

We’re using role based authentication in our apps, where each User object has one role that can in turn imply other roles. I’m leaving out some method, but you can imagine the IRole interface like that:

public interface IRole {
public List getImpliedRoles();
 
public boolean hasRole(final IRole role);
}

we’re using an implementation that’s quite close to this:

public enum SessionRole implements IRole {
 
ANONYMOUS(),
USER(),
MODERATOR(USER),
ADMIN(MODERATOR);
 
private final List _impliedRoles;
 
private SessionRole() {
   _impliedRoles = Collections.emptyList();
}
 
private SessionRole(final IRole... impliedRoles) {
    _name = createName();
 
   final HashSet set = new HashSet();
   for (final IRole impliedRole : impliedRoles) {
   set.add(impliedRole);
   set.addAll(impliedRole.getImpliedRoles());
}
 
_impliedRoles = Collections.unmodifiableList(new ArrayList(set));
}
 
public List getImpliedRoles() {
return _impliedRoles;
}
 
public boolean hasRole(final IRole role) {
return this == role || _impliedRoles.contains(role);
}
 
}

And for the sake of completeness, here the indexed entity (only showing the relevant fields):

public class ForumEntry {
 
private String _headline;
 
private String _text;
 
private IRole _readRole;
 
}

The visibility of the forum wasn’t a problem at all – where it got complicated was when performing a search. Of course the lucene index contains all posts and thus the search runs over all entries and also returns all entries – leading to results showing up for regular users that should be for moderators’ eyes only.

The first ideas that came to my mind (and also the first I abandoned) were:

Performing a regular search without considering any roles at all, then sort out the ones the user isn’t allowed to see
Save all roles that are allowed to see the entry to the ForumEntry
Add a hook into the indexing process and save the moderator entries in an other index (probably the worst idea)

1. is quite bad as far as design is concerned and not really handy when it comes to paging through the results, 2. would bloat the index a bit and each IRole would not only have to know its implied roles, but also its inferring roles (the roles that are higher in hierarchy). E.g. USER would have to know that MODERATOR and ADMIN have at least the same rights. This might seem easy for this case but gets complicated with a more complex role structure. 3. no that’s really bad … we want to use the same index (and also the same UI) for all searches.

The solution is pretty simple to implement but was very hard to find: CompassQueryFilters. As the name suggests, a CompassQueryFilter is

A filter used to filter out query results.

For filtering my ForumEntry entities based on the user role, I use a boolean query builder and add a should-rule for each implied role and for the role itself. A should role for a boolean query applies when at least one of the rules applies to the matching hits. The code goes something like that:

CompassSession session = ...;
IRole userRole = ...;
 
final CompassQueryBuilder build = session.queryBuilder();
final CompassQueryStringBuilder queryStringBuilder = build.queryString(qry);
final CompassQuery q = queryStringBuilder.toQuery();
final CompassQueryFilterBuilder queryFilterBuilder = session.queryFilterBuilder();
final CompassQueryBuilder.CompassBooleanQueryBuilder booleanQueryBuilder = build.bool();
 
if (userRole.getImpliedRoles() != null) {
for (final IRole r : userRole.getImpliedRoles()) {
booleanQueryBuilder.addShould(build.term("readRole", r.toString().toLowerCase()));
}
}
booleanQueryBuilder.addShould(build.term("readRole", userRole.toString().toLowerCase()));
 
final CompassQueryFilter queryFilter = filterBuilder.query(booleanQueryBuilder.toQuery());
q.setFilter(queryFilter);

One last caveat: the indexed IRole field of ForumEntry cannot be empty or null, the whole filtering process would then yield wrong results. A simple solution is to add a base role to the existing roles which every role implies, something like that:

public enum SessionRole implements IRole {
 
DEFAULT(),
ANONYMOUS(DEFAULT),
USER(DEFAULT),
MODERATOR(USER),
ADMIN(MODERATOR);
 
}

Wicket: A Neat Url Encoding Strategy and some basic SEO

Michael Sparer — Tue, 09 Dec 2008 09:03:00 +0000

After a year of working extensively with Apache Wicket I want to share some stuff I’ve been using over and over again and might be useful for you as well. In this first post it’s a custom URL Encoding Strategy and some related basic SEO.

Pages often depend on entities – e.g. a Customer or a product and as those pages should be bookmarkable as well, there has to be a unique way to tell the page which entity to use. As entities either don’t have unique names or one wants to avoid having an (database) index over the name column, the index-ID is for most applications the way to go. As it’s usually undesireable to expose the database ID of an entity in the URL, it has to be encoded and decoded somehow to get from myshop.com/customer/12 to myshop.com/customer/ea34cff … or something like that.

Using a custom Url-mounting strategy makes it a breeze:

public abstract class IndexedIdEncryptingUrlCodingStrategy extends LowercaseOnlyIndexedUrlEncodingStrategy {
 
private final int[] _encryptedParams;
// serverroot is e.g. http://www.mypage.com - required for 301 redirects in baseclass
public IndexedIdEncryptingUrlCodingStrategy(final String mountPath, final String serverRoot, final Class bookmarkablePageClass, final int... encryptedParmas) {
super(mountPath, serverRoot, bookmarkablePageClass);
_encryptedParams = encryptedParmas;
}
 
public IndexedIdEncryptingUrlCodingStrategy(final String mountPath, final String serverRoot, final Class bookmarkablePageClass, final String pageMapName, final int... encryptedParmas) {
super(mountPath, serverRoot, bookmarkablePageClass);
_encryptedParams = encryptedParmas;
}
 
@SuppressWarnings("unchecked")
@Override
protected void appendParameters(final AppendingStringBuffer url, final Map params) {
for (final int index : _encryptedParams) {
final String indexStr = Integer.toString(index);
final Object val = params.get(indexStr);
if (val != null &amp;&amp; val instanceof Integer) {
params.put(indexStr, encodeId((Integer) val));
}
}
super.appendParameters(url, params);
}
 
@Override
protected ValueMap onDecodeParameters(final String urlFragment, final ValueMap urlParameters) {
 
for (final int index : _encryptedParams) {
final String indexStr = Integer.toString(index);
final Object val = urlParameters.get(indexStr);
if (val != null &amp;&amp; val instanceof String) {
urlParameters.put(indexStr, decodeId((String) val));
}
}
 
return urlParameters;
}
 
protected int[] getEncryptedParams() {
return _encryptedParams;
}
 
protected abstract String encodeId(Integer val);
 
protected abstract Integer decodeId(String val);
 
}

This class’s baseclass is used for SEO only: it just checks if there are only lowercase letters in the URL and does a 301 redirect to the lowercase variant of the page if not.
Why lowercase only: The first of the Ten SEO Commandments (if there was such a thing) is: “Thou shalt only have one URL per page” – and for bots http://www.myshop.com/customer/Foobar and http://www.myshop.com/customer/fOoBar are two different URLs – which is a bad thing as far as duplicate content is concerned. Well, it could also be uppercase only or camelcase only – but lowercase makes it easier to read for humans and is also easier to implement than camelcase.

Then just mount the page like that:

mount(new IndexedIdEncryptingUrlCodingStrategy("/customer", "http://www.mypage.com", HomePage.class, "id"));

and construct the page params like that:

final PageParameters params = new PageParameters();
params.add("id", myEntity.getId());
 
// add e.g. a link
add(new BookmarkablePageLink("link", HomePage.class, params));

Download the files:
IndexedIdEncryptingUrlCodingStrategy.java
LowercaseOnlyIndexedUrlEncodingStrategy.java

In my next post I’ll get a bit more into Wicket & SEO: Show a more SEO way to encode URLs, how to deal with the JSESSIONID and how to apply the code shown today on a Page. Stay tuned – and consider subscribing

PS: If “thou convet my code”: the code is public domain, feel free to do whatever you like to do with it. If you have any suggestions or implement enhancements, just post them in the comments.

Fun Friday Second Life

Michael Sparer — Fri, 25 Jul 2008 08:26:00 +0000

I found a link to some interesting college art work while browsing SEOMoz’s roundup thursday.
College students could gain an additional point in an exam if they could ‘answer’ the following question:

Choose a particular piece of web-related software, such as a particular web browser, operating system or text editor. Then draw a picture of what a typical user of that software looks like.

Stefan was sure he could have answered the exam question better than the rest … and for sure he did. but have a look for yourself.

A Custom reuseable LoadableDetachableModel for Spring/Hibernate

Michael Sparer — Wed, 21 May 2008 16:28:00 +0000

the annoyance

org.hibernate.LazyInitializationException - could not initialize proxy - the owning Session was closed org.hibernate.LazyInitializationException: could not initialize proxy - the owning Session was closed

the problem

Generally, detached objects in Hibernate are a bad thing when it comes to reusing objects between web-requests. Detached objects are objects that have been persistent but are not attached to a Hibernate session. They’re easily produced but can be a mess if it comes to Lazy-Loading or updating them. Just consider the following case

Foo foo = _myDAO.getFoo(); // attached object


add(new Link("fooLink", new Model(foo)) {
  protected void onClick() {
    Foo foo = (Foo) getModelObject(); // now it's detached as this is called in the next request
    Bar bar = foo.getBar(); // possible lazy-loading exception if bar gets loaded lazily

} }

the solution

First be sure to use the OpenSessionInViewFilter sothat a new hibernate session is opened when the request gets attached and closed when it gets detached.

Then the solution of course is a to use a LoadableDetachableModel that gets a fresh instance from the persistence-layer on each request. Could look like that:

IModel fooModel = new LoadableDetachableModel() {


  protected Object load() {
    return _myDAO.getFoo();
  }
}
add(new Link("fooLink", fooModel) {
  protected void onClick() {
    Foo foo = (Foo) getModelObject(); // attached as LDM has loaded a fresh instance
    Bar bar = foo.getBar(); // no lazy loading exception as still attached

} }

the improvement

LoadableDetachableModels are a nice thing but it’s quite a lot of work to inject your DAO (or service) and then create a new LDM for each of your objects in each and every one of your panels. so we came up with a more generic version of the LoadableDetachableModel. First, let your model-objects implement an Interface, say IPersistentObject with the single method

Serializable getId();

As your object are likely to have a primary key named id it’ll be no problem anyway. Then create the following class:

public class PersistentObjectModel extends LoadableDetachableModel {


  private static final long serialVersionUID = 1L;
  private final Class _clazz;
  private final Serializable _id;
  @SpringBean(name = "myDao")
  private IMyDao _myDao;
  @SuppressWarnings("unchecked")
  public PersistentObjectModel(final T object) {
    super(object);
           // object may be a hibernate proxy, so get the actual class
    if (object instanceof HibernateProxy) {
      _clazz = (Class) ((HibernateProxy) object).getHibernateLazyInitializer()
                                   .getImplementation().getClass();
    } else {
      _clazz = (Class) object.getClass();
    }
    _id = object.getId();
    // inject the bean
    InjectorHolder.getInjector().inject(this);
  }
  public PersistentObjectModel(final Class clazz, final Serializable id) {
    clazz = clazz;
    _id = id;
    // inject the bean
    InjectorHolder.getInjector().inject(this);
  }
  @Override
  protected T load() {
    // the DAO then simply passes the params to hibernate (or spring's hibernatetemplate
    return _myDao.getById(_clazz, _id);
  }
  @SuppressWarnings("unchecked")
  @Override
  public T getObject() {
    return (T) super.getObject();
  }

}

This makes working with hibernate objects really easy. There are two options:

Use the object e.g.
setModel(new PersistentObjectModel(foo));

Use only class and id.
This might happen e.g. if you pass your id using PageParameters (but better encrypt it )
setModel(new PersistentObjectModel(Foo.class, 1));

Hope that saved you some time writing boilerplate code have fun

Hibernate: could not read column value from result set?

Michael Sparer — Fri, 16 May 2008 13:26:00 +0000

I got this Error today after changing the primary key of my model objects from int to Serializable (as I wanted all of my objects, even the ones with a String primary key to use the same API, which was limited to int). But when I started the app I got the Error and the following Exception:

org.hibernate.type.SerializationException: could not deserialize
at org.hibernate.util.SerializationHelper.deserialize(SerializationHelper.java:217)
at org.hibernate.util.SerializationHelper.deserialize(SerializationHelper.java:240)
at org.hibernate.type.SerializableType.fromBytes(SerializableType.java:82)
at org.hibernate.type.SerializableType.get(SerializableType.java:39)
at org.hibernate.type.NullableType.nullSafeGet(NullableType.java:163)
at org.hibernate.type.NullableType.nullSafeGet(NullableType.java:154)
…..

I even got this one as cause:

Caused by: java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2279)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2748)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:780)
at java.io.ObjectInputStream.(ObjectInputStream.java:280)
….

EOF? wtf? Don’t get too confused with that EOF, the solution was simply to add the type of the id in the mapping file.

Before:

After:

type=”integer”> hibernate type is –>

Wicket ChoiceRenderer for Enums

Michael Sparer — Thu, 10 Jan 2008 17:25:00 +0000

I now got to the point where I wanted to put some i18n into my DropDownChoices. What I found out pretty soon was that you need an IChoiceRenderer to customise your DropDownChoices. If you don’t use one, you’ll get by default short values and toString display names. E.g. the enum

enum { JOHN, PAUL, GEORGE, RINGO }

produces an output like that

But what I want is “John” instead of JOHN or “John Lennon, Rhythm Guitar” or anything but JOHN. On the mailinglist and in the wiki I discovered some complicated, overhead solutions using helper objects, so I decided to implement my own reusable enum-renderer that reads the display values from a properties file. All you have to do is to pass the Component that is responsible for the properties file.

public class EnumChoiceRenderer implements IChoiceRenderer {


private static final long serialVersionUID = 1L;
private final Component _resourceProvider;
public EnumChoiceRenderer(final Component resourceProvider) {
 _resourceProvider = resourceProvider;
}
public Object getDisplayValue(final Object object) {
 final Enum v = (Enum) object;
 return _resourceProvider.getString(v.name());
}
public String getIdValue(final Object object, final int index) {
 return Integer.toString(index);
}

}

Yeah that was quite easy, but maybe saves some of you the burdon of using helper-classes

Oh yes, the properties file looks something like that:
JOHN=John Lennon, Rhythm Guitar PAUL=Paul McCartney, Bass GEORGE=George Harrison, Lead Guitar RINGO=Ringo Starr, Drums
what then produces the desired output:

svn: Base checksum mismatch

Michael Sparer — Fri, 14 Dec 2007 13:27:00 +0000

Today I had difficulties commiting a source file as I got this error

svn: Checksum mismatch for /path/to/SourceFile.javaexpected: [checksum],actual: [checksum]

The problem occurred after moving a package into a different source-folder, with the mentioned file as new file in it. If you’re experiencing the same problem, just stop trying to fix it “the nice way”. Rather follow this easy workaround that suggests checking out the folder where the file lies into a temp-directory, then override your existing folder in your project with the newly checked out one.
It’s quick and dirty but it’s gonna save you a load of time

wicketstuff-push, bayeux and the memory leak

Michael Sparer — Thu, 15 Nov 2007 08:46:00 +0000

I’ve been fiddling around with reverse-ajax (or comet or pushlets) these days/weeks. You know, that stuff where the webserver keeps an open connection to the client and pushes new events to the client with just the transportation-delay – but no client interference. As described in my recent posts, I decided to use Apache Wicket for my webapp, so I looked around if there’s something in the wicketstuff project that fits my needs. And yes, there is the wicketstuff-push project that offers different implementations of server pushes. Xavier Hanin, one of the men behind Ivy, and Vincent Demay did great effort there to implement a timerbased PushImplementation and also a Cometd-based implementation of a event subscribe/publish mechanism. The latter uses an implementation of the Bayeux-spec and the Dojo-library together with wicketstuff-dojo to subscribe to channels and to process the pushes on client side.

Yesterday I took the time to have a closer look on how the Java implementation of the Bayeux protocol works and found out that each new client that subscribes to a channel gets stored in the Bayeux’s client-HashMap … and stays there. So apparently it is up to the user to remove the clients from there. Otherwise the HashMap would grow towards an OutOfMemoryError, which needed some 400.000 clients on my machine.
So how to get them out? Well it ain’t just that easy to say “on session unbinding I simply remove the client with the corresponding session id” as the client id the Bayeux protocol uses isn’t the http-session id. The client id also changes more frequently, a simple reload of the page is enough to have a new client id. So for my application, I subclassed CometdServlet (over which all request and thus all subscriptions are processed) and made a http-session – bayeux-client-session mapping. This means that I always know which session has which client id, and if the client-id differs, it simply gets removed from the Bayeux-map.

Problem solved

P.S.: Bayeux does the same with channels, so be careful with the amount of channels in your webapp
P.P.S.: Wicket’s first release candidate for version 1.3 is out *thumbs-up*

Some wicket impressions

Michael Sparer — Wed, 29 Aug 2007 12:47:00 +0000

I already mentioned Apache Wicket in my first post and immediately received advice to not gloss “over a great framework too quickly”. This advice came from a guy named jonathan … coincidentally one of Wicket’s most ambitioned developers is Jonathan Locke, who’s always present in discussions about Wicket … so there may be a connection to that guy who left a comment to my first post .

So I gave it a try. As the Wicket website suggests to use version 1.3.0beta, the first I did was to download it as maven dependency.


 org.apache.wicket
 wicket
 ${wicket.version}

org.apache.wicket wicket-extensions ${wicket.version}
Then I took a look around to find some code snippets or examples that could give me a smooth start. The guestbook example on the website was the first a had a look at (well after the hello world, which doesn’t count ). The guestbook was easy to understand and ran without problems …
Wicket’s AJAX support was the next thing I had a look at. I found some nice examples in the Wicket Library and on wicketstuff.org (in fact the same on both websites). And yes, it was the guestbook with added ajax behaviour which I tried first. I adapted it a bit to my needs and tried to run it. It ended up by throwing a MarkupNotFoundException which suggested to
Enable debug messages for org.apache.wicket.util.resource to get a list of all filenames tried
as Guestbook.html could not be found. I enabled debug messages by adding the following line to my log4j.properties file.
log4j.category.org.apache.wicket.util.resource=debug
The debug message I got was
[DEBUG] ResourceStreamLocator Attempting to locate resource 'org/example/wicket/Guestbook.html' on path [folders = [], webapppaths: []]
I thought “uh, there aren’t any webapppaths or folders in wicket’s path, gotta add some”. I then entered the path in the web.xml file as property of the Wicket filter. Well, the path was then inside the brackets, but the exception still got thrown.
Eventually I saw the (my) mistake: there was a tag in the markup html, which was completely new to me. Having read the wiki article about markup inheritance and having removed the tag, the Guestbook ran without problems. Yepp it was my mistake to copy-paste the thing and to overlook the tag, but couldn’t the error message be a bit better/concise?

Alastair Maw’s BeanEditPanel reminded me a lot of Tapestry’s BeanEditForm, but of course doesn’t have all the BeanEditForm’s functionality (yet). But it was a great starting point for me to get a bit more into wicket … I already extended the BeanEditPanel a bit … yes, to be honest, the more I work with Wicket, the more I like it. The use of plain html and wicket IDs is a great concept and I also like the ease of adding AJAX behaviour. Just have a look at the ajax examples.

A nice summary of wicket’s features can be found in Peter Thomas’s Blog. I also recommend to read Karthik Gurumurthy’s introduction.

The only complaints I have so far are that I could not find the source code, which would be quite comfortable to have attached to the jar, as I sometimes like to see what’s going on behind the curtain… and no I don’t want to check it out from the SVN .
There’s also no link to the 1.3.0beta javadocs on the official website. I found the link in a newsgroup post.

What I’m going to find out next is how the Validators work in Wicket. In a example I found, a Validator was added to a text field for instance like that
TextField name = new TextField("name"); number.add(StringValidator.minimumLength(5));
But contrary to the javadocs, a method FormComponent.add(IValidator) is not available … which I found a bit strange …

That’s all for now, and comments, especially to the validator stuff, are always welcome

Creating custom components with Tapestry 5

Michael Sparer — Mon, 27 Aug 2007 15:08:00 +0000

As far as I’m concerned, I consider creating custom, reusable components as the supreme discipline of every web-framework. I remember fiddling about a custom tabbed panel in JSF some years ago … those were the days when it wasn’t easy to find JSF third-party components. I also remember that it took me ages to add some sort of AJAX behaviour, and I can’t really remember if I finally succeeded But times have changed, creating custom components is part of every almost every framework’s ref-docs and it is … or should be … easy to create them. Ideally just with combining already existing components to new ones.

As already mentioned in my previous post, I’m playing around with Tapestry 5 these days. So I decided to create my own component, a guestbook component. The guestbook should be as generic as possible, meaning that all kinds of Java beans should be valid entries. I.e. you should not be dependent on some fixed values such as name, e-mail and the message … well that really called for a BeanEditForm to enter the values … and I thought about using some kind of Grid to display the guestbook entries.

As a complete newbie in creating tapestry components, the first way to go was the Tapestry wiki’s howto page for T5. There, Eric Vullings’s short introduction about creating custom components merely gave me a quick overview over the rendering phases to override/implement when creating a component.
Then I took a cursory glance at the BeanEditForm’s and Grid’s source code to see how the components work and how they could be extended/used for my custom guestbook component. I quickly realised that it takes a bit more than a cursory glance to get the hang of the Grid (including the GridRows, GridColumns and GridPager components) but it needs less effort to figure out how the BeanEditForm works.

I decided to take the BeanEditForm straight away as part of my component and to adapt some of the Grid’s functionality to display the entries. I soon found myself copying most of the Grid’s code …. that was the right time to stop and to think about my approach once again. I was already down in adding my own coercer to the AppModule which was a lot of fun but nevertheless copying code can’t be the solution .
So I (ab)used the Grid for my custom needs. To keep it short (sorry for those of you who didn’t get in touch with the Grid so far ), I let my Guestbook implement the GridModelProvider interface, subclassed GridRows in order to be able to use the a different template-html without losing the GridRows’s functionality and did the same with the GridPager component.
There it was, the Guestbook – a BeanEditForm above with GridRows and a GridPager below. All it needed now was to override the onSubmit method (or to use the @OnEvent(value = “submit”) annotation) to do with the new entry whatever I wanted.

But after all, it just didn’t feel right to abuse the Grid and to call that my “custom component”. I think I’ll never use it in a real-life application, but I got a good insight into how components work in Tapestry5 by playing with those two. Here my two pennies worth on the whole stuff:

The ‘mighty’ components in Tapestry5 seem to be hard to extend – maybe it is better to use them as they are, without trying to combine them to form a new component (which might be what the developers wanted). I believe that it wouldn’t be too difficult to create mighty components on your own, if you already created some small components. There were two little things I noticed during dealing with the GridPager component:

The GridPager component shouldn’t be limited to the Grid. In its current state it requires a GridDataSource object, which actually is the only link to the Grid. Basically it’s just a matter of a different naming. Thus creating a super-interface of GridDataSource (e.g. DataSource with the method getNrOfItems():int) would make the GridPager reusable for other components without having the Grid-stuff in it.

The CSS-class shouldn’t be hard-coded. Overriding the css would be a possibility, but it would look much neater to pass it as a parameter.

Well, that’s all for today. I’d love to hear about your experience with creating your custom components. So if you have something to say, or questions, just leave a comment below!

Unfortunately I did not find the time to look a bit more into Maven. Now even Tapestry creator Howard Lewis Ship swears like a trooper in his recent blog entry . I’ll definitely have to take a more thorough look into Maven …