Wicket Interface Speed-Up: Merging Resources for Fewer HTTP Requests
August 28th, 2008 by Stefan Fußenegger | Published in Wicket | 14 Comments
This entry is part of my “Wicket Interface Speed-Up” series.
Now that I spent some time on configuring client-side caching and resource versioning for aggressive caching, I am going to finalize this series with an article on merging of resources and some code that you can easily use right away to speed-up your own applications in minutes.
Merging several (e.g JS) files into a single, monolithic file is an efficient way of reducing the number of HTTP requests. In theory it’s quite simple: just copy and paste all files together and you’re done. But no, sorry, it’s not that easy. If you do so, you’ll run in a bunch of new problems:
- Resources won’t live next to their owning components, therefore
- Existing HeaderContributor.forCss(…) and .forJs(..) will stop working
- Any component will come with that big file, which makes reusing components less enjoyable.
These are the cons of merging resources – but the cons you’ll smile at after reading this tutorial. (If you have other points to add, feel free to use the comment form at the bottom)
It is possible to merge resources without changing the resources themselves and without changing any code of the components they belong to. All it takes is some mount-magic, that will look like this:
protected void init() { mountMergedSharedResource("style", "all.css", new Class[] {PanelOne.class, ComponentB.class, MyForm.class}, new String[] {"PanelOne.css", "ComponentB.css", "MyForm.css" }); }
This code will mount a resource at /style/all-[version].css (e.g. /css/all-1234.css) that has all the contents that would normally be at /resources/my.package.PanelOne/PanelOne.css, /resources/another.package.ComponentB/ComponentB.css” and /resources/yes.another.package.MyForm/MyForm.css.
Of course, the usual way of referencing resources from a component will keep working:
add(HeaderContributor.forCss(PanelOne.class, "PanelOne.css")); add(HeaderContributor.forCss(ComponentB.class, "ComponentB.css")); add(HeaderContributor.forCss(MyForm.class, "MyForm.css"));
Each header contributor will lead to this style sheet reference (of course only one, regardless of the number of added header contributors for those resources):
<link rel="stylesheet" type="text/css" href="css/all-1234.css" />
But what happens behind the scenes? I implemented a MergedResourceStream that aggregates the contents of any number of ResourceStreams into a normal Java String, i.e. the contents of all resources will be kept in memory. I implemented four different types of merged resources
- MergedResource
any file type, will be served uncompressed - CompressedMergedResource
any file type, will be served compressed - CompressedMergedJsResource
JavaScript files, will be served compressed and stripped using Wicket’s built in JavascriptStripper.stripCommentsAndWhitespace(..) - CompressedMergedCssResource
CSS files, will be served compressed and stripped using YUI Compressor.
Okay, now here is the code for all my loyal – aren’t you? 
– readers: merged-resources.tar.gz
UPDATE: commited my code to wicketstuff, get it from SVN: wicketstuff-merged-resources and wicketstuff-merged-resources-examples.
I created two projects. One is called merged-resources and can be added to your project using maven. The second one is called merged-resources-test with two simple test cases and a sample application. To add merged-resources to your project simply add the following to your list of dependencies:
<dependency> <groupid>at.molindo.wicket</groupid> <artifactid>merged-resources</artifactid> <version>1.3.4-SNAPSHOT</version> </dependency>
(Unfortunately, there’s no Maven repository available at the moment, so you’ll have to build it yourself: `mvn install`)
Here is how to use it:
protected void init() { IResourceVersionProvider p = new RevisionVersionProvider(); ResourceMountHelper h = new ResourceMountHelper(this, p); h.mountMergedSharedResource("style", "all.css", true, new Class[] {PanelOne.class, ComponentB.class, MyForm.class}, new String[] {"PanelOne.css", "ComponentB.css", "MyForm.css" }); h.mountMergedSharedResource("script", "all.js", true, new Class[] {PanelOne.class, ComponentB.class, MyForm.class}, new String[] {"PanelOne.js", "ComponentB.js", "MyForm.js" }); }
By the way, MergedResourceStream uses Wicket’s ModificationWatcher, so any changes you make at development time will be immediately available without restarting your application.
Have fun and thanks for reading!
UPDATE: There is an ongoing discussion on my “Wicket Interface Speed-Up” on Wicket’s mailing list.
August 28th, 2008 at 11:13 am (#)
Hi Stefan,great work you have done, and thanks for releasing the source code! You should really get this into public version control, maybe via wicketstuff. This would make it much easier for others to contribute (eg. me).The first issue I stumbled upon was your usage of Arrays.copyOf. Thats a JDK 1.6 method while my project requires 1.5. I’m not sure what the copy is good for anyway, unless for defensive programming. For now I just removed the copying.The more interesting issue: In your test project there is no shared base library (ala jquery.js). In my case I have several files which are shared across most pages, while I can specify only one file for each component. To make the test more realistic, the Homepage should include two scripts (library + page specific) and one of the components should include one additional library script as a companion to page specific code (eg. validation library + validation setup code for MyForm).The revision reading works basically well, though it would be nice to be able to use existing revisions. This is the start of jquery.js:(function(){/* * jQuery 1.2.6 – New Wave Javascript * * Copyright (c) 2008 John Resig (jquery.com) * Dual licensed under the MIT (MIT-LICENSE.txt) * and GPL (GPL-LICENSE.txt) licenses. * * $Date: 2008-05-24 14:22:17 -0400 (Sat, 24 May 2008) $ * $Rev: 5685 $ */Most jQuery plugins have the same header in the same format.Another versioning issue: I’ve got two files merged into one all-xxxx.js. Both include a revision number. The number of the first file is used in the path of the file. When changing the revision of the second file, the name isn’t changed, therefore the browser doesn’t know that it has to load the file again, breaking cache invalidation. I guess this is the most critical issue so far.Looking forward to your reply and further collaboration.
August 28th, 2008 at 1:49 pm (#)
Nevermind the “add multiple files to one page” stuff, I figured it out – just adding multiple files in a single scope works fine.I’ll be trying to resolve the other issues as well and let you know about my progress.
August 28th, 2008 at 2:04 pm (#)
Hi Jörn,I’m planning to make the released code a wicketstuff project. Currently, I’m waiting for commit access though.Arrays.copyOf: defensive programming in deed, remove it if needed.Shared base library: I’m not sure, if I understand to problem correctly. However, I should probably mention that you *must not* mount a resource with more than one merged resource. So don’t create merged resources “page specific”.Revision reading: Feel free to implement whatever you need. All you need is to provide your own implementation of IResourceVersionProvider. But yes, RevisionVersionProvider could be a little more flexible. However, it was only meant as an example implementaiton of IResourceVersionProvider.Versioning issue:If both resources are versioned, the greater version is used. By default, resource versions are mandatory. Missing resources will therefore cause a WicketRuntimeException at startup. This *should* therefore work. For further discussion, please join my thread on the Wicket mailing list:http://www.nabble.com/Discussion-on-%22Wicket-Interface-Speed-Up%22-td19197540.html
August 28th, 2008 at 2:06 pm (#)
I guess I figured out the versioning issue, too. So far I just hardcoded revisions… Once the files are actually under version control, the hightest revision number is always the one to put into the filename, and that already happens just fine. Therefore it also doesn’t make much sense to use eg. revision numbers from jQuery’s repository, as those aren’t consistent with the current project and any custom script in there.What I’m now looking into is a way to reduce the duplication between WicketApplication and the various components. I don’t want to repeat the same file enumerations in both places.
August 28th, 2008 at 3:16 pm (#)
what do you mean by duplication? you’ll always have to add all required resources to your components (using HeaderContributor.for…). All you have to do now is determine which resources could be merged. My strategy was to create merged CSS and JS files for my whole site (all.css, all.js) and for JS functions and styles only needed for authenticated users (user.css, user.js). I also merged some resources that are specific to some higher frequented pages. That’s it, took me about an hour to configure and test, works like a charm.
August 28th, 2008 at 4:22 pm (#)
I see. My approach differs somewhat, therefore I have 1:1 duplication based on your API.Instead of figuring out whats used where, I’m just bundling everything into a single file. With gzipping its about 60k, with minifying and maybe obfuscating (didn’t work for me so far) it should get down further. Those 60k are loaded just once, and no additional js file is loaded when further browsing the site. Page specific code is always executed based on existence of element ids, with id-selectors being extremely fast. In additiona, page specific files are very small anyway, so taking them out would save only a few kb from all.js.
August 31st, 2008 at 12:43 pm (#)
Wouldn’t this be better submitted to the Wicket team for inclusion in the core product? Sounds like an excellent contribution.
August 31st, 2008 at 1:55 pm (#)
I started a discussion on this topic at the wicket mailing list, see: http://www.nabble.com/Discussion-on-%22Wicket-Interface-Speed-Up%22-td19197540.htmlFurthermore, I submitted my code to wicketstuff.org’s SVN repository: https://wicket-stuff.svn.sourceforge.net/svnroot/wicket-stuff/trunk/wicketstuff-merged-resources/ and https://wicket-stuff.svn.sourceforge.net/svnroot/wicket-stuff/trunk/wicketstuff-merged-resources-examples/
August 11th, 2009 at 12:24 pm (#)
Where can I find documentation for the 3.0-SNAPSHOT? I’m trying to migrate from 1.3.6-SHNAPSHOT, after updating to Wicket 1.4.0.
August 11th, 2009 at 12:38 pm (#)
To answer this myself: http://wicketstuff.org/confluence/display/STUFFWIKI/wicketstuff-merged-resources
August 11th, 2009 at 12:39 pm (#)
Hi Joern,
the best information can be found in wicketstuff wiki: http://wicketstuff.org/confluence/display/STUFFWIKI/wicketstuff-merged-resources
There is a simple example how to use ResourceMount instead of ResourceMountHelper (just compare the two examples – they are both doing the same thing). If you have any further questions, feel free to contact me (I’ll send contact details by mail) and I’ll do my best to get you started.
And thanks for poking me. It just reminded me, that I planed to write a post on the changes.
Cheers
July 5th, 2011 at 7:59 am (#)
Hi Stefan,
Your project looks very interesting but is it still maintained? I noticed that WicketStuff is moved to GitHub, but your project does not seem to be there. It is still available at SourceForge so I will use that for now.
Thanks for your work!
-Stijn
July 5th, 2011 at 8:02 am (#)
Ok never mind, I was looking with my eyes closed!
It *is* at GitHub, here:
https://github.com/wicketstuff/wicketstuff-import-backup/tree/master/wicketstuff-merged-resources
https://github.com/wicketstuff/wicketstuff-import-backup/tree/master/wicketstuff-merged-resources-examples
July 6th, 2011 at 7:08 am (#)
Hi Stijn,
the project is still maintained. I moved it out the wicketstuff tree though (for various reasons):
https://github.com/molindo/wicketstuff-merged-resources
See README there for details and don’t hesitate to ask questions, open issues or fork
Cheers, Stefan