Friday, December 25, 2009

RunJS: GitHub, build options, features, file sizes

A few updates on RunJS, a JavaScript file/module loader (see the README for more documentation):
  • RunJS is now on GitHub
  • Plugins for RunJS are supported. i18n bundles have been pulled out as a plugin, and a new text plugin allows you to set text files (think HTML/XML/SVG files) as dependencies for a module. The plugin will use async XMLHttpRequest (XHR) to fetch those files and will pass the text of those files as an argument to a module's module definition function. The RunJS build system will then *inline* those text files with the module, so that the XHR calls be removed in deployed code, and allow cross-domain use of those text files.
  • Any function return type is allowed from the module definition function. Before only objects and functions were allowed and functions had to be called out in a special way. Now, that special call out is removed and any return type is allowed. The cost was an extra call, run.get() that needs to be used in circular dependency cases. See the Circular Dependencies section in the README.
  • The build system that comes with RunJS now supports build pragmas.
The build pragma support was used to build RunJS in a couple of different configurations. I am trying to get a handle on where the bulk of implementation lies, and what features add to its file size. Here is the breakdown (warning, Google Doc iframe inclusion, but interesting numbers inlined in this post after the iframe):

Let's look at the non-license sizes, since they give a better indication of code density. Google's Closure Compiler did the minification for this evaluation.

The normal config, with no plugins included (but with plugin support) is 7,970 bytes minified, 3,167 gzipped. Including both the i18n and text plugins with run.js bumps it up to 11,759 minified, 4,655 gzipped.

The interesting number for me is the version of run.js without plugin support, no run.modify, no multiversion support and no page load (run.ready/DOMContentLoaded callbacks). This version of run has just the following features:
  • support for the run() module format
  • nested dependency resolution
  • configure paths to modules
  • load just plain .js files that do not define run.js modules (scripts that do not call run(), for example jQuery, or plugins for jQuery).
That bare bones loader comes in at 5,086 minified and 2,204 gzipped. The one you should use, the one with the license, is 5,245 minified and 2,317 bytes gzipped. I need to work on the size of that license block!

That size could probably be brought down a tiny bit (probably reaching the 2,000 gzip size) if I were to really be aggressive and remove all context references, but that would be a mess to maintain and there would be no easy upgrade path to multiversion support.

I believe that is the lower limit a functional loader that does nested dependencies via run() module calls. I view run.ready/DOMContentLoaded support more of a necessity for a loader, so unless you already had an implementation for that, I suggest the version that has run.ready() support, which comes in (with license) at 5,867 minifed, 2,522 gzipped.

The nice thing about the build pragma setup for RunJS, you can upgrade run without having to change your code if you find you want more features, like plugin support, or i18n/text dependency support via plugins.

I am interested in trying to sell more front-end JavaScript toolkits on this loader. For some, I can see the bare-bones 2.3K gzipped loader a nice way to step into it, and then their users have the option to swap out a more powerful version via a different RunJS build output.

I have put up the different build outputs for 0.0.6 if you want to grab one of the minified versions and play with it. Here is the minimum set of compliance tests which use the smallest loader (no modify/plugins/page load/context support) mentioned above. See the README for documentation.

Right now I believe around 2KB gzipped is close to the lower bound for a stand-alone code loader in the browser. At least for a loader I would consider using: anything that uses XHR and eval are dead to me. Using plain script src="" tags helps the xdomain case, and just fits better with debugging. While Dojo has used an XHR-based loader for quite a while (and it will continue to be supported), it just does not work as well with the browser as a script-tag based loader. Any loader should also do nested dependency loading too -- if a module in a script has dependencies in other modules, be sure to evaluate the dependencies in the right order.

As a point of comparison, consider LABjs. I feel a kinship with the author of LABjs, Kyle Simpson, even though we have never talked. We are both focusing on efficient code loading in the browser. I recommend LABjs if it fits your style.

While LABjs does not quite do nested dependency resolution, it does something related where you can tell it to wait to load a script before continuing to load other scripts. LABjs is not trying to push a module format like run is, but targeted more at existing code that does not have the concept of a module format.

By the way, RunJS can also handle loading these types of files. Where LABjs has a wait() call for holding off loading scripts that depend on another script being loaded (like a framework), RunJS uses nested run calls.

Example from the LABjs page:


Equivalent example with RunJS:

run(["run", "framework.js"],
function(run) {
run("plugin.framework.js", "myplugin.framework.js"],
function() {

Taking the 1.0.2rc1 version of LABjs and using Closure Compiler on it (without the license) gives LABjs a size of 4,360 bytes minified and 2,170 gzipped. As a reminder, the equivalent RunJS file is 5,086 minified and 2,204 gzipped. I may be able to do better with making the structure of the RunJS code more amenable to minification, but the gzip sizes come up fairly close. I do not believe the code tricks I would do to help minification will help the gzip size any.

Both LABjs and RunJS end up around 2KB gzipped. So, about 2KB gzipped seems close to the lower limit on a standalone loader, one that uses script tags/plays nice with the browser and can do nested dependencies. I would like to be proven wrong though, and ideally by modifying RunJS to fit that lower limit. :) I am sure the code can be improved.

But remember the guidelines, no goofy XHR stuff/something that works well with the browser and can handle nested dependencies. No script tags with inlined source/eval tricks. Even though Firefox and WebKit make eval debugging easier, it is still not as nice as regular script src tags.

Irakli Gozalishvili believes web workers might help, but I do not see it. The workers are restricted to message passing, and anything interesting in a web browser will likely need to touch the DOM, so a web worker solution will just be another async-XHR-like approach, where you will need to eval the scripts or inline-script inject to get all the scripts for them to see each other and the DOM.

Irakli does have an async-XHR based loader for CommonJS modules. As of today, it comes in at 1,527 minified, 838 gzipped (license not included). But it uses XHR, so limited to the same domain as the page, and debugging support is just not as nice across browsers. It also uses CommonJS module syntax, but I have decided CommonJS modules do not play well out of the box in the browser, and I believe the format's "module", "exports", and "require.main" parts are unnecessary.

Thursday, December 10, 2009

Dojo 1.4 Favorite Features

Dojo 1.4 is out! There is a metric ton of changes. Here are some of my favorite things about the release. I focus mostly on Dojo Core, and mostly in the non-animation parts of it, so my list is skewed for that focus. However, there are lots of other changes, some in the animation functionality, and in Dijit and Dojox. Check out the 1.4 release notes to get a more complete picture.

One of the things I want to do for Dojo Core is to bring the DOM APIs, particularly the methods on dojo.NodeList (the return object for dojo.query() calls, Dojo's CSS selector method) more in-line with what is available in jQuery. jQuery has demonstrated that its APIs resonate strongly with developers. Where it makes sense and fits Dojo's philosophy, we should also provide those APIs, to make it easier for developers. These Dojo 1.4 changes reflect that goal:
  • dojo.ready(), just an alias for dojo.addOnLoad().
  • dojo.NodeList-traverse: A helper module that adds methods to dojo.NodeList. Its goal is to bring in some methods to NodeList that exist in jQuery for DOM traversal, specifically: children, closest, parent, parents, siblings, next, nextAll, prev, prevAll, andSelf, first, last, even, odd.
  • dojo.NodeList-manipulate: A helper module that adds methods to dojo.NodeList. Its goal is to bring in some methods to NodeList that exist in jQuery for DOM manipulation, specifically: innerHTML, html, text, val, append, appendTo, prepend, prependTo, after, insertAfter, before, insertBefore, remove, wrap, wrapAll, wrapInner, replaceWith, replaceAll, clone.
  • IO pipeline topics: get notifications of IO events via dojo.subscribe/dojo.publish. Handy for putting up a generic "loading" indicator when any sort of IO call happens. These topics are not strictly how jQuery exposes this functionality, but we can leverage the power of dojo.publish/subscribe to implement this feature.
Some other new Dojo Core 1.4 features that are really sweet:
  • dojo.cache(): allows you to reference external HTML files and use them as if they are strings. It is integrated into the build system, so you can avoid the XHR calls to get the external text files by just doing a build. No extra build option is needed. This is a great way to construct HTML -- by writing plain HTML instead of building awkward strings in code or using JS DOM-building calls, which can obscure what the HTML actually looks like.
  • dojo.position(): A faster, more understandable replacement for dojo.coords(). If you were using dojo.coords() before, odds are good that you probably want to switch to dojo.position(). Douglas Hays stepped up and put in this great new method.
  • dojo.declare(): It is faster and more robust. Many thanks to Eugene Lazutkin for doing this work. It took a lot of patience and perseverance to get this new version up to snuff and keep it backward compatible.
  • dojo.hash(): An easy way to set the URL hash (fragment ID) and to watch changes to the hash. This allows you to create pages that reflect the proper state as shown by the URL in the browser. This was a contribution from community member Rob Retchless and other IBM Jazz team members.
For the build system, support was added for Google's Closure Compiler, so you can experiment with using it for minifying your code. Right now we just support the "simple" minification done by Closure Compiler, not the advanced features.

It was a little while coming, but it is great to have Dojo 1.4 out. Thanks to the community for making the toolkit better!