Section 6.5. On-Demand JavaScript | Ajax Design Patterns

6.5. On-Demand JavaScript

Behaviour, Bootstrap, CrossDomain, Dynamic, JavaScript, LazyLoading, OnDemand, ScriptTag

Figure 6-13. On-Demand JavaScript

6.5.1. Goal Story

Bill's logging into his bank web site. The login form comes up straight away, and as he types in his username and password, the browser is quietly downloading the JavaScript necessary for the rest of the application.

6.5.2. Problem

How can you deploy lots of JavaScript code?

6.5.3. Forces

Ajax Apps make heavy use of JavaScript. Richer browser behavior means bulkier JavaScript to download.
Downloading JavaScript has a performance impact. Interaction cannot fully begin until all initial JavaScript has been loaded.
Bandwidth is also a concern with heavy JavaScript content. Often, not all JavaScript that's loaded is actually used, leading to a waste of bandwidth.

6.5.4. Solution

Download and run JavaScript snippets. The initial page load includes some JavaScript code, whichamong other thingscontains the bootstrapping code necessary to pull down further JavaScript. There are two techniques available: Script Tag Creation and Service Eval. Each will now be described, following by three main applications of the pattern.

With Script Tag Creation, use DOM manipulation to inject a new script element into the page. Perhaps surprisingly, the effect will be exactly the same as if the <script> tag had been encountered on startup: the referenced JavaScript will be downloaded and executed automatically. The tag can be attached to either the head or the body, though the former is more common as that's where you'd usually find script tags:

   var head = document.getElementsByTagName("head")[0];   script = document.createElement('script');   script.id = 'importedScriptId';   script.type = 'text/javascript';   script.src = "http://path.to.javascript/file.js";   head.appendChild(script);

How will you know when the script has loaded? It seems browsers vary in their behavior here, with some loading the script synchronously and some not. In IE, you can be notified with a mechanism similar to XMLHttprequest's (http://www.xml.com/lpt/a/2005/11/09/fixing-ajax-xmlhttprequest-considered-harmful.htmlonreadystatechange). With other browsers, you might need to keep polling for some indication it's loaded (where the indication depends on what the script is meant to do). If you happen to control the server script, you could implement a notification mechanism; i.e., complete the script by calling back to a listener, if it exists. There's actually a promising proposal, JSONP (http://bob.pythonmac.org/archives/2005/12/05/remote-json-jsonp/), which aims to have a simple, flexible mechanism like this, standard across the entire industry.

Script Tag Creation doesn't let you retrieve any old text; it has to be valid JavaScript. And your script won't be able to read it directly because the browser will use it only for evaluation. So how can you get hold of some data from an external server? The most common way is for the remote JavaScript to assign a variable to the required data:

   var characters = new Array("Mario", "Sonic", "Lara");

The content is usually a data structure, just like a standard JSON Message, but it requires an assignment as well, in order to be referenced in the code. It could also be a function that returns the desired value. Either way, the browser code has to use whatever name is mentioned in the script, which isn't ideal. Again, the idea of a flexible script mechanism like J is worth considering, as it would let the caller decide on the variable name.

Service Eval is the other technique for On-Demand JavaScript, though not as prominent as Script Tag Creation for reasons we'll discuss later. Under Service Eval, a Web Service is called with a standard XMLHttpRequest Call, it outputs some JavaScript as response content, and the JavaScript is then executed with an eval( ) call. We're just inspecting the XMLHttpRequest Call's responseText property, which we could manipulate before evaluating, so the body doesn't have to be a complete, valid piece of JavaScript (unlike with Script Tag Creation).

Any code not inside a function will be executed immediately. To add new functionality for later on, the JavaScript can add directly to the current window or to an object that's already known to exist. For example, the following can be sent:

   self.changePassword = function(oldPassword, newpassword) {     ...   }

The XMLHttpRequest callback function just needs to treat the response as plain-text and pass it to eval( ). A warning: here again, asynchronicity rears its ugly head. You can't assume the new code will be available immediately after requesting it, so don't do this:

   if (!self.changePassword) {     requestPasswordModuleFromServer( );   }   changePassword(old, new) // Won't work the first time because                             // changePassword's not loaded yet.

Instead, you either need to make a synchronous call to the server, add a loop to keep checking for the new function, or explicitly make the call in the response handler.

On-Demand JavaScript has three distinct applications:

Lazy Loading: Defer loading of bulky JavaScript code until later on. Works with either On-Demand JavaScript technique (Service Eval or Script Tag Creation).
Behavior Message: Have the server respond with a kind of "Behavior Message," which dictates the browser's next action. Works with either On-Demand JavaScript technique (Service Eval or Script Tag Creation).
Cross-Domain Scripting: Using Script Tag Creation, bypass the standard "same-origin" policy that normally necessitates a Cross-Domain Proxy. Works only with Script Tag Creation.

Let's look at Lazy Loading first. Conventionally, best practice has been to avoid including JavaScript unobtrusivelyby including it in one or more script tags:

   <html>     <head>       <script type="text/javascript" src="/books/2/755/1/html/2/search.js"></script>       <script type="text/javascript" src="/books/2/755/1/html/2/validation.js"></script>       <script type="text/javascript" src="/books/2/755/1/html/2/visuals"></script>     </head>     ...   </html>

Lazy Loading builds on this approach to suggest just a minimal initialization module in the initial HTML:

   <html>     <head>       <script type="text/javascript" src="/books/2/755/1/html/2/init.js"></script>     </head>     ...   </html>

The initialization module declares whatever actions are required to start up the page and perhaps enough to cover typical usage. In addition, it must perform a bootstrapping function, pulling down new JavaScript on demand.

The second application, Behavior Message, is a variant of the HTML Message pattern. Whereas Lazy Loading sets up library code for ongoing use, Behavior Messages take some transient code and runs eval( ) on it immediatelythe script is not wrapped inside a function (although it may define some functions that it calls). Effectively, the browser is asking the server what to do next.

Cross-Domain Scripting is the third application of this pattern. script tags have always been able to include source JavaScript from external domains. The rule not only applies to static <script> tags, but also to dynamically created script tags as in the Script Tag Creation technique. Thus, unlike with XMLHttpRequest and IFrame, your script can directly access external content this way. And because the src property can be any URL, you can pass in arguments as CGI variables. The idea is becoming rather popular, with companies such as Yahoo! offering JavaScript APIs specifically for this approach (see "Real-World Examples," later in this section).

Running a script from an external domain can be useful when you trust it, and ideally control it. Other times, it's a definite security risk. Douglas Crockford, creator of JSON warns of the havoc an external script can wreak (http://www.mindsack.com/uxe/dynodes/) (emphasis mine):

That script can deliver the data, but it runs with the same authority as scripts on the base page, so it is able steal cookies or misuse the authorization of the user with the server. A rogue script can do destructive things to the relationship between the user and the base server.... The unrestricted script tag hack is the last big security hole in browsers. It cannot be easily fixed because the whole advertising infrastructure depends on the hole. Be very cautious.

6.5.5. Decisions

6.5.5.1. Will you use Service Eval or Script Tag Creation?

The choice is between Service Eval and Script Tag Creation depends on several factors. Service Eval has the following benefits over Script Tag Creation:

Being based on XMLHttpRequest, there's a standard mechanism for being notified when the script is ready, so there's no risk of calling functions that don't yet exist.
You get access to the raw script code.
There's more flexibility on the message format: you can, for example, send several JavaScript snippets inside different XML nodes, and have the browser script extract them out.

And Script Tag Creation has a couple of benefits over Service Eval:

You can load JavaScript from external domains. This is the only significant functional difference between the two styles.
The JavaScript will automatically be evaluated in much the same way as the JavaScript linked in the static HTML is evaluated when the <script> tag is first encountered. Thus, you don't have to explicitly add variables and functions to the document in order to use them later; you just declare them normally.

6.5.5.2. With Lazy Loading, how will you break modules down?

You'll need to decide how to carve up your JavaScript. Standard principles of software development apply: a module should be well-focused, and intermodule dependencies should be avoided where possible. In addition, there are web-specific concerns:

Ideally, any given module is either not used at all, or used in entirety. What you don't want is a 5000-line module that's downloaded to retrieve a 3-line function. It's a waste of bandwidth that defeats the main purpose of On-Demand JavaScript.
You need one or more modules present on startup. At least one is required to kick off further downloads as required.
Keep in mind that code will probably be cached locally. Alexander Kirk has done some experimentation with caching of On-Demand JavaScript (http://alexander.kirk.at/2005/10/11/caching-of-downloaded-code-testing-results/), and it turns out that all major browsers will cache code created with Script Tag Generation. You can usually ensure that responses from XMLHttpRequest are also cached.

6.5.5.3. With Lazy Loading, at what stage will the script download the JavaScript?

It's easiest to download the JavaScript just before it's required, but that's not always the best approach. There will be some delay in downloading the JavaScript, which means the user will be waiting around if you grab it at the last possible moment. When the user's actions or the system state suggest some JavaScript will soon be needed, consider downloading it immediately.

Predicting if JavaScript is needed is an example of Predictive Fetch (Chapter 13)grabbing something on the hunch that it might be needed. You need to make a trade-off about the likelihood it's required against the hassle caused if you hold off until later. For example, imagine you have some JavaScript to validate a completed form. If you wait until the end, you'll definitely not be wasting bandwidth, but it's at the expense of the user's satisfaction. You could download it when there's one field to go, or two fields, or when the form's first loaded. Each option increases the chance of a wasted download, but increases the chance of a smoother validation procedure.

6.5.6. Real-World Examples

6.5.6.1. MapBuilder

MapBuilder (http://mapbuilder.sourceforge.net) is a framework for mapping web sites (Figure 6-14). It uses On-Demand JavaScript to reduce the amount of code being downloaded. The application is based on a Model-View-Controller paradigm. Model information is declared in an XML file, along with the JavaScript required to load corresponding widgets. When the page starts up, only the required JavaScript is downloaded, instead of the entire code base. This is therefore a partial implementation of this pattern; it does ensure that only the minimal subset of JavaScript is ever used, but it doesn't load pieces of the code in a lazy, on-demand style.

Figure 6-14. MapBuilder

6.5.6.2. Delicious/Yahoo! APIs

Social bookmarking web site Delicious (http://del.icio.us), and its owner, Yahoo!, both offer JSON-based APIs, with appropriate hooks to allow direct access from the browser via Script Tag Creation. The Delicious API (http://del.icio.us/help/json) will create a new object with the result (or populate it if it's already present). The Yahoo! API (http://developer.yahoo.net/common/json.html) allows you to specify a callback function in your script that the JSON will be passed to.

6.5.6.3. Dojo packaging framework

Dojo (http://dojotoolkit.org/download) is a comprehensive framework aiming to simplify JavaScript development. As such, it provides a number of scripts, of which an individual project might only use a subset. To manage the scripts, there's a Java-like package system, which lets you pull in new JavaScript as required (http://dojo.jot.com/WikiHome/Documents/DojoPackageSystem). You need only include a single JavaScript file directly:

   <script type="text/javascript" src="/books/2/755/1/html/2//dojo/dojo.js"></script>

You then pull in packages on demand with the Dojo API:

   dojo.hostenv.moduleLoaded("dojo.aDojoPackage.*");

Running the above command will cause Dojo to automatically download the modules under dojo.aDojoPackage.^[*]

^[*] It's possible not all modules will be downloaded, because the precise set can be defined by the package author and can be made dependent on the calling environment (whether browser or command line).

6.5.6.4. JSAN import system

The JavaScript Archive Network (JSAN) (http://openjsan.org) is an online repository of scripts. As well, it includes a library for importing JavaScript modules, also a convention similar to Java. The following call:

   JSAN.use('Module.To.Include');

is mapped to Module/To/Include.js. JSAN.includePath defines all the possible top-level directories where this path can reside. If the includePath is ["/","/js"], JSAN will look for /Module/To/Include and /js/Module/To/Include.

6.5.7. Code Example, AjaxPatterns On-Demand JavaScript Wiki

6.5.7.1. Introducing On-Demand JavaScript to the Wiki Demo

In the Basic Wiki Demo (http://ajaxify.com/run/wiki), all JavaScript is downloaded at once. But many times, users will only read from the wikiwhy download the code to write to it? So this demo refactors to On-Demand JavaScript in three stages:

The uploadMessage function is extracted to a second JavaScript file, upload.js. There's no On-Demand JavaScript yet because both files are included.
A further refactoring introduces On-Demand JavaScript by ensuring that upload.js file is only downloaded if and when an upload occurs. This version uses Script Tag Creation.
In yet another refactoring, the Script Tag Creation technique is replaced with Service Eval.

6.5.7.2. Separate JavaScript: Extracting upload.js

In the first refactoring (http://ajaxlocal/run/wiki/separateJS/), the upload function is simply moved to a separate JavaScript file. The initial HTML includes the new file:

   <script type="text/javascript" src="/books/2/755/1/html/2/wiki.js"></script>   <script type="text/javascript" src="/books/2/755/1/html/2/upload.js">

The new upload.js now contains the uploadMessage function. Reflecting the separation, a couple of parameters are introduced to help decouple the function from the main wiki script:

   function uploadMessages(pendingMessages, messageResponseHandler) {     ...   }

The calling code is almost the same as before:

   uploadMessages(pendingMessages, onMessagesLoaded);

We've thus far gained a bit of modularity, but don't have a boost to performance yet, since both files must be downloaded on startup.

6.5.7.3. Script Tag Creation On-Demand JavaScript

With On-Demand JavaScript, the upload.js is no longer required on startup, so its reference no longer appears in the initial HTML, leaving just the main module, wiki.js:

   <script type="text/javascript" src="/books/2/755/1/html/2/wiki.js"></script>

A new function has been added to download the script. To avoid downloading it multiple times, a guard clause checks if uploadMessages already exists, and if so, immediately returns. Following the Script Tag Creation technique, it adds a script element to the document's head, initialized with the upload.js URL and a standard JavaScript type attribute:

   function ensureUploadScriptIsLoaded( ) {     if (self.uploadMessages) { // Already exists       return;     }     var head = document.getElementsByTagName("head")[0];     script = document.createElement('script');     script.id = 'uploadScript';     script.type = 'text/javascript';     script.src = "upload.js";     head.appendChild(script);   }

The calling script just has to invoke this function. However, as mentioned earlier in the "Solution," it's possibly downloaded asynchronously, so a check must be made here too:

   ensureUploadScriptIsLoaded( );   if (self.uploadMessages) { // If not loaded yet, wait for next sync     uploadMessages(pendingMessages, onMessagesLoaded);     ....

If scripts are loaded asynchronously by the browser, the test will actually fail the first time, because the download function will return before the script's been downloaded. But in this particular application, that doesn't actually matter muchthe whole synchronization sequence is run every five seconds anyway. If the upload function isn't there yet, it should be there in five seconds, and that's fine for our purposes.

6.5.7.4. Service Eval On-Demand JavaScript

The Script Tag Creation code is refactored here to use an Service Eval instead, where an XMLHttpRequest Call retrieves upload.js and evals it. The initial scriptwiki.js differs only in its implementation of the JavaScript retrieval. The text response is simply passed to eval:

   function ensureUploadScriptIsLoaded( ) {     if (self.uploadMessages) { // Already exists       return;     }     ajaxCaller.getPlainText("upload.js", function(jsText) { eval(jsText); });   }

The JavaScript response must also change. If it simply defined a global function like before; i.e.:

   function uploadMessages(pendingMessages, messageResponseHandler) {     ...   }

then the function would die when the evaluation completes. Instead, we can achieve the same effect by declaring the function as follows (this will attach the function to the current window, so it will live on after the script is evaluated).

   uploadMessages = function(pendingMessages, messageResponseHandler) {     ...   }

6.5.8. Related Patterns

6.5.8.1. HTML Message

The Behavior Message usage of this pattern is a companion to HTML Message (Chapter 9) and follows a similar server-centric philosophy in which the server-side dynamically controls browser activity.

6.5.8.2. Predictive Fetch

Apply Predictive Fetch (Chapter 13) to On-Demand JavaScript by downloading JavaScript when you anticipate it will soon be required.

6.5.8.3. Multi-Stage Download

The Lazy Loading application of this pattern is a like Multi-Stage Download, which also defers downloading. The emphasis in Multi-Stage Download is downloading semantic and display content rather than downloading JavaScript. In addition, that pattern s more about downloading according to a prescheduled sequence rather than downloading on demand.

6.5.9. Want to Know More?

Dynamic Data Using the DOM and Remote Scripting, A Tutorial by Thomas Brattli (http://www.dhtmlcentral.com/tutorials/tutorials.asp?id=11)