Speed of JavaScript framework's DOM selection within mobile widgets [blog@stefankolb.de]

JavaScript frameworks - or libraries -, such as Dojo, ExtJS, jQuery, MooTools, Prototype, and YUI, - to mention only a few - are used widely throughout web applications nowadays. They are great for accomplishing common tasks while being easy to use - at least most of the time - for the experienced web/JavaScript developer.

Being involved with the Vodafone Widgets team for the last couple of weeks, I was curious to see how these JavaScript frameworks perform in regards to DOM selection on mobile devices. To find out, I created a mobile widget out of the Slickspeed Selectors Test and ran several hundred tests on the currently available mobile devices that support widgets and are approved by Vodafone.

Tested mobile phones

Currently, there are eleven (11) phones available that Vodafone approved for the Vodafone Apps Manager runtime. You can read more about the widget runtime at the Betavine/Vodafone Widget Zone. In short: It is a slightly modified version of the Opera Widget platform and is based on standardised web technologies.

The tested mobile phones are (in alphabetical order):

Nokia 6110 Communicator
Nokia 6124 classic
Nokia 6210 Navigator
Nokia E66
Nokia N73
Nokia N78
Nokia N85
Nokia N95
Nokia N96
Samsung SGH-i550
A phone that is currently not publicly available; therefore, I can not publish any test results on this phone

In general, every phone that runs on Symbian S60 v3.0 Feature Pack 1 is capable of running the Vodafone Apps Manager. But the mobile phones mentioned above are tested and approved by Vodafone.

Tested JavaScript frameworks

The tested JavaScript frameworks are (in alphabetical order):

Dojo v1.2.3
Dojo v1.3.0
jQuery v1.2.6
jQuery v1.3.2
MooTools v1.2.1
Prototype v1.6.0.3
YUI v2.7.0 Selector

How the tests were executed

To make use of the rendering engine that is available within the Vodafone Apps Manager runtime - instead of the phone’s native browser -, I wrote a widget that is based on the SlickSpeed Selectors Test created by the people at mootools.net. The code of the test suite is avilable at http://slickspeed.googlecode.com/svn/trunk/.

There were a couple of things I had to bear in mind. First, because of the limitations in screen size and resolution, the look of the test suite was slightly modified. Furthermore, only one framework at a time instead of several frameworks at once was loaded and tested. This was done because of the memory limitations some of the tested mobile devices have. There is simply not enough memory available to load all the frameworks at once. Additionally, I implemented a little menu to make it easier for me to choose from the available framework test (see screenshot 1 below).

The original SlickSpeed Selectors Test makes use of PHP to create the XHTML and parts of the JavaScript dynamically. Within the Vodafone Apps Manager, there is no PHP interpreter available. Therefore, to avoid any delays caused by network traffic while loading the files from a distant server, the code written in PHP was completely replaced by pure XHTML and JavaScript.

Every framework was tested five times on every mobile phone. The average of these five tests per framework and phone was then taken over into the test results. All in all, 385 tests were carried out.

Here is an example of how a test was carried out (the screenshots were taken from a Nokia N95):

First, the main menu is shown, in which one has to choose the framework they want to test. For this demo, the Dojo framework in version 1.3.0 was choosen (this choice was purely random).

Screenshot 1: Mobile Framework Tests - Menu

The (modified) SlickSpeed test suite is then loaded.

Screenshot 2: Mobile Framework Tests - Loaded SlickSpeed Suite

After a click on start tests the different DOM selection tests are preformed. This will take some time, depending on the mobile phone the test is run on.

Screenshot 3: Mobile Framework Tests - Tests are running

After all DOM selection tests are finished, the final test result is displayed at the bottom.

Screenshot 4: Mobile Framework Tests - Final result

Results

Below you will find several .pdf-files containing the results of the tests. I grouped the results by mobile phone and by framework.

Results by mobile phone

Results by framework

As you can see, the tests on a mobile phone take a lot more time than they do on a desktop computer. For example, the tests for jQuery v1.2.6 take 118ms on my MacBook 2.4Ghz Core 2 Duo with 2GB RAM, while they take about 3000ms on a Nokia N85. Always keep in mind that when you write your own widgets, processors installed into mobile phones are not as powerful as they are in desktop computers or laptops.

Final words

Of course, a lot more testing has to be done. But for now it is clearly visible that some frameworks perform better than others in terms of DOM selection. By far the slowest framework in my tests across all devices was the YUI v2.7.0 framework. The fastest frameworks were the two version of the Dojo framework, with version 1.3.0 performing slightly better than version 1.2.3.

It is also clear from the results that the performance depends on the mobile device. The Nokia N73 was the slowest phone, no matter which framework was tested on that device. The fastest phone was the Nokia E66, closely followed by the Nokia N85.

Hopefully, the tested - well established - web frameworks will soon be optimized to perform better on mobile phones. After that, I am sure, they will be of great value for the mobile widget developer, just like they are for web developers today.