Among other tests, the main thing I wanted to know is how long does it take to read or write 1000 entries from/to localStorage. The keys were of the form “key000″, the values “val000″, with increasing numbers. So the tests were using minimal data – only 6  bytes – but the results are still useful to get an idea of how it goes – and how the browsers perform compared to each other.

Desktop

All test were run on a Mac Mini. Opera and IE were tested on a virtual machine, so there might be an additional performance penalty based on virtualization. However, here are the numbers (average time needed to run the respective method 1000 times):

Safari 4:
- getItem: 1.4ms
- setItem: 1.7ms
- key: 0.8ms

Chrome:
- getItem: 240ms
- setIem: 280ms
- key: 230ms

Firefox:
- getItem: 77ms
- setItem: 2100ms
- key: 25ms

Opera:
- getItem: 19ms
- setItem: 20ms
- key: 18ms

IE8:
- getItem: 5ms
- setItem: 72ms
- key: 3.6ms

Mobile

As we can see, Safari is lightening fast. That made me want to run the tests on a mobile phone that has a modern Safari browser. I took an HTC Desire, wich comes with Android 2.1, and thus has a Safari capable of localStorage. Here are the results:

HTC Desire:
- getItem: 16.4ms
- setItem: 19.5ms
- key: 8.7ms

This is pretty amazing, considering that it is a phone. It still beats most of the browseres on the desktop.

If I find that test page again (uh, yeah, it’s lost somehow), I’ll run the tests again with newer browser releases and publish the link here, so you can run the tests yourself.

So here you go: dojox.storage.encrypted, a Blowfish encrypted storage. It sits on top of dojox.storage, and you get all the dojo storage manager goodness, mainly the automatic selection of the best storage provider available. It exposes the complete API that dojox.storage does. If an attacker gains access to the storage area, he can still nuke the storage, but the data found within will be useless.

I’m not completely happy with it’s architecture, and I didn’t have the time yet to fully test it with all providers, but it works pretty well with localStorage and the performance is not too bad. If you have a spare minute and are interested, feel free to to test it yourself (ah I know, you don’t have the time yourself, but I’d be thankful!)…

Usage

Just use it as you would use dojox.storage, except that you need to set your passphrase before you start working with the storage:

dojo.require("dojox.storage.encrypted");
var sto = dojox.storage.encrypted;
sto.setPassphrase("my super secret passphrase");
sto.put('key','value');
var value = sto.get('key');

It uses/requires dojox.encoding.crypto.Blowfish but that’s been in dojo for ages, so no worries.

To use it, just put the encrypted.js file in your dojox/storage directory.

Files

the code (encrypted.js)
the test page

Update II: Ben Hockey proposed to do the whole thing without using the pub/sub system. I for one think its a good idea, but in case you want the topic version, I’ll leave the test page and add another one with the topic-less version. Thanks Ben!

Update III: Dojo 1.5 introduces robust Promises with dojo.deferred as Krys Zyp explains in this SitePen post.

—

One of the things/ideas/concepts that I really like about CommonJS is Promises.

Promises? A brief explanation from the CommonJS API:

Promises provide a well-defined interface for interacting with an object that represents the result of an action that is performed asynchronously, and may or may not be finished at any given point in time.

To hear more about Promises, I highly recommend these two posts.

That concept is not entirely new. The dojo toolkit has dojo.Deferred, which does a similar job. You create a Deferred, add (multiple) callbacks and error handlers to it, and when it fires off, all your handlers get called. But dojo.Deferred is pretty heavyweight. It’s loads of code. And it works with .addCallback() and .addErrback() methods – not with .then() methods, which provide maximum readability.

The goal

What I want to have is the follwing code running (with dojo):

asyncComputeTheAnswerToEverything().
    then(addTwo).
    then(printResult, onError);

(This is the example code from the proposed CommonJS Promise API)

And I want the execution to continue right after doSomethingAsnyc() – the async call should still be non-blocking. The .then() methods should recieve the results of the previous part of the chain, but still be non-blocking. Luckily, dojo provides a tool that allows us to achieve this: the publish/subscribe system.

Let’s do it

When we have unique topics for each promise made, we can subscribe the .then() method to the current topic in the chain, and publish the next topic when the data handlers are done computing. Within the then() method we need to check if the data we recieve is an error or not, to determine wich handler we need to call. The spec also demands a progress handler – if we wanted to implement that, we needed to add another topic, but I left that out.

We only need to kick off  the whole thing when we create the first promise, then the rest takes care for itself. Including the changes proposed by Micheil, what you have to do is pretty trivial and you don’t have to care about unique topics or the pub/sub system at all. The asyncComputeTheAnswerToEverything() function mentioned above would look like the following:

var asyncComputeTheAnswerToEverything = function(){
	var promise = new dojox.Promise();
	window.setTimeout(function(){ // some async operation
		var data = 42;
		promise.emit(data);
	},2000);
	return promise;
};

You create a Promise and store a reference to it. You return the promise and use the reference to it to kick off the chain when the data is there. Everything else runs on it’s own now.
Go to the test page or theand check the source – it contains the Promise code and the example functions. Open the console and run testSpec(); to start off the example code from the CommonJS API proposal.

Usage

As a namespace to put the Promise into I chose dojox – but choose whatever floats your boat. You can just drop the promise code somewhere into your code, or, if want to dojo.require("dojox.Promise") it, create a file named Promise.js that contains the code and place a dojo.provide("dojox.Promise"); as the first line in it. Save the file in the dojox directory and you’re done. (Well, to be honest, this violates the dojox naming convention, but if its just in your project, it’s ok and it will work.)

Any questions left? Feel free to leave a comment, @ me at twitter or drop me a line.

References / Files / More

• CommonJS Promise API Proposal
• Post by Kris Zyp @ SitePen blog about CommonJS
• Post on www.michaelharrison.ws about promises and more
• the test page (using topics and pub/sub)
• the test page (w/o topics, recommended)

The Curve

Every dojo.Animation needs a curve. If you call animateProperty or it’s shorthand anim, you don’t have to provide the curve for yourself, dojo will do it for you – if you build an animation using new dojo.Animation(), you have to provide your own curve. You can think of the curve as of a line that the animation walks on. The line is already present before the animation starts to walk it, and it can always take a break and pause in the middle and eventually continue walking where it had stopped. To create a curve, you need to call new dojo._Line(start, end). In return, you get an object with three properties: end, start and a getValue() method.

var anim = new dojo.Animation({
	curve: new dojo._Line(0,100)
});

Easing

The curve will then be modified by a easing function. By default, this is the linear dojo._defaultEasing. Easing breathes a bit of life into the curve, you can say. Other properties of dojo.Animation that affect the curve are rate and duration. rate is the timespan between two frames. To calculate the targeted fps, use (1000/rate). By default, dojo aims at 50 fps.

var anim = new dojo.Animation({
	curve: new dojo._Line(0,100),
	easing: dojo.fx.easing.quadInOut
});

Waking Life

To get some numbers out of dojo.Animation, let’s let it go wild and play(). When using animateProperty or the likes, you just fire and forget. In our case, we need to get the numbers. To achieve this, we connect to our animation’s onAnimate event. This is fired on every frame of the animation. When your connected function is called, you get – who would have guessed – the current value of the curve. Now you have a function called at a given rate, for a given period of time and you get get a numeric value that is already calculated for you. Decent!

dojo.connect(anim,'onAnimate',function(curveValue){
	// do something
});

To see some curves in action, go to this test page, open the console and type play(1); and play(2); and check the source code to see what’s happening.

Multidimensional Curves

dojo._Line generates only one-dimensional curves. But, there is dojox.fx._core, which contains an (old) version of dojox.fx._Line, which is capable of creating multi-dimensional curves. To use it, you have to explicitely dojo.require("dojox.fx._core"). It’s a bit odd, but the _core file contains only the _Line, and it is not loaded when requiring “dojox.fx”. Well, whatever, you can pass arrays of numbers into dojox.fx._Line:

new dojox.fx._Line([0,2],[100,6]);

Now you have a multi-dimensional curve. Or, if you could say, you have two curves in one. You can (technically) pass an unlimited amount of numbers into dojox.fx._Line. When you hand this curve to a dojo.Animation, it will animate all dimensions of the curve. And it will modify all dimensions of the curve according to easing, rate and duration. If you connect a function to onAnimate, it will now recieve an array of numbers.

var anim = new dojo.Animation({
	curve: new dojox.fx._Line([0,2],[100,6]),
	easing: dojo.fx.easing.quadInOut
});

Head over to the test page again, and type play(3); into the console. Now, the width of the bars is also animated – by the same animation function that calculates the height of the bars.

What now?

If your mind is still not blown away with ideas about how to use this, let me give you a start:

• animate position, width and height of a browser window and use it as a screensaver for some unknowing, yet thankful visitor…
• take some CSS 2D and 3D transisitions/transforms and animate away…
• canvas! No need to say another word…
• Still too banal? Go WebGL! Grab some 3DContext and animate an evil ball of jelly jumping around!

In short: dojo’s animation system allows you to not only animate CSS properties, but gives you a good tool for  animations in general. You get the curves on the one hand, but you also get the “player” for these curves, including play/pause/stop/gotoPercent methods. And dojo.fx.easing has a lot of easing functions in stock.

Uh, persistant local storage?

Ahm, yeah, in case you don’t know, that means a client-side storage, be it in a browser, or in an Air app. It’s not very popular, but that concept has been around for a long time. So why local? There are two major reasons for it: First, we need this for apps and tools that work offline – and apps and tools that work online but need an offline backup and sync later. Kris Zyp wrote a post about the JsonRestStore and OfflineRest back in 2008, he goes a little into detail there. Secondly, we need it for apps and tools that rely on persistence for other reasons, no matter if online or offline – like it happended to me. When I ran into Lawnchair, I had the idea to build a tool that was monitoring CSS selector usage on websites and apps (you know how you sometimes lose control about CSS selectors in webapps…). To achieve this, I needed to store data locally, and persistent.

What dojo offers

As I was using dojo’s excellent CSSRuleStore to find out what selectors where used, I wanted to switch from Lawnchair to a dojo solution. Dojo has dojox.storage (the above mentioned OfflineRest uses dojox.storage), which provides wrappers for the following backends: Gears, Flash, globalStorage and Adobe Air (globalStorage is Firefox’s early implementation of the early WhatWG storage draft). Hm. Not that much. Gears and Flash are 3rd party solutions, globalStorage is a Firefox-only-thingy (and old, though it’s still supported in current Versions) and Air is non-browser. That means: No IE without plugins. No Safari when Flash is not available.

What dojo does not offer (but should)

Guaranteed, reliable support for *any* browser. Without depending on 3rd party plugins. As of now, to have this, you’d need to use Lawnchair or persist,js, which is pretty complete. Speaking of it, let me quote from it’s readme (it’s a good read on local persistent storage in general):

The most notable attempt at addressing this problem [storing client-side persistent data] is probably Dojo Storage. Unfortunately, Dojo Storage does not support Internet Explorer without Flash, and it does not support Safari or other WebKit-based
browsers at all (at least, not without Flash). Also, Dojo Storage is
not standalone; it requires a several other Dojo components in order to
operate.

Whoa, even other JS tools mention the lack of support in dojo…

New providers

Attached to this post are wrappers for the following mechanisms:

• localStorage: this is the current spec, and it is supported by IE8, Firefox 3, Safari 4, Opera 10, Chrome 5.
• userData behavior: this is Microsofts client-side storage for IE < 8.
• cookie: worst case fallback solution…

With these three, there’s an almost 100% coverage of browser situations (see storage overview). There are still no-go scenarios, of course, such as IE7 with userData disabled and cookies disabled, but there’s no way to address them all.

Notes on BehaviorStorageProvider

This one uses the userData behavior, present in IE from version 5 up. It works with DOM elements that have a special behavior assigned. You can then use  set/get/removeAttribute() methods to store data. It has some limitations though – of course…

• Maximum size varies from 64k to 1M – depending on the zone the site is in. So, you can’t rely on more than 64k.
• UserData can be switched off in security settings.
• I couldn’t find a way to read out stored keys if you don’t know them. Maybe this is some security thingy or just a bug – but it means that some methods in the provider don’t work. (that is, getNamespaces, getKeys and clear). To work around this, we have to take some of the precious storage space and store the keys and namespaces seperately.

Usage

Copy the the providers into your /dojox/storage directory and add the needed dojo.requires to /dojox/storage/_common.js:

dojo.require("dojox.storage.LocalStorageProvider");
dojo.require("dojox.storage.GearsStorageProvider");
dojo.require("dojox.storage.BehaviorStorageProvider");
//>>excludeStart("offlineProfileExclude", kwArgs.dojoxStorageBuildOption == "offline");
dojo.require("dojox.storage.WhatWGStorageProvider");
dojo.require("dojox.storage.FlashStorageProvider");
//>>excludeEnd("offlineProfileExclude");
dojo.require("dojox.storage.CookieStorageProvider");

What’s left?

Well, it would be pretty cool to have a dojo.data API compliant store that uses dojox.storage as backend, so that you could write your apps using dojo.data calls and have a reliable offline backup. I started working on that, but then came Christmas, parents, vacation, new year and a lot of beer. But hey, it’s 2010 now, and there’s plenty of time left!

Storage Overview

Files

LocalStorageProvider.js
BehaviorStorageProvider.js
CookieStorageProvider.js
Storage Test Page

Besides indexOf(), there are some other ways to achieve this, so – let’s have a competition and find out who’s the fastest!

The Contestants

replace and length

One possibility is to replace the searched string with an empty string, read the length property of the modified string and compare it to the length of the haystack. If it’s different, the haystack contains the needle. Reading the length property is extra work, but comparing two integers is faster than comparing two strings, so maybe it’s faster in general.

replace

Again, replace the searched string with an empty string. Then compare the the modified string with the haystack. If they are different, the haystack contains the needle.

split

Take the haystack and try to split it using the needle as the seperator. If the result’s length is greater than one, the haystack contains the needle.

indexOf

Find the first occurance of needle in haystack; if the result is something else than -1, the haystack contains the needle.

Results

For testing, I did 10,000 iterations and retrieved the execution time in ms. The methods were tested in the order presented above.

On Chromium (Mac build, Version 4.0.203.0 here), replace + length is slightly faster than replace, and both are faster than split. indexOf is by far the fastest.

1) true: 7 / false: 3.5
2) true: 7.5 / false: 3.5
3) true: 10 / false: 8
4) true: 2.5 / false: 3

Safari 4 has nearly the same results as Chromium, but the numbers tend to differ a lot from test to test.

1) true: 4-18, avg 10 / false: 2.5
2) true: 4-19, avg 10 / false: 3
3) true: 6-22. avg 13 / false: 6-23 avg. 16
4) true: 2 / false: 3

On Firefox 3.5, split a bit faster than the two replace methods, but indexOf is again by far the fastest.

1) true: 13 / false 10
2) true: 13 / false 10
3) true: 10 / false 9
4) true: 1.5 / false: 2

On IE 8 (run in a VM), both replace versions perform almost the same, and faster than split. Again, indexOf is fastest. Only 2 – 3 times faster than the replace methods, but still the fastest.

1) true: 35 / false: 25
2) true: 30 / false: 25
3) true: 60 / false: 50
4) true: 15 / false: 15

You can run the tests yourself, if you are interested, the test page is here.

Summary

The results are pretty obvious: indexOf() outperforms the other contestants. Which is not really a surprise, considering that whatever Javascript does during indexOf() –  it has also to do the same before being able to do a split() or replace(). So, the proposed way for a contains() method is the following:

dojo.string.contains = function(/* string */ needle, /* string */ haystack, /* bool */ caseInsensitive) {
    if(caseInsensitive) {
        needle = needle.toLowerCase();
        haystack = haystack.toLowerCase();
    }
    return haystack.indexOf(needle) !== -1;
}

If you want to use contains() in your code, just copy the above lines somewhere in your code. Again, don’t forget to dojo.require(”dojo.string”) before.

Or, if you want to have beginsWith(), endsWith() and contains() all-in-one, use this: dojo.string.addons.js

Consider the following code:

var nearbyZipcodes = dojo.filter(givenZipcodes,function(zipcode){
    return dojo.string.beginsWith(zipcode,'12');
});

No need to tell you what this does, right? So, let’s do this then! The only thing left is to think about performance: Is there a difference between String.substr() and String.substring()? And can we get faster than the two?

If we have very short needles to look for in our haystack, one could consider the following approach:

dojo.string.beginsWith = function(/* string */ needle, /* string */ haystack) {
    var i,
        len = needle.length;
    if(needle.length > haystack.length) {
        return false;
    }
    for(i = 0; i < len; i++) {
        if(needle.charAt(i) !== haystack.charAt(i)) {
            return false;
        }
    }
    return true;
}

For very short needles, or when we expect close to no hits, this might be faster. So I set up a test page and let the different methods run against each other. The results clearly spoke against the char iteration method: On Firefox, iteration was always slower, even when the iteration method could return false after the first character. And when it had to iterate more often, times went up (I somehow had in mind Tracemonkey was perfect for simple iterations – but in this case it won’t help). Only on Safari the iteration method could compete with native substr() / substring() – but was never significantly faster. So, we’ll stick to the native methods (there was no real difference between the two).

More convenience?

Depending on your datasource, you might want to trim the input. No problem, as we use dojo, we can use it’s super fast trim and end up with the following:

dojo.string.beginsWith = function(/* string */ needle, /* string */ haystack, /* bool */ trimBefore) {
    if(trimBefore) {
        needle = dojo.string.trim(needle)
    }
    if(needle.length > haystack.length) {
        return false;
    }
    return haystack.substr(0,needle.length) === needle;
}

So simple, so sweet.

You can run the tests for yourself, the page is located here: test_beginsWith.html

If you want to use beginsWith in your code, just put the lines above it anywhere in your code – but don’t forget to dojo.require(’dojo.string’) before.

As there is no such fieldset with dojo, I decided to write my own widget. While doing so, I learnt an interesting lesson…

First, I wrote it pretty simple, inheriting the widget from dijit._Widget and dijit._Templated. It had a toggle button next to its legend, which could be clicked. The toggle was a simple display none/block change.

But I wanted it to be accessible via the keyboard – so I added a tabIndex and a key handler. Next, I wanted it to be able to hold and parse widgets, so I inherited it from dijit.layout.ContentPane so I could make use of the content setter. Then I wanted it to animate smoothly, so I added animations, and so on… You get the point, I guess. Well, this went on for a while, until I made an unpleasant discovery: My fieldset was almost a clone of dijit.TitlePane. Ugh.

I had put all the time into creating a copy of something, that already existed.

So I inherited the fieldset widget from dijit.TitlePane and ended up with less than 50 lines of code. And it was a matter of minutes to do so. Ok, lesson learned: If you need something that doesn’t exist and you are going to code it – first check if there’s something that behaves similar to what you want. Or similar to what you might want in the future. If I had invested a little more time in thinking about the fieldset widget before starting to code, it would have been obvious that I wanted the above mentioned features one day. And I would have saved a lot of time.

Well, however, here it is: a collapsible fieldset, coming with all the ContentPane goodness (child widgets, remote loading,…).

You can create it via markup (I put it into a namespace called “app”; choose a namespace you like):

<fieldset dojotype="app.Fieldset">
    <legend>Legend</legend>
    Content
</fieldset>

or programmatically:

var fieldset = new app.Fieldset({legend:'dojo fieldset', id: 'someId'},dojo.create('div',{},dojo.body()));

The fieldset takes pretty much the same arguments as dijit.TitlePane. To change the legend, use dojo.attr('legend','Teh new legend'), to change the content dojo.attr('content',content) as usual. To stay close to TitlePane, I left _setTitleAttr() in it, but it is an alias for _setLegendAttr().

The code is simple (after I inherited from TitlePane), I only added a method to setup the legend tag and changed the methods that handle hover states, as the TitlePane’s methods paid no respect to the dijit’s base class. The fieldset is closed by default, but you can add open="true" to the markup (or add open: true to the constructor args) to make it start open. You can style it as you would style any other widget.

The testpage is over here: test_Fieldset.html
Code: Fieldset.js
Template: Fieldset.html

Update: If you have downloaded the code and intend to use it, please re-download it, as I updated a method and fixed a namespace confusion (and added doc).

So, here you go:

It’s pretty simple, really. What to do? Well, call window.open() while the popup blocker still allows it, and store a reference to the window object. Then, later, do with it what you want. That is, changing the location in most cases.

Huh, that’s all? No magic? Nope, sorry. The popup blocker’s rules cannot be bent.

Say, you have a connection like this (I stay with the example code of the prior post):

dojo.connect(dojo.byId('b9'),'onclick',function(evt){
    dojo.publish('/opener/callOpen');
});

Then call window.open in your subscription:

dojo.subscribe('/opener/callOpen',function() {
    var winRef = window.open();
    dojo.xhrGet({
        url: window.location.href,
        load: dojo.hitch(opener,function(){
            this.open(winRef);
        })
    });
});

And let your open() know there might be a window object coming in:

opener = {
    open: function(windowReference) {
        var newWin = windowReference || window.open();
        // see if we get a window object
        console.log(newWin);
    }
};

In case your new window will be open and visible for some time until you can work with it, don’t forget to let the user know that the window will get some content in a few, e.g. point the new window’s url to some nice “please wait” page.

Screenshot

One of the things that fascinated me most about computers since I was a teen, was their ability to simulate a three-dimensional space where one could move around in.

That’s why I was so intrigued when I read Jacob Seidelin’s post about “Creating pseudo 3D Games“. I had to play around with it, as I wanted to have mouse control in it, but performance was so bad that it was impossible. So I laid aside that idea. Eventually, he continued this project and used it for something pretty awesome called “WolfenFlickr“, where you can walk around and look at Flickr photos, attached to walls. As I said, pretty awesome.

Anyway, what I wanted was mouse navigation and strafe keys, as you have it in shooters – this kind of moving around always gives me the feeling of being “free“ in a way. Experiencing that freedom in a web page (w/o flash, of course) was still on my mind.

And the I stumbled upon Ben Joffe’s textured Canvascape 3D-Walker. The performance of the canvas element drawing the image slices was better than letting the browser render single elements for each slice – much better. In fact, the idea of mouse control immeadiately popped up again and I had to give it a try. And – it worked! Of course it has some drawbacks (that is, when the cursor leaves the browser window, there’s no more control…), but it gave me that feeling of freedom while moving around!

Fooling around with the canvas element was fun and tought me about it’s image methods. And I hope to find more time to take it further…

Of course there’s an aweful lot of work ahead if one would want this to be pretty (it’s all bricks now, no interaction, perspective issues, no smooth movement on strafe, and so on…) and performance is still an issue with high-res textures, but it is a lot of fun moving around.

Here’s what I added to Ben Joffe’s Walker:

• mouse control
• strafe movement
• walk and crouch
• high-res textures

I dropped IE-Support for this and disabled the mini map for performance reasons.

You’ll need Chrome/Chromium to see it in action in high details mode, but it’s definitely worth it!

Go give it try! Move the mouse cursor into the center of your browser window, press “M“ and enjoy!

PS: Chromium builds for Mac can be found here.

Note: The modifications are all just minor, all credits for this go out to Ben Joffe!