So here you go: dojox.storage.encrypted, a Blowfish encrypted storage. It sits on top of dojox.storage, and you get all the dojo storage manager goodness, mainly the automatic selection of the best storage provider available. It exposes the complete API that dojox.storage does. If an attacker gains access to the storage area, he can still nuke the storage, but the data found within will be useless.

I’m not completely happy with it’s architecture, and I didn’t have the time yet to fully test it with all providers, but it works pretty well with localStorage and the performance is not too bad. If you have a spare minute and are interested, feel free to to test it yourself (ah I know, you don’t have the time yourself, but I’d be thankful!)…

Usage

Just use it as you would use dojox.storage, except that you need to set your passphrase before you start working with the storage:

dojo.require("dojox.storage.encrypted");
var sto = dojox.storage.encrypted;
sto.setPassphrase("my super secret passphrase");
sto.put('key','value');
var value = sto.get('key');

It uses/requires dojox.encoding.crypto.Blowfish but that’s been in dojo for ages, so no worries.

To use it, just put the encrypted.js file in your dojox/storage directory.

Files

the code (encrypted.js)
the test page

Uh, persistant local storage?

Ahm, yeah, in case you don’t know, that means a client-side storage, be it in a browser, or in an Air app. It’s not very popular, but that concept has been around for a long time. So why local? There are two major reasons for it: First, we need this for apps and tools that work offline – and apps and tools that work online but need an offline backup and sync later. Kris Zyp wrote a post about the JsonRestStore and OfflineRest back in 2008, he goes a little into detail there. Secondly, we need it for apps and tools that rely on persistence for other reasons, no matter if online or offline – like it happended to me. When I ran into Lawnchair, I had the idea to build a tool that was monitoring CSS selector usage on websites and apps (you know how you sometimes lose control about CSS selectors in webapps…). To achieve this, I needed to store data locally, and persistent.

What dojo offers

As I was using dojo’s excellent CSSRuleStore to find out what selectors where used, I wanted to switch from Lawnchair to a dojo solution. Dojo has dojox.storage (the above mentioned OfflineRest uses dojox.storage), which provides wrappers for the following backends: Gears, Flash, globalStorage and Adobe Air (globalStorage is Firefox’s early implementation of the early WhatWG storage draft). Hm. Not that much. Gears and Flash are 3rd party solutions, globalStorage is a Firefox-only-thingy (and old, though it’s still supported in current Versions) and Air is non-browser. That means: No IE without plugins. No Safari when Flash is not available.

What dojo does not offer (but should)

Guaranteed, reliable support for *any* browser. Without depending on 3rd party plugins. As of now, to have this, you’d need to use Lawnchair or persist,js, which is pretty complete. Speaking of it, let me quote from it’s readme (it’s a good read on local persistent storage in general):

The most notable attempt at addressing this problem [storing client-side persistent data] is probably Dojo Storage. Unfortunately, Dojo Storage does not support Internet Explorer without Flash, and it does not support Safari or other WebKit-based
browsers at all (at least, not without Flash). Also, Dojo Storage is
not standalone; it requires a several other Dojo components in order to
operate.

Whoa, even other JS tools mention the lack of support in dojo…

New providers

Attached to this post are wrappers for the following mechanisms:

• localStorage: this is the current spec, and it is supported by IE8, Firefox 3, Safari 4, Opera 10, Chrome 5.
• userData behavior: this is Microsofts client-side storage for IE < 8.
• cookie: worst case fallback solution…

With these three, there’s an almost 100% coverage of browser situations (see storage overview). There are still no-go scenarios, of course, such as IE7 with userData disabled and cookies disabled, but there’s no way to address them all.

Notes on BehaviorStorageProvider

This one uses the userData behavior, present in IE from version 5 up. It works with DOM elements that have a special behavior assigned. You can then use  set/get/removeAttribute() methods to store data. It has some limitations though – of course…

• Maximum size varies from 64k to 1M – depending on the zone the site is in. So, you can’t rely on more than 64k.
• UserData can be switched off in security settings.
• I couldn’t find a way to read out stored keys if you don’t know them. Maybe this is some security thingy or just a bug – but it means that some methods in the provider don’t work. (that is, getNamespaces, getKeys and clear). To work around this, we have to take some of the precious storage space and store the keys and namespaces seperately.

Usage

Copy the the providers into your /dojox/storage directory and add the needed dojo.requires to /dojox/storage/_common.js:

dojo.require("dojox.storage.LocalStorageProvider");
dojo.require("dojox.storage.GearsStorageProvider");
dojo.require("dojox.storage.BehaviorStorageProvider");
//>>excludeStart("offlineProfileExclude", kwArgs.dojoxStorageBuildOption == "offline");
dojo.require("dojox.storage.WhatWGStorageProvider");
dojo.require("dojox.storage.FlashStorageProvider");
//>>excludeEnd("offlineProfileExclude");
dojo.require("dojox.storage.CookieStorageProvider");

What’s left?

Well, it would be pretty cool to have a dojo.data API compliant store that uses dojox.storage as backend, so that you could write your apps using dojo.data calls and have a reliable offline backup. I started working on that, but then came Christmas, parents, vacation, new year and a lot of beer. But hey, it’s 2010 now, and there’s plenty of time left!

Storage Overview

Files

LocalStorageProvider.js
BehaviorStorageProvider.js
CookieStorageProvider.js
Storage Test Page

Besides indexOf(), there are some other ways to achieve this, so – let’s have a competition and find out who’s the fastest!

The Contestants

replace and length

One possibility is to replace the searched string with an empty string, read the length property of the modified string and compare it to the length of the haystack. If it’s different, the haystack contains the needle. Reading the length property is extra work, but comparing two integers is faster than comparing two strings, so maybe it’s faster in general.

replace

Again, replace the searched string with an empty string. Then compare the the modified string with the haystack. If they are different, the haystack contains the needle.

split

Take the haystack and try to split it using the needle as the seperator. If the result’s length is greater than one, the haystack contains the needle.

indexOf

Find the first occurance of needle in haystack; if the result is something else than -1, the haystack contains the needle.

Results

For testing, I did 10,000 iterations and retrieved the execution time in ms. The methods were tested in the order presented above.

On Chromium (Mac build, Version 4.0.203.0 here), replace + length is slightly faster than replace, and both are faster than split. indexOf is by far the fastest.

1) true: 7 / false: 3.5
2) true: 7.5 / false: 3.5
3) true: 10 / false: 8
4) true: 2.5 / false: 3

Safari 4 has nearly the same results as Chromium, but the numbers tend to differ a lot from test to test.

1) true: 4-18, avg 10 / false: 2.5
2) true: 4-19, avg 10 / false: 3
3) true: 6-22. avg 13 / false: 6-23 avg. 16
4) true: 2 / false: 3

On Firefox 3.5, split a bit faster than the two replace methods, but indexOf is again by far the fastest.

1) true: 13 / false 10
2) true: 13 / false 10
3) true: 10 / false 9
4) true: 1.5 / false: 2

On IE 8 (run in a VM), both replace versions perform almost the same, and faster than split. Again, indexOf is fastest. Only 2 – 3 times faster than the replace methods, but still the fastest.

1) true: 35 / false: 25
2) true: 30 / false: 25
3) true: 60 / false: 50
4) true: 15 / false: 15

You can run the tests yourself, if you are interested, the test page is here.

Summary

The results are pretty obvious: indexOf() outperforms the other contestants. Which is not really a surprise, considering that whatever Javascript does during indexOf() –  it has also to do the same before being able to do a split() or replace(). So, the proposed way for a contains() method is the following:

dojo.string.contains = function(/* string */ needle, /* string */ haystack, /* bool */ caseInsensitive) {
    if(caseInsensitive) {
        needle = needle.toLowerCase();
        haystack = haystack.toLowerCase();
    }
    return haystack.indexOf(needle) !== -1;
}

If you want to use contains() in your code, just copy the above lines somewhere in your code. Again, don’t forget to dojo.require(”dojo.string”) before.

Or, if you want to have beginsWith(), endsWith() and contains() all-in-one, use this: dojo.string.addons.js

Consider the following code:

var nearbyZipcodes = dojo.filter(givenZipcodes,function(zipcode){
    return dojo.string.beginsWith(zipcode,'12');
});

No need to tell you what this does, right? So, let’s do this then! The only thing left is to think about performance: Is there a difference between String.substr() and String.substring()? And can we get faster than the two?

If we have very short needles to look for in our haystack, one could consider the following approach:

dojo.string.beginsWith = function(/* string */ needle, /* string */ haystack) {
    var i,
        len = needle.length;
    if(needle.length > haystack.length) {
        return false;
    }
    for(i = 0; i < len; i++) {
        if(needle.charAt(i) !== haystack.charAt(i)) {
            return false;
        }
    }
    return true;
}

For very short needles, or when we expect close to no hits, this might be faster. So I set up a test page and let the different methods run against each other. The results clearly spoke against the char iteration method: On Firefox, iteration was always slower, even when the iteration method could return false after the first character. And when it had to iterate more often, times went up (I somehow had in mind Tracemonkey was perfect for simple iterations – but in this case it won’t help). Only on Safari the iteration method could compete with native substr() / substring() – but was never significantly faster. So, we’ll stick to the native methods (there was no real difference between the two).

More convenience?

Depending on your datasource, you might want to trim the input. No problem, as we use dojo, we can use it’s super fast trim and end up with the following:

dojo.string.beginsWith = function(/* string */ needle, /* string */ haystack, /* bool */ trimBefore) {
    if(trimBefore) {
        needle = dojo.string.trim(needle)
    }
    if(needle.length > haystack.length) {
        return false;
    }
    return haystack.substr(0,needle.length) === needle;
}

So simple, so sweet.

You can run the tests for yourself, the page is located here: test_beginsWith.html

If you want to use beginsWith in your code, just put the lines above it anywhere in your code – but don’t forget to dojo.require(’dojo.string’) before.

As there is no such fieldset with dojo, I decided to write my own widget. While doing so, I learnt an interesting lesson…

First, I wrote it pretty simple, inheriting the widget from dijit._Widget and dijit._Templated. It had a toggle button next to its legend, which could be clicked. The toggle was a simple display none/block change.

But I wanted it to be accessible via the keyboard – so I added a tabIndex and a key handler. Next, I wanted it to be able to hold and parse widgets, so I inherited it from dijit.layout.ContentPane so I could make use of the content setter. Then I wanted it to animate smoothly, so I added animations, and so on… You get the point, I guess. Well, this went on for a while, until I made an unpleasant discovery: My fieldset was almost a clone of dijit.TitlePane. Ugh.

I had put all the time into creating a copy of something, that already existed.

So I inherited the fieldset widget from dijit.TitlePane and ended up with less than 50 lines of code. And it was a matter of minutes to do so. Ok, lesson learned: If you need something that doesn’t exist and you are going to code it – first check if there’s something that behaves similar to what you want. Or similar to what you might want in the future. If I had invested a little more time in thinking about the fieldset widget before starting to code, it would have been obvious that I wanted the above mentioned features one day. And I would have saved a lot of time.

Well, however, here it is: a collapsible fieldset, coming with all the ContentPane goodness (child widgets, remote loading,…).

You can create it via markup (I put it into a namespace called “app”; choose a namespace you like):

<fieldset dojotype="app.Fieldset">
    <legend>Legend</legend>
    Content
</fieldset>

or programmatically:

var fieldset = new app.Fieldset({legend:'dojo fieldset', id: 'someId'},dojo.create('div',{},dojo.body()));

The fieldset takes pretty much the same arguments as dijit.TitlePane. To change the legend, use dojo.attr('legend','Teh new legend'), to change the content dojo.attr('content',content) as usual. To stay close to TitlePane, I left _setTitleAttr() in it, but it is an alias for _setLegendAttr().

The code is simple (after I inherited from TitlePane), I only added a method to setup the legend tag and changed the methods that handle hover states, as the TitlePane’s methods paid no respect to the dijit’s base class. The fieldset is closed by default, but you can add open="true" to the markup (or add open: true to the constructor args) to make it start open. You can style it as you would style any other widget.

The testpage is over here: test_Fieldset.html
Code: Fieldset.js
Template: Fieldset.html

Update: If you have downloaded the code and intend to use it, please re-download it, as I updated a method and fixed a namespace confusion (and added doc).

It has become quiet here the last weeks – due to an awesome trip to Italy. Returning home, I decided to put Google Analytics on this page.

As I always hate waiting for the ga.js to load while visiting websites, I wanted a mechanism to delay the load of it. Why this is a good idea has been discussed often enough elsewhere, so we take it as fact. Peter Higgins once wrote such a wrapper for dojo, and as of dojo 1.3, it’s officially included in dojox (see Alex Russel’s post). But this Wordpress theme runs jQuery, and I didn’t want to have two frameworks in one website – so I searched for something that did the job for jQuery.

But I didn’t really find what I was looking for. Well, there are some jQuery plugins that provide functionality like I wanted. Some were overpowered, some poorly written – and in the end, they all relied on jQuery.getScript(). The problem is, that getScript() adds a random parameter to the GET call, so that the called script is never cached. While this may be useful for some situations, in this case it’s really annoying and undesireable. In addition, I liked the idea of dojox.analytics.Urchin to not really wait for the script to be loaded, but instead to go see every now and then if the _gat global variable is present, and to give up after some time in case something went wrong.

So I decided to “port” dojox.analytics.Urchin to jQuery. It works almost the same as it’s dojo origin, with two major differences: First, you can’t have several instances of it running, you call it directly via jQuery.urchin.load(args). Second, there’s an option waitForDOM, set to true as default, that delays the loading of ga.js until the DOM is ready. If you call jQuery.urchin.load() from within a jQuery(document).ready() block, set this to false.

Example? Here you go:

<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="jquery.urchin.js"></script>

<script type="text/javascript">
jQuery.urchin.load({acct:'UA-111111-1'});
</script>

Or, if you need more options:

<script type="text/javascript">
$(document).ready(function() {
    // do some stuff
    // ...
    // load ga.js
    $.urchin.load({
        acct:'2234-2342',
        timeout: 8000,
        waitForDOM: false
    });
});
</script>

You can get the file here: jquery.urchin.js

Uncompressed version: jquery.urchin.uncompressed.js

So I want to share today a very primitive static data handler that I use for quite some time now, and wich has always served me well. By static, I mean data that doesn’t change during runtime, like smtp server connection data, database connection data and the likes.

The idea is to have a seperate directory (named “static” in my case) with .data files in it. Every .data file should care about a certain aspect of static data needed in an application. This directory is then blocked via an .htaccess file, containing the single line:

# file: .htaccess
Deny From All

During the bootsrap process of your app, the collector is called to walk through that directory and gather the data from all the files:

// file: bootstrap.php
// load static data
YourProject_StaticData::getInstance()->collectStaticData();

These files are pretty much standard config files, so they are readable via php’s function parse_ini_file. That function has two major limitations: It cannot handle hierarchy beyond two levels and it breaks when it encounters a quote character in a value. These limitations are discussed elsewhere, but with this static data collector we can at least push the first one a level further away. A data file could look like this:

# file: mailconfig.data
[smtp]
host     = somehost.tld
auth     = true
username = user@somehost.tld
password = somepass
[lists]
whitelist = "goodone@example.com,admin@somehost.tld"
blacklist = "badone@example.com"
[options]
checkwhitelist = false
checkblacklist = true
# ...

To get the connection data to the applications’s smtp account, you’d simply call:

$smtpConfig = YourProject_StaticData::getInstance()->getSubSection('mailconfig','smtp');

Fairly easy to use, pretty and readable. And it also is very handy for something that *very* often annoys me when looking into some client’s code: establishing a connection to a database. To different databases in most cases, for development, testing and deployment. I’ve seen pretty awkward attempts to handle this – but it can be done with one line of code, regardless of where the code is running. Say, you use Doctrine as ORM for your database, and have a data file like the following, using host names as keys (besides all my love for the NoSQL movement, I run into a mysql database on nearly every project):

# file: dbms.data
project.local    = "mysql://user:pass@localhost/project_db"
test.project.tld = "mysql://testuser:testpass@project.tld/test_project_db"
project.tld      = "mysql://user:pass@project.tld/project_db"

you can easily connect to your database by defining a constant that contains the current host name (wich is a good idea, in general) and call

// file: bootstrap.php
//set up a connection
Doctrine_Manager::connection(
    YourProject_StaticData::getInstance()->getSubSection('dbms',HOST),
    'mainConnection'
)->setCharset("utf8");

Clean and short. And independant from your location – just see to it that your data file contains the right connection data for each host, and never touch that line of code in your bootstrapping process again.

The PHP class uses the singleton design pattern (as you already know from the code above), and is less than 100 lines of code in size. Including doc. It’s as simple as it is handy. And it can be easily extended – if the need arises, what never happened to me.

I hope you can find it as useful as I do!

You can grab the file over here: StaticData.php