persistent local storage with dojo

Update: The providers attached to this post are part of dojox.storage now – no need to download them, just get a copy of dojo, and everything is there.

Update II: dojox.storage won’t make it into dojo 2.0 and is not compatible with the new dojo/store API. For future-safe persistent client-side storage, see Storehouse – Client Side Object Storage for the Dojo Store API


Last year, Brian Leroux‘s Lawnchair caught my interest – it is an easy and fast way to access local persistant local storage.

Uh, persistant local storage?

Ahm, yeah, in case you don’t know, that means a client-side storage, be it in a browser, or in an Air app. It’s not very popular, but that concept has been around for a long time. So why local? There are two major reasons for it: First, we need this for apps and tools that work offline – and apps and tools that work online but need an offline backup and sync later. Kris Zyp wrote a post about the JsonRestStore and OfflineRest back in 2008, he goes a little into detail there. Secondly, we need it for apps and tools that rely on persistence for other reasons, no matter if online or offline – like it happended to me. When I ran into Lawnchair, I had the idea to build a tool that was monitoring CSS selector usage on websites and apps (you know how you sometimes lose control about CSS selectors in webapps…). To achieve this, I needed to store data locally, and persistent.

What dojo offers

As I was using dojo’s excellent CSSRuleStore to find out what selectors where used, I wanted to switch from Lawnchair to a dojo solution. Dojo has dojox.storage (the above mentioned OfflineRest uses dojox.storage), which provides wrappers for the following backends: Gears, Flash, globalStorage and Adobe Air (globalStorage is Firefox’s early implementation of the early WhatWG storage draft). Hm. Not that much. Gears and Flash are 3rd party solutions, globalStorage is a Firefox-only-thingy (and old, though it’s still supported in current Versions) and Air is non-browser. That means: No IE without plugins. No Safari when Flash is not available.

What dojo does not offer (but should)

Guaranteed, reliable support for *any* browser. Without depending on 3rd party plugins. As of now, to have this, you’d need to use Lawnchair or persist,js, which is pretty complete. Speaking of it, let me quote from it’s readme (it’s a good read on local persistent storage in general):

The most notable attempt at addressing this problem [storing client-side persistent data] is probably Dojo Storage. Unfortunately, Dojo Storage does not support Internet Explorer without Flash, and it does not support Safari or other WebKit-based
browsers at all (at least, not without Flash). Also, Dojo Storage is
not standalone; it requires a several other Dojo components in order to
operate.

Whoa, even other JS tools mention the lack of support in dojo…

New providers

Attached to this post are wrappers for the following mechanisms:

  • localStorage: this is the current spec, and it is supported by IE8, Firefox 3, Safari 4, Opera 10, Chrome 5.
  • userData behavior: this is Microsofts client-side storage for IE < 8.
  • cookie: worst case fallback solution…

With these three, there’s an almost 100% coverage of browser situations (see storage overview). There are still no-go scenarios, of course, such as IE7 with userData disabled and cookies disabled, but there’s no way to address them all.

Notes on BehaviorStorageProvider

This one uses the userData behavior, present in IE from version 5 up. It works with DOM elements that have a special behavior assigned. You can then use  set/get/removeAttribute() methods to store data. It has some limitations though – of course…

  • Maximum size varies from 64k to 1M – depending on the zone the site is in. So, you can’t rely on more than 64k.
  • UserData can be switched off in security settings.
  • I couldn’t find a way to read out stored keys if you don’t know them. Maybe this is some security thingy or just a bug – but it means that some methods in the provider don’t work. (that is, getNamespaces, getKeys and clear). To work around this, we have to take some of the precious storage space and store the keys and namespaces seperately.

Usage

Copy the the providers into your /dojox/storage directory and add the needed dojo.requires to /dojox/storage/_common.js:

dojo.require("dojox.storage.LocalStorageProvider");
dojo.require("dojox.storage.GearsStorageProvider");
dojo.require("dojox.storage.BehaviorStorageProvider");
//>>excludeStart("offlineProfileExclude", kwArgs.dojoxStorageBuildOption == "offline");
dojo.require("dojox.storage.WhatWGStorageProvider");
dojo.require("dojox.storage.FlashStorageProvider");
//>>excludeEnd("offlineProfileExclude");
dojo.require("dojox.storage.CookieStorageProvider");

What’s left?

Well, it would be pretty cool to have a dojo.data API compliant store that uses dojox.storage as backend, so that you could write your apps using dojo.data calls and have a reliable offline backup. I started working on that, but then came Christmas, parents, vacation, new year and a lot of beer. But hey, it’s 2010 now, and there’s plenty of time left!

Storage Overview

IE 6 / IE7 BehaviorStorageProvider
FlashStorageProvider
GearsStorageProvider
CookieStorageProvider
IE 8 LocalStorageProvider
GearsStorageProvider
FlashStorageProvider
CookieStorageProvider
Safari 3 FlashStorageProvider
CookieStorageProvider
Safari 4 LocalStorageProvider
FlashStorageProvider
CookieStorageProvider
Chrome 5 LocalStorageProvider
FlashStorageProvider
CookieStorageProvider
Firefox 2 WhatWGStorageProvider (= globalStorage)
GearsStorageProvider
FlashStorageProvider
CookieStorageProvider
Firefox 3 LocalStorageProvider
GearsStorageProvider
FlashStorageProvider
CookieStorageProvider
Opera 10 LocalStorageProvider
FlashStorageProvider (?)
CookieStorageProvider

Files

LocalStorageProvider.js
BehaviorStorageProvider.js
CookieStorageProvider.js
Storage Test Page

8 Responses to persistent local storage with dojo


  1. That’s pretty awesome that you wrote so many new Dojo Storage providers! Did you submit them to the Dojo project? I’m sure they would be useful to other folks. Happy new year!

    Best,
    Brad Neuberg


  2. @Brad Neuberg

    Thanks, and a happy new year to you too!

    The localStorageProvider is already in the Bugtracker, and I’ll add the other two shortly.

  3. Mikael Morvan

    Thanks a lot for your new storage providers. I had a problem with IE 8 64 bits that doesn’t support Flash Storage.
    Your new storage saved my life ;))))


  4. @Mikael Morvan

    Great, I’m happy to hear so! And thanks for using the providers :)

  5. Pascal Robert

    Hi,

    Where is the WebKitSqliteProvider? Googling for it only returns 3 results, including this page.

    BTW, Chrome also supports sqlite storage.


  6. @Pascal Robert

    Sorry for causing confusion, but there is no WebKitSqlite provider. This is because all Sqlite implementations – including Safari’s and Chrome’s – work completely asynchronous. Dojox.storage, in contrast, provides synchronous read access to storage (methods like get, getAllKeys, etc.). I had plans to find a way to somehow still create a provider for Sqlite databases, but it would have been ugly to such an extent, that I decided to drop that plan. The only exception, however, is the Sqlite implementation offered by Google Gears which has synchronous read access.

    Sorry again for causing confusion on this. I will remove all references to a WebKitSqlite provider to avoid this in the future.


  1. [...] Dieser Eintrag wurde auf Twitter von Tobias Klipstein, Jens Arps erwähnt. Jens Arps sagte: Client-side storage: Adding providers for localStorage, userData behavior and Cookie to dojox.storage http://bit.ly/7rqfvl #dojo #storage [...]


  2. [...] currently offers support for Flash, Gears and globalStorage. But with the files you can grab at my previous post on storage, you gain access to localStorage, userData behavior and cookies. Working with dojox.storage is [...]

[Comments are automatically closed after 30 days.]

Comment via Google+