Update: Stephan recommended to put the video on top,  so here it goes: Some impressions from RavenJS, enjoy (fullscreen and headphones recommended)!

The Beginning

When we started, I had little to no knowledge about working with 3D and OpenGL – I’m a JavaScript guy, after all. When I first fired up Blender and tried to work with it I was pretty close to dropping the whole idea again. Same happened when I saw how shader code looked like. But I so much wanted this – and the existing wrappers like GLGE or three.js enable us to still work with WebGL even if we don’t have the background knowledge.

So I decided to go with GLGE as I had already played around it with it’s examples before and because it already had nice collada support, which I decided to have as format for meshes and animations.
The first step was to extract some assets from the original game, collect mesh and texture data and get it into Blender. Luckily, Gothic has a highly active modding community, and this community is supported by the original game developers. So there were a couple of tools that could be used to aid in this, and it didn’t take long and we had a sword rotating in a browser.

But if it was possible to load a sword into GLGE, then shouldn’t it also be technically possible to load parts of the world into it? Yes, it was. It took some time, and it’s still far from perfect, but it worked. The next step was to create a first person camera that would react to mouse and key events – and had collision detection. Real collision detection, a height map would get us nowhere, as the levels had bridges and indoor locations, and I needed moving objects, too. Luckily, GLGE had some demos using raycasting to achieve this that I could use as start and modify until it worked as needed.

To recreate the dark intense atmosphere I played around with the right ambient lighting, fog types and added a sky box. Well, I would have loved to have a real sky dome, but, alas, I have no idea how to do this… Finally, I added objects, both static and animated.

In action

I can’t and won’t put online a live demo, as I obviously have no rights to do so. The copyright of the meshes and textures used belongs to the original developers, Piranha Bytes, and the RavenJS project is in now way endorsed, encouraged or supported by nor affiliated with them. But I can show you some impressions of what it looks like so far (fullscreen and headphones recommended):

Update: Video is now on top

(The song in this video is called “A special thing” by “amanyth” and published under a Creative Commons license.)

Issues

Besides the mere amount of free time that is needed and lacking, there are currently four main issues that are a bit of a show-stopper:

Animations caused a big deal of problems, and it’s still not prossible to run more than one animation sequence – running multiple animations will screw up bone position. While the animations still run fine in Blender, something goes wrong during the export. I’m still to find out if this can be overcome by using an alternate format for export.

Another thing that is still unsolved is “double-faced textures” that have transparent parts (look at the trees in the video). Those transparent parts become plain black when rendered in the browser. I’ve read up on this, and it seems that this scenario is indeed not trivial… From what I understood so far, in our case there’s two shapes “glued” to each other and each one has its own material, but I still have no idea what exactly goes wrong.

A real biggie, performance-wise, is that there is no logic yet that handles display of objects that are not visible. In the original game e.g., doors and the likes are “closed” by vertices with solid (non-transparent) textures and everything that’s behind it is taken out from the whole rendering/object interaction/whatever process. Only when you get really close you can see through these and see the intereour of the adjectant room. If these vertices get actually removed or if you “portal” through them I don’t know.

Mouse lock. Of course. No further explanation needed, we’re all looking forward to seeing this in the browser.

Future

I always loved 3D games. And the fact that it’s possible to render 3D worlds in a browser – with acceptable performance – is just stunning. The majority of work is still ahead, I’m well aware of that, but I *so much* want this to happen… So I just hope that some of the major problems can be solved and this project can go on!

During JSConf.eu, we released EmbedJS 0.2. That release was mainly about a fundamental architectural change: the move to the AMD pattern for our features. Let me explain why we decided to do this, why we think this is a big step forward and what this means for the user.

Modules

AMD stands for “Asynchronous Module Definition”. So, what are modules? I’ll quote the definition from the CommonJS Module specification, as it describes it very well:

This specification addresses how modules should be written in order to be interoperable among a class of module systems that can be both client and server side, secure or insecure, implemented today or supported by future systems with syntax extensions. These modules are offered privacy of their top scope, facility for importing singleton objects from other modules, and exporting their own API.

So, modules are isolated parts of code that live in their own scope and are kind of self-contained. Well, that is pretty much how we see the individual Features in EmbedJS – we made our feature implementations as small as possible and reduced dependencies as much as possible. Many of EmbedJS’ features are also browser-independent, so why not make it possible to use some of the tools in a non-browser environment?

(If you want to learn more on modules and why we need them, I recommend @briancavalier‘s presentation on modules.)

Features in EmbedJS

So far, I didn’t really explain what I mean with “Features”, so let me do another quote, this time from the Project Page:

EmbedJS uses the concept of features: Functionalities are split up into features, as fine-grained as possible, and each feature might have multiple implementations.

Our Features are kinda small and contain only few methods, sometimes even just one. The asyc-promise feature, for example, will get you Promises, and the transport-jsonp feature will get you a method for pull in data via jsonp – and that’s it.  If multiple methods are contained within one feature, this is because they are too closely related to each other to split them up, like the html-class feature, which contains the addClass, removeClass, toggleClass and hasClass methods.

AMD vs CommonJS Modules 1.1

Now that we knew modules were the way to go, there was another, quite important question to solve: There are two common patterns for modules, so which one do we want to apply? One pattern is described in the above mentioned CommonJS Wiki, the other pattern is described in the AMD wiki.

Well, there have been some blog posts lately that compare the pros and cons of each of them, so I don’t want to repeat all of this – but I’m hiving a similar tendency towards AMD. Besides this, there was another big reason to go with AMD: Remember, EmbedJS is based on the Dojo Toolkit, and it’s a Dojo Foundation Project – and we hope that the efforts that went into making EmbedJS might flow back into the Dojo Toolkit one day. And as the Dojo Toolkit uses the AMD approach, the decision was quickly made to do so as well.

Implications for EmbedJS

There are several huge benefits for EmbedJS in using the AMD approach. The most noteworthy surely is the ease-of-use for developers. In your code, when you find you want an EmbedJS method, you look up the feature name that provides this method, add it to the require statement and there you go. Man, easy.

A second important thing is that EmbedJS now plays really nice with AMD based projects. In fact, you could see EmbedJS now as a collection of small useful modules, that can be easily integrated – and also easily extended.

The third major benefit is that we can now get rid of all the custom tools we built around EmbedJS – you can load features with any AMD loader (though we use and recommend RequireJS) and build it with any AMD optimizer/builder (we recommend r.js).

Let’s have an extra word on

Building

It was always possible with EmbedJS to create a custom build that would contain only the features you wanted/needed in your project. But, you had to use our build tool that we created. And learning using Yet-Another-Build-Tool™ is always… well, let’s just say it would be cool to avoid it (and, not to forget, so is maintaining it). We did a lot to improve that, we created a GUI that allowed you to just “click together” the features you wanted, and we worked on a dynamic loader that would ease development. But still, the situation was far from ideal, and people just ended up going for our ready-made “kitchensink” builds that contained all of the features.

Now, we still provide those ready-made builds that contain only the implementations for a given platform (and are stripped of require/define calls), but I really hope that using the AMD approach and using an AMD optimizer to deploy will become the more adopted style.

Q: Cross-target? A: The Feature Plugin

One goal of EmbedJS is to allow for easy cross-device development. We got rid of run-time branching but instead went for an approach that held the different implementations for a given feature in separate files, so that one only has to deploy the actually needed code to a given device – and not all the alternate implementations, that sit useless in the device’s memory and never get used. We needed to achieve that with “in-house” means provided by the AMD approach. Lucky us, there’s the concept of loader-plugins, and so I created a feature-plugin for AMD loaders. The idea is simple yet effective: You create an implementation map that maps a feature name to an implementation, and if there are multiple implementations, you supply tests that tells the plugin which feature to use.

When you’re ready to deploy, you have two choices: I f you really don’t know on what target your code is going to end up, you can use the dynamic implementation map containing the tests for the optimizer, and you’ll get a build that contains all of the available implementations. Then, during runtime, the feature plugin will run the tests and load the specific implementation. But, if you do know what target your code will run on, you provide a specific implementation map that doesn’t contain tests but directly maps to the according implementations, and you will only have those in your built file.

Synopsis

AMD is exactly what we needed to make EmbedJS more what we want it to be – an easy to use JavaScript toolbox that is especially handy for cross-target scenarios such as cross-platform mobile development. So, please don’t hesitate and check it out on Github, read the project page and check out the examples!

Resources / Further Reading

• EmbedJS on GitHub
• EmbedJS Project Page
• EmbedJS API Docs
• AMD-Feature plugin
• AMD Wiki
• CommonJS Wiki

As Firefox 4 is now stable, chances are that the async IDB (IndexedDB) API is not going to change anymore. But IDB is not localStorage – it can be a major pain to work with. So, here is a guide on how to use IDB in Firefox as a key-value store.

TL;DR

There is example code available at the end of the post. If you don‘t want to read this lengthy post and/or prefer to jump right in the middle of the action, grab the example implementation and play around with it.

Yes, it is. But I still argue that in WebApp context a key-value store will cover about 90% of use cases. So I won‘t cover in this post how to create fancy key ranges or how to create multiple indices. But I will cover the five basic methods that you will need to work with any store: get, set, remove, getAll and clear – this will enable you to work with IDB and give you an understanding of how it works. You‘re then of course heartly encouraged to explore it and it‘s features further!

As long as the sync API is not there, all operations are async. That means that you will make requests. All the time. These requests will have an onerror and an onsuccess property, where you can put your callbacks/handlers in. If your request was successful, you will get an event in your callback. That event contains a lot of useful and useless information, but what you will want to look at in most cases is event.target.result – this is where your result lives in. In most cases, you will have to start a transaction before; inside of the transaction, you can access the objectStore and make your request. However, there are different types of transactions, but you will get the details below.

Yep, it‘s not that you can just start away and store your data, there‘s some work to do ahead…

Open the Database

Easy one:

event.target.result then contains a reference to the database. Store it, you will need it.

Creating/Opening the ObjectStore

That‘s a little tougher. Some operations, such as creating and deleting object stores, require the database to be in a „mutation transaction state“. Wow. Well, there‘s only one way to put the db into this state, and that‘s via a setVersion request. If you are opening the db for the first time ever, you will need to do this anyway. You can, of course, change the version of the database anytime, which might be a useful thing if you, e.g., change the structure of how your objectStore saves the data. To check what version the database in the user‘s browser has, you can check the version property in the reference to the database anytime.

Inside of the callback function, you can now check if your store already exists, or if you need to create it. The reference to your db contains an object called objectStoreNames, which is a list of existing objectStores. As this is a DOMStringList, it features the handy contains() method:

If the store is not there, you need to create it via the createObjectStore() method. It takes two arguments: the name of the store, and an object with two properties: keyPath and autoIncrement. keyPath is… well… the path to the key. Think of it as the one column that is the primary key in your data table (you all did some MySQL at some time, I bet). You can do really fancy things with the keyPath in objectStores, but it‘s perfectly fine to just use something like „id“ as the keyPath. If you want to store an object and want to provide the keyPath, all you need to do is let the object you want to store have a property with the name of the keyPath, and a unique value. Uh, it‘s not that complicated, examples will follow. AutoIncrement is, well, autoIncrement. You have the option to let the objectStore take care for the uniqueness of it‘s data‘s keyPath properties, or do it yourself.

You immediately get a reference to the newly created objectStore, so save it.

If the store is already there, you need to open it. Um, well, you don‘t really have to open it, but you might want to obtain a reference to it, just to make sure that it really exists. To do this, you need to start a transaction via the transaction() method. It accepts three arguments: an array of store names (the ones which you want to work with), an integer denoting whether you want to read and/or write during that transaction, and a timeout. There are constants for read/write you can use for better readability in your code, you can find them at IDBTransaction.READ_ONLY and IDBTransaction.READ_WRITE. The timeout parameter is optional. That method returns a transaction object that contains a method called objectStore, which you can use to access your store. This is the „default“ way transactions work; the mutation transaction above is an eception to this, where you need to be in the callback of the transaction to modify the database.

In our case, we can start an „empty“ transaction:

There you have your reference to the store.

Now that the store is ready to use, it‘s time to store some data in it. To do so, you need to start a transaction, access the object store, and call the store‘s put method:

That‘s all. This type of transaction is what you also use for other data manipulation tasks, for example to retrieve data:

You will need to provide the key of the stored entry to get it; as you need to do when you want to remove it:

If you want to get all data objects from the store:

And, finally, if you want to clear (i.e., delete all data objects) the store:

It isn‘t. Yeah, well, it is, as you can store objects and don‘t have to stringify them before. But basically, to get the real advantages of IDB, you will need to dive deeper into it. Features like keyRanges remove the need to do map/reduce action when you search for items in your store. But now you have an idea of how the whole thing works. If you want to go further, browse in the spec or the MDC docs and try things out!

Example Code

There is an example page over here: http://static.uxebu.com/~jens/indexeddb/ff-idb.html.

It uses an ObjectStore wrapper I created using dojo that covers the basic methods mentioned above, the JavaScript is here: http://static.uxebu.com/~jens/indexeddb/idb-ff.js.

When you open the page, open the console and you will see some messages there. Check out the source to see what‘s going on; it‘s basically a round trip through all the basic methods.

Further Reading

• The spec: http://www.w3.org/TR/IndexedDB/
• The MDC article: https://developer.mozilla.org/en/IndexedDB

Last week, we tagged the current state of EmbedJS 0.1. This is a large step for us, and something we have been waiting for and wanting to do for a long time. And with doing so, the need arises to answer a lot of questions – and we better start sooner than later. So, here it is, the first part of „Explaining EmbedJS“.

What is it?

It‘s a JavaScript Toolkit, especially aimed at embedded devices. Embedded devices, that is phones, tablet devices, TVs, car dashboards and the like. Devices that are capable of running JavaScript. We are sure that this list is going to grow over time – and will hopefully one day also include devices like your fridge. EmbedJS is based on the Dojo Toolkit.

Why?

Because the toolkits and frameworks that exist focus on desktop browsers and therefore have a different approach – an approach we didn‘t find that suitable for embedded/mobile development. And the JS frameworks that target mobile devices are flawed, too: either they target only a specific device or platform, or they provide far more than we do need and want.

What we wanted was a toolkit that offered a unified API across different platforms, but with the best implementation that was available for each platform. And we didn‘t want to ship code that implemented a certain feature for, say, an old Blackberry device to an iPhone. Plus, we wanted that API to be small – enough to assist us in our daily work, but not that bloated that we would ship code to a phone that we wouldn‘t need in most of our projects.

The Dojo Toolkit

We at uxebu have a strong affinity to the Dojo Toolkit – most of us are committers – for a good reason: we believe that it has done many things right, and that it has an excellent code quality. Thus, finding a starting point for our venture to what was later to be EmbedJS was pretty easy: It was the very core of the Dojo Toolkit, dojo._base.

We examined the code and split it into its smallest parts (features, how we call them in EmbedJS). After having the features all nicely separated, we could look for the best implementation for each of the platforms we wanted to support, and put it into a separate file. Then we optimized our code, tweaking here and there, but also using the code from dojo._base as it was, because it already was the best implementation.

And we needed to find means to put all these tiny parts together again, respecting any given dependecies, to create a file that then contained all the feature‘s implementations for a given platform – so the EmbedJS Tools were born, including our highly flexible build system.

API

EmbedJS‘ API is a small subset on Dojo‘s API. It provides around 70 methods, from the fields language enhancement (like Array manipulation or JSON), OO (like classes and inheritance), transport (like JSONP or XHR), event system (like connecting to DOM events or methods), DOM manipulation (like style or node creation) and misc things like Promises or pub/sub.

We kept the method signatures intact so that you can import your existing knowledge from working with the Dojo Toolkit to EmbedJS. There are only a few minor changes where we

a) found a new name highly useful, e.g dojo.io.script.get() is embed.jsonp()

b) we needed to strip given functionality for the sake of code size.

All methods in EmbedJS are available under the embed namespace as well as under the dojo namespace. I.e. to call the connect method you can use dojo.connect() and embed.connect().

The full API documentations is here, though it‘s undergoing some updates right now.

Summary

I didn‘t want to dig too deep in the first article on EmbedJS, but I guess you got the point: EmbedJS is a highly optimized Dojo core, and offers you a specialized, optimal build for a given platform. To explain how the build system works, how we manage dependencies, how we separate our features and our implementations and how one can create a project-oriented custom build will need another blog post – but if you are too curious, you are invited to take a look at the source code at github; a lot is self-explenatory there. A further source of information is the EmbedJS project page, as well as the EmbedJS Tools source.

Among other tests, the main thing I wanted to know is how long does it take to read or write 1000 entries from/to localStorage. The keys were of the form “key000″, the values “val000″, with increasing numbers. So the tests were using minimal data – only 6  bytes – but the results are still useful to get an idea of how it goes – and how the browsers perform compared to each other.

Desktop

All test were run on a Mac Mini. Opera and IE were tested on a virtual machine, so there might be an additional performance penalty based on virtualization. However, here are the numbers (average time needed to run the respective method 1000 times):

Safari 4:
- getItem: 1.4ms
- setItem: 1.7ms
- key: 0.8ms

Chrome:
- getItem: 240ms
- setIem: 280ms
- key: 230ms

Firefox:
- getItem: 77ms
- setItem: 2100ms
- key: 25ms

Opera:
- getItem: 19ms
- setItem: 20ms
- key: 18ms

IE8:
- getItem: 5ms
- setItem: 72ms
- key: 3.6ms

Mobile

As we can see, Safari is lightening fast. That made me want to run the tests on a mobile phone that has a modern Safari browser. I took an HTC Desire, wich comes with Android 2.1, and thus has a Safari capable of localStorage. Here are the results:

HTC Desire:
- getItem: 16.4ms
- setItem: 19.5ms
- key: 8.7ms

This is pretty amazing, considering that it is a phone. It still beats most of the browseres on the desktop.

If I find that test page again (uh, yeah, it’s lost somehow), I’ll run the tests again with newer browser releases and publish the link here, so you can run the tests yourself.

So here you go: dojox.storage.encrypted, a Blowfish encrypted storage. It sits on top of dojox.storage, and you get all the dojo storage manager goodness, mainly the automatic selection of the best storage provider available. It exposes the complete API that dojox.storage does. If an attacker gains access to the storage area, he can still nuke the storage, but the data found within will be useless.

I’m not completely happy with it’s architecture, and I didn’t have the time yet to fully test it with all providers, but it works pretty well with localStorage and the performance is not too bad. If you have a spare minute and are interested, feel free to to test it yourself (ah I know, you don’t have the time yourself, but I’d be thankful!)…

Usage

Just use it as you would use dojox.storage, except that you need to set your passphrase before you start working with the storage:

dojo.require("dojox.storage.encrypted");
var sto = dojox.storage.encrypted;
sto.setPassphrase("my super secret passphrase");
sto.put('key','value');
var value = sto.get('key');

It uses/requires dojox.encoding.crypto.Blowfish but that’s been in dojo for ages, so no worries.

To use it, just put the encrypted.js file in your dojox/storage directory.

Files

the code (encrypted.js)
the test page

Update II: Ben Hockey proposed to do the whole thing without using the pub/sub system. I for one think its a good idea, but in case you want the topic version, I’ll leave the test page and add another one with the topic-less version. Thanks Ben!

Update III: Dojo 1.5 introduces robust Promises with dojo.deferred as Krys Zyp explains in this SitePen post.

—

One of the things/ideas/concepts that I really like about CommonJS is Promises.

Promises? A brief explanation from the CommonJS API:

Promises provide a well-defined interface for interacting with an object that represents the result of an action that is performed asynchronously, and may or may not be finished at any given point in time.

To hear more about Promises, I highly recommend these two posts.

That concept is not entirely new. The dojo toolkit has dojo.Deferred, which does a similar job. You create a Deferred, add (multiple) callbacks and error handlers to it, and when it fires off, all your handlers get called. But dojo.Deferred is pretty heavyweight. It’s loads of code. And it works with .addCallback() and .addErrback() methods – not with .then() methods, which provide maximum readability.

The goal

What I want to have is the follwing code running (with dojo):

asyncComputeTheAnswerToEverything().
    then(addTwo).
    then(printResult, onError);

(This is the example code from the proposed CommonJS Promise API)

And I want the execution to continue right after doSomethingAsnyc() – the async call should still be non-blocking. The .then() methods should recieve the results of the previous part of the chain, but still be non-blocking. Luckily, dojo provides a tool that allows us to achieve this: the publish/subscribe system.

Let’s do it

When we have unique topics for each promise made, we can subscribe the .then() method to the current topic in the chain, and publish the next topic when the data handlers are done computing. Within the then() method we need to check if the data we recieve is an error or not, to determine wich handler we need to call. The spec also demands a progress handler – if we wanted to implement that, we needed to add another topic, but I left that out.

We only need to kick off  the whole thing when we create the first promise, then the rest takes care for itself. Including the changes proposed by Micheil, what you have to do is pretty trivial and you don’t have to care about unique topics or the pub/sub system at all. The asyncComputeTheAnswerToEverything() function mentioned above would look like the following:

var asyncComputeTheAnswerToEverything = function(){
	var promise = new dojox.Promise();
	window.setTimeout(function(){ // some async operation
		var data = 42;
		promise.emit(data);
	},2000);
	return promise;
};

You create a Promise and store a reference to it. You return the promise and use the reference to it to kick off the chain when the data is there. Everything else runs on it’s own now.
Go to the test page or theand check the source – it contains the Promise code and the example functions. Open the console and run testSpec(); to start off the example code from the CommonJS API proposal.

Usage

As a namespace to put the Promise into I chose dojox – but choose whatever floats your boat. You can just drop the promise code somewhere into your code, or, if want to dojo.require("dojox.Promise") it, create a file named Promise.js that contains the code and place a dojo.provide("dojox.Promise"); as the first line in it. Save the file in the dojox directory and you’re done. (Well, to be honest, this violates the dojox naming convention, but if its just in your project, it’s ok and it will work.)

Any questions left? Feel free to leave a comment, @ me at twitter or drop me a line.

References / Files / More

• CommonJS Promise API Proposal
• Post by Kris Zyp @ SitePen blog about CommonJS
• Post on www.michaelharrison.ws about promises and more
• the test page (using topics and pub/sub)
• the test page (w/o topics, recommended)

The Curve

Every dojo.Animation needs a curve. If you call animateProperty or it’s shorthand anim, you don’t have to provide the curve for yourself, dojo will do it for you – if you build an animation using new dojo.Animation(), you have to provide your own curve. You can think of the curve as of a line that the animation walks on. The line is already present before the animation starts to walk it, and it can always take a break and pause in the middle and eventually continue walking where it had stopped. To create a curve, you need to call new dojo._Line(start, end). In return, you get an object with three properties: end, start and a getValue() method.

var anim = new dojo.Animation({
	curve: new dojo._Line(0,100)
});

Easing

The curve will then be modified by a easing function. By default, this is the linear dojo._defaultEasing. Easing breathes a bit of life into the curve, you can say. Other properties of dojo.Animation that affect the curve are rate and duration. rate is the timespan between two frames. To calculate the targeted fps, use (1000/rate). By default, dojo aims at 50 fps.

var anim = new dojo.Animation({
	curve: new dojo._Line(0,100),
	easing: dojo.fx.easing.quadInOut
});

Waking Life

To get some numbers out of dojo.Animation, let’s let it go wild and play(). When using animateProperty or the likes, you just fire and forget. In our case, we need to get the numbers. To achieve this, we connect to our animation’s onAnimate event. This is fired on every frame of the animation. When your connected function is called, you get – who would have guessed – the current value of the curve. Now you have a function called at a given rate, for a given period of time and you get get a numeric value that is already calculated for you. Decent!

dojo.connect(anim,'onAnimate',function(curveValue){
	// do something
});

To see some curves in action, go to this test page, open the console and type play(1); and play(2); and check the source code to see what’s happening.

Multidimensional Curves

dojo._Line generates only one-dimensional curves. But, there is dojox.fx._core, which contains an (old) version of dojox.fx._Line, which is capable of creating multi-dimensional curves. To use it, you have to explicitely dojo.require("dojox.fx._core"). It’s a bit odd, but the _core file contains only the _Line, and it is not loaded when requiring “dojox.fx”. Well, whatever, you can pass arrays of numbers into dojox.fx._Line:

new dojox.fx._Line([0,2],[100,6]);

Now you have a multi-dimensional curve. Or, if you could say, you have two curves in one. You can (technically) pass an unlimited amount of numbers into dojox.fx._Line. When you hand this curve to a dojo.Animation, it will animate all dimensions of the curve. And it will modify all dimensions of the curve according to easing, rate and duration. If you connect a function to onAnimate, it will now recieve an array of numbers.

var anim = new dojo.Animation({
	curve: new dojox.fx._Line([0,2],[100,6]),
	easing: dojo.fx.easing.quadInOut
});

Head over to the test page again, and type play(3); into the console. Now, the width of the bars is also animated – by the same animation function that calculates the height of the bars.

What now?

If your mind is still not blown away with ideas about how to use this, let me give you a start:

• animate position, width and height of a browser window and use it as a screensaver for some unknowing, yet thankful visitor…
• take some CSS 2D and 3D transisitions/transforms and animate away…
• canvas! No need to say another word…
• Still too banal? Go WebGL! Grab some 3DContext and animate an evil ball of jelly jumping around!

In short: dojo’s animation system allows you to not only animate CSS properties, but gives you a good tool for  animations in general. You get the curves on the one hand, but you also get the “player” for these curves, including play/pause/stop/gotoPercent methods. And dojo.fx.easing has a lot of easing functions in stock.

Uh, persistant local storage?

Ahm, yeah, in case you don’t know, that means a client-side storage, be it in a browser, or in an Air app. It’s not very popular, but that concept has been around for a long time. So why local? There are two major reasons for it: First, we need this for apps and tools that work offline – and apps and tools that work online but need an offline backup and sync later. Kris Zyp wrote a post about the JsonRestStore and OfflineRest back in 2008, he goes a little into detail there. Secondly, we need it for apps and tools that rely on persistence for other reasons, no matter if online or offline – like it happended to me. When I ran into Lawnchair, I had the idea to build a tool that was monitoring CSS selector usage on websites and apps (you know how you sometimes lose control about CSS selectors in webapps…). To achieve this, I needed to store data locally, and persistent.

What dojo offers

As I was using dojo’s excellent CSSRuleStore to find out what selectors where used, I wanted to switch from Lawnchair to a dojo solution. Dojo has dojox.storage (the above mentioned OfflineRest uses dojox.storage), which provides wrappers for the following backends: Gears, Flash, globalStorage and Adobe Air (globalStorage is Firefox’s early implementation of the early WhatWG storage draft). Hm. Not that much. Gears and Flash are 3rd party solutions, globalStorage is a Firefox-only-thingy (and old, though it’s still supported in current Versions) and Air is non-browser. That means: No IE without plugins. No Safari when Flash is not available.

What dojo does not offer (but should)

Guaranteed, reliable support for *any* browser. Without depending on 3rd party plugins. As of now, to have this, you’d need to use Lawnchair or persist,js, which is pretty complete. Speaking of it, let me quote from it’s readme (it’s a good read on local persistent storage in general):

The most notable attempt at addressing this problem [storing client-side persistent data] is probably Dojo Storage. Unfortunately, Dojo Storage does not support Internet Explorer without Flash, and it does not support Safari or other WebKit-based
browsers at all (at least, not without Flash). Also, Dojo Storage is
not standalone; it requires a several other Dojo components in order to
operate.

Whoa, even other JS tools mention the lack of support in dojo…

New providers

Attached to this post are wrappers for the following mechanisms:

• localStorage: this is the current spec, and it is supported by IE8, Firefox 3, Safari 4, Opera 10, Chrome 5.
• userData behavior: this is Microsofts client-side storage for IE < 8.
• cookie: worst case fallback solution…

With these three, there’s an almost 100% coverage of browser situations (see storage overview). There are still no-go scenarios, of course, such as IE7 with userData disabled and cookies disabled, but there’s no way to address them all.

Notes on BehaviorStorageProvider

This one uses the userData behavior, present in IE from version 5 up. It works with DOM elements that have a special behavior assigned. You can then use  set/get/removeAttribute() methods to store data. It has some limitations though – of course…

• Maximum size varies from 64k to 1M – depending on the zone the site is in. So, you can’t rely on more than 64k.
• UserData can be switched off in security settings.
• I couldn’t find a way to read out stored keys if you don’t know them. Maybe this is some security thingy or just a bug – but it means that some methods in the provider don’t work. (that is, getNamespaces, getKeys and clear). To work around this, we have to take some of the precious storage space and store the keys and namespaces seperately.

Usage

Copy the the providers into your /dojox/storage directory and add the needed dojo.requires to /dojox/storage/_common.js:

dojo.require("dojox.storage.LocalStorageProvider");
dojo.require("dojox.storage.GearsStorageProvider");
dojo.require("dojox.storage.BehaviorStorageProvider");
//>>excludeStart("offlineProfileExclude", kwArgs.dojoxStorageBuildOption == "offline");
dojo.require("dojox.storage.WhatWGStorageProvider");
dojo.require("dojox.storage.FlashStorageProvider");
//>>excludeEnd("offlineProfileExclude");
dojo.require("dojox.storage.CookieStorageProvider");

What’s left?

Well, it would be pretty cool to have a dojo.data API compliant store that uses dojox.storage as backend, so that you could write your apps using dojo.data calls and have a reliable offline backup. I started working on that, but then came Christmas, parents, vacation, new year and a lot of beer. But hey, it’s 2010 now, and there’s plenty of time left!

Storage Overview

Files

LocalStorageProvider.js
BehaviorStorageProvider.js
CookieStorageProvider.js
Storage Test Page

Besides indexOf(), there are some other ways to achieve this, so – let’s have a competition and find out who’s the fastest!

The Contestants

replace and length

One possibility is to replace the searched string with an empty string, read the length property of the modified string and compare it to the length of the haystack. If it’s different, the haystack contains the needle. Reading the length property is extra work, but comparing two integers is faster than comparing two strings, so maybe it’s faster in general.

replace

Again, replace the searched string with an empty string. Then compare the the modified string with the haystack. If they are different, the haystack contains the needle.

split

Take the haystack and try to split it using the needle as the seperator. If the result’s length is greater than one, the haystack contains the needle.

indexOf

Find the first occurance of needle in haystack; if the result is something else than -1, the haystack contains the needle.

Results

For testing, I did 10,000 iterations and retrieved the execution time in ms. The methods were tested in the order presented above.

On Chromium (Mac build, Version 4.0.203.0 here), replace + length is slightly faster than replace, and both are faster than split. indexOf is by far the fastest.

1) true: 7 / false: 3.5
2) true: 7.5 / false: 3.5
3) true: 10 / false: 8
4) true: 2.5 / false: 3

Safari 4 has nearly the same results as Chromium, but the numbers tend to differ a lot from test to test.

1) true: 4-18, avg 10 / false: 2.5
2) true: 4-19, avg 10 / false: 3
3) true: 6-22. avg 13 / false: 6-23 avg. 16
4) true: 2 / false: 3

On Firefox 3.5, split a bit faster than the two replace methods, but indexOf is again by far the fastest.

1) true: 13 / false 10
2) true: 13 / false 10
3) true: 10 / false 9
4) true: 1.5 / false: 2

On IE 8 (run in a VM), both replace versions perform almost the same, and faster than split. Again, indexOf is fastest. Only 2 – 3 times faster than the replace methods, but still the fastest.

1) true: 35 / false: 25
2) true: 30 / false: 25
3) true: 60 / false: 50
4) true: 15 / false: 15

You can run the tests yourself, if you are interested, the test page is here.

Summary

The results are pretty obvious: indexOf() outperforms the other contestants. Which is not really a surprise, considering that whatever Javascript does during indexOf() –  it has also to do the same before being able to do a split() or replace(). So, the proposed way for a contains() method is the following:

dojo.string.contains = function(/* string */ needle, /* string */ haystack, /* bool */ caseInsensitive) {
    if(caseInsensitive) {
        needle = needle.toLowerCase();
        haystack = haystack.toLowerCase();
    }
    return haystack.indexOf(needle) !== -1;
}

If you want to use contains() in your code, just copy the above lines somewhere in your code. Again, don’t forget to dojo.require(“dojo.string”) before.

Or, if you want to have beginsWith(), endsWith() and contains() all-in-one, use this: dojo.string.addons.js