It is mainly meant to serve as an example implementation, so that you could have a look at the code and see how to work with IndexedDB. But I figured that people are also interested in actually using it, as it abstracts away many of the tedious internals of IndexedDB (like transactions) – and it is perfectly fine to use IDBWrapper for all non-overly-complex scenarios.

So here’s a tutorial about how to work with IDBWrapper and add a little background info about IndexedDB internals every now and then, instead of writing Yet-Another-Super-Technical-IDB-Blargh. Part one will cover some info about what IndexedDB is, getting IDBWrapper to run and how to read and write data to a store. Part two will be about querying the store.

What is IndexedDB – why is it special?

IndexedDB really differs from other client-side storage engines. It’s an ObjectStore, whereas localStorage is a key-value store and Sqlite is a relational database. Think of it as a database like those SQL thingies, but with more freedom but without a query language. Uh, roughly, yeah. You store and retrieve JavaScript objects (unlike localStorage) and you don’t have fixed tables with predefined columns and types (unlike Sqlite).

Getting IDBWrapper

You can download/clone/fork IDBWrapper at GitHub. To include it in your page/project, you have two options: You can either include the IDBStore.js file via a script tag, or you can require it using an AMD loader like RequireJS.

Let’s go!

In this tutorial, we’ll create a people store where we store information about humans – a group of customers, for example.

Step one: open the store
A note on the properties we pass to the constructor:

• dbName, dbDescription, storeName: just choose meaningful names
• dbVersion: start with ’1.0′, and only change this when the structure of your db/store changes
• keyPath: the property you want as unique primary key
• autoIncrement: set this to true if you do not want to take care of the uniqueness of the primary key yourself
• onStoreReady: a function to be called when your store is ready to work with

IDBwrapper will open the database, check if a store with the given name exists and if not, it will create it using the above properties. You’ll then get an IDBStore instance back that you can use to work with the store. Once the store is created, it’s persistent. But, if you need to delete and re-create it (for example, if you happened to pick the wrong value for keyPath), you can delete the store using the deleteObjectStore method.

Inside IndexedDB: IDBWrapper works with one store; when you do a new IDBStore() it represents one store in an IndexedDB. If you want multiple stores inside of the same IndexedDB, you’d have to do a new IDBStore() for each store. This is an artificial limitation of IDBWrapper; if you work with IndexedDB directly, you can lock multiple stores when starting a transaction. This be useful when you want to update one store depending on the information found in another store, and you want to prevent this other store’s contents to change during your update action.

Step 2: storing a customer

Note how we don’t have an id property on the dude object, as we specified autoIncrement as true, so the database will take care of this. In the onsuccess callback we’ll get the id of the newly inserted record back. Also, see how we attached an array of emails to the record? In a relational database you’d set up a different table for email addresses – no need to do that in IndexedDB.

Inside IndexedDB: Make sure to wait until the store is ready before operating on it! The whole nature of IndexedDB is asynchronous. Whatever you do, you will have to work with callbacks. There is, however, also a draft for a specification of a synchronous API, but that is not implemented in any browser (and I don’t think this is going to happen anytime soon).

Step 3: reading the customer back from the store

We just pass the id we got after inserting to the get method, and there we have our record from the store. It’s exactly the same thing we stored earlier, with one exception: the object now has the id property which has been attached to it by the database.

Inside IndexedDB: What we pass as first parameter to the get method is the keyPath value. That means when we pass 1 to it, the database will look for an object that has 1 as value in it’s keyPath property, which we set to ‘id’ when we created the store.

Step 4: updating data

To update a record, you also use the put method: If there already is an object in the store with the same keyPath value (the id), it will simply overwrite it. Note that it will really, really overwrite the existing object, which means you cannot simply put a modified property of it, but have to put the whole object, including all properties.

Inside IndexedDB: keyPath values are also Type-sensitive. That means, if the id of our dude is 1 and of type Number, you have to pass numeric 1 to the get method to retrieve the dude, and for updates the value of the id property needs to be numeric 1. If you use “1″ as String, IndexedDB will think of it as a different value.

Step 5: getting all items

Nothing really to say about it. Just one note: IDBStore has default error handlers for every async method that prints potential errors to the console. So if you’re just playing around you don’t need to pass your own handler to the methods.

Step 6: deleting an item

Different browsers and versions return different values to the success callback; that’s why there is an extra check in the success callback.

Step 7: clearing the store

If you need to clear the store from all stored entries, you can use the clear method. Note that this won’t reset Chrome’s autoIncrement counter.

That’s it, thanks for listening!

Now you know everything to do basic IndexedDB data operations with IDBWrapper. Make sure to check out the examples and the documentation!

Part two of this tutorial about querying the store will follow soon.

Resources

• IDBWrapper on GitHub
• IDBWrapper Documentation
• IDBWrapper Examples

Update: Stephan recommended to put the video on top,  so here it goes: Some impressions from RavenJS, enjoy (fullscreen and headphones recommended)!

The Beginning

When we started, I had little to no knowledge about working with 3D and OpenGL – I’m a JavaScript guy, after all. When I first fired up Blender and tried to work with it I was pretty close to dropping the whole idea again. Same happened when I saw how shader code looked like. But I so much wanted this – and the existing wrappers like GLGE or three.js enable us to still work with WebGL even if we don’t have the background knowledge.

So I decided to go with GLGE as I had already played around it with it’s examples before and because it already had nice collada support, which I decided to have as format for meshes and animations.
The first step was to extract some assets from the original game, collect mesh and texture data and get it into Blender. Luckily, Gothic has a highly active modding community, and this community is supported by the original game developers. So there were a couple of tools that could be used to aid in this, and it didn’t take long and we had a sword rotating in a browser.

But if it was possible to load a sword into GLGE, then shouldn’t it also be technically possible to load parts of the world into it? Yes, it was. It took some time, and it’s still far from perfect, but it worked. The next step was to create a first person camera that would react to mouse and key events – and had collision detection. Real collision detection, a height map would get us nowhere, as the levels had bridges and indoor locations, and I needed moving objects, too. Luckily, GLGE had some demos using raycasting to achieve this that I could use as start and modify until it worked as needed.

To recreate the dark intense atmosphere I played around with the right ambient lighting, fog types and added a sky box. Well, I would have loved to have a real sky dome, but, alas, I have no idea how to do this… Finally, I added objects, both static and animated.

In action

I can’t and won’t put online a live demo, as I obviously have no rights to do so. The copyright of the meshes and textures used belongs to the original developers, Piranha Bytes, and the RavenJS project is in now way endorsed, encouraged or supported by nor affiliated with them. But I can show you some impressions of what it looks like so far (fullscreen and headphones recommended):

Update: Video is now on top

(The song in this video is called “A special thing” by “amanyth” and published under a Creative Commons license.)

Issues

Besides the mere amount of free time that is needed and lacking, there are currently four main issues that are a bit of a show-stopper:

Animations caused a big deal of problems, and it’s still not prossible to run more than one animation sequence – running multiple animations will screw up bone position. While the animations still run fine in Blender, something goes wrong during the export. I’m still to find out if this can be overcome by using an alternate format for export.

Another thing that is still unsolved is “double-faced textures” that have transparent parts (look at the trees in the video). Those transparent parts become plain black when rendered in the browser. I’ve read up on this, and it seems that this scenario is indeed not trivial… From what I understood so far, in our case there’s two shapes “glued” to each other and each one has its own material, but I still have no idea what exactly goes wrong.

A real biggie, performance-wise, is that there is no logic yet that handles display of objects that are not visible. In the original game e.g., doors and the likes are “closed” by vertices with solid (non-transparent) textures and everything that’s behind it is taken out from the whole rendering/object interaction/whatever process. Only when you get really close you can see through these and see the intereour of the adjectant room. If these vertices get actually removed or if you “portal” through them I don’t know.

Mouse lock. Of course. No further explanation needed, we’re all looking forward to seeing this in the browser.

Future

I always loved 3D games. And the fact that it’s possible to render 3D worlds in a browser – with acceptable performance – is just stunning. The majority of work is still ahead, I’m well aware of that, but I *so much* want this to happen… So I just hope that some of the major problems can be solved and this project can go on!

During JSConf.eu, we released EmbedJS 0.2. That release was mainly about a fundamental architectural change: the move to the AMD pattern for our features. Let me explain why we decided to do this, why we think this is a big step forward and what this means for the user.

Modules

AMD stands for “Asynchronous Module Definition”. So, what are modules? I’ll quote the definition from the CommonJS Module specification, as it describes it very well:

This specification addresses how modules should be written in order to be interoperable among a class of module systems that can be both client and server side, secure or insecure, implemented today or supported by future systems with syntax extensions. These modules are offered privacy of their top scope, facility for importing singleton objects from other modules, and exporting their own API.

So, modules are isolated parts of code that live in their own scope and are kind of self-contained. Well, that is pretty much how we see the individual Features in EmbedJS – we made our feature implementations as small as possible and reduced dependencies as much as possible. Many of EmbedJS’ features are also browser-independent, so why not make it possible to use some of the tools in a non-browser environment?

(If you want to learn more on modules and why we need them, I recommend @briancavalier‘s presentation on modules.)

Features in EmbedJS

So far, I didn’t really explain what I mean with “Features”, so let me do another quote, this time from the Project Page:

EmbedJS uses the concept of features: Functionalities are split up into features, as fine-grained as possible, and each feature might have multiple implementations.

Our Features are kinda small and contain only few methods, sometimes even just one. The asyc-promise feature, for example, will get you Promises, and the transport-jsonp feature will get you a method for pull in data via jsonp – and that’s it.  If multiple methods are contained within one feature, this is because they are too closely related to each other to split them up, like the html-class feature, which contains the addClass, removeClass, toggleClass and hasClass methods.

AMD vs CommonJS Modules 1.1

Now that we knew modules were the way to go, there was another, quite important question to solve: There are two common patterns for modules, so which one do we want to apply? One pattern is described in the above mentioned CommonJS Wiki, the other pattern is described in the AMD wiki.

Well, there have been some blog posts lately that compare the pros and cons of each of them, so I don’t want to repeat all of this – but I’m hiving a similar tendency towards AMD. Besides this, there was another big reason to go with AMD: Remember, EmbedJS is based on the Dojo Toolkit, and it’s a Dojo Foundation Project – and we hope that the efforts that went into making EmbedJS might flow back into the Dojo Toolkit one day. And as the Dojo Toolkit uses the AMD approach, the decision was quickly made to do so as well.

Implications for EmbedJS

There are several huge benefits for EmbedJS in using the AMD approach. The most noteworthy surely is the ease-of-use for developers. In your code, when you find you want an EmbedJS method, you look up the feature name that provides this method, add it to the require statement and there you go. Man, easy.

A second important thing is that EmbedJS now plays really nice with AMD based projects. In fact, you could see EmbedJS now as a collection of small useful modules, that can be easily integrated – and also easily extended.

The third major benefit is that we can now get rid of all the custom tools we built around EmbedJS – you can load features with any AMD loader (though we use and recommend RequireJS) and build it with any AMD optimizer/builder (we recommend r.js).

Let’s have an extra word on

Building

It was always possible with EmbedJS to create a custom build that would contain only the features you wanted/needed in your project. But, you had to use our build tool that we created. And learning using Yet-Another-Build-Tool™ is always… well, let’s just say it would be cool to avoid it (and, not to forget, so is maintaining it). We did a lot to improve that, we created a GUI that allowed you to just “click together” the features you wanted, and we worked on a dynamic loader that would ease development. But still, the situation was far from ideal, and people just ended up going for our ready-made “kitchensink” builds that contained all of the features.

Now, we still provide those ready-made builds that contain only the implementations for a given platform (and are stripped of require/define calls), but I really hope that using the AMD approach and using an AMD optimizer to deploy will become the more adopted style.

Q: Cross-target? A: The Feature Plugin

One goal of EmbedJS is to allow for easy cross-device development. We got rid of run-time branching but instead went for an approach that held the different implementations for a given feature in separate files, so that one only has to deploy the actually needed code to a given device – and not all the alternate implementations, that sit useless in the device’s memory and never get used. We needed to achieve that with “in-house” means provided by the AMD approach. Lucky us, there’s the concept of loader-plugins, and so I created a feature-plugin for AMD loaders. The idea is simple yet effective: You create an implementation map that maps a feature name to an implementation, and if there are multiple implementations, you supply tests that tells the plugin which feature to use.

When you’re ready to deploy, you have two choices: I f you really don’t know on what target your code is going to end up, you can use the dynamic implementation map containing the tests for the optimizer, and you’ll get a build that contains all of the available implementations. Then, during runtime, the feature plugin will run the tests and load the specific implementation. But, if you do know what target your code will run on, you provide a specific implementation map that doesn’t contain tests but directly maps to the according implementations, and you will only have those in your built file.

Synopsis

AMD is exactly what we needed to make EmbedJS more what we want it to be – an easy to use JavaScript toolbox that is especially handy for cross-target scenarios such as cross-platform mobile development. So, please don’t hesitate and check it out on Github, read the project page and check out the examples!

Resources / Further Reading

• EmbedJS on GitHub
• EmbedJS Project Page
• EmbedJS API Docs
• AMD-Feature plugin
• AMD Wiki
• CommonJS Wiki

As of version 1.6, dojo comes with the new Dojo Object Store API. This is an awesome thing, as it greatly simplifies the work with data stores in Dojo. Everybody who had to do with the traditional dojo.data API felt it was overly complex and hard to use – this has finally changed now. There are also wrappers from and to the old and new APIs, so that you can do stuff like using your traditional data-aware widgets with a new Object Store. And the goodness doesn’t end here; but more on this later. If you haven’t done so yet, you might want to read the excellent post on the new Dojo Object Stores by Kris Zyp where he explains all the awesomeness he created.

Dojo also comes with two fresh implementations for the API, a non-persistent memory store and a JsonRest store that interacts with a server through RESTful HTTP requests. You can also observe changes made to objects in the store. What is missing is a store that uses a client-side persistent backend, which would be useful for a couple of reasons (e.g., as one commenter in that post asks, to use it as chache store for dojo.store.Cache). As Kris already mentions, it’s a piece of cake to create one, so let’s go ahead and do it!

Choosing the Backend

The post mentions two possible candidates: dojox.storage and StorageJS, both already having a similar API, so making them compliant to the Dojo Object Store API is fairly easy. For this tutorial I choose StorageJS – because it is very lightweight and still provides all we need (and, well, for a couple of other reasons that don’t belong in this post). So, what is the exact API we need to comply to? Check it out in the docs. See the first sentence in that paragraph? “Every method in the API is optional, it’s presence indicating support for that feature.” Wow, that’s nice!

Basic compliance

Let’s start with the methods get(), put(), add() and remove(). That will make our store pretty usable for most store scenarios.

First, create a ‘store’ object we can work with (StorageJS creates a global variable called storage where all it’s methods reside):

‘store’ now contains everything that ‘storage’ does, but we can leave the original ‘storage’ object alone and modify our ‘store’ object if needed. Time for the first method, let’s start with get(). StorageJS works with Strings as keys and values only, so we need to parse what we get from it.

Easy one, now move on to put(). The Object Store API allows user to do stuff like this: store.put({foo: 'bar'}, {id: 3}) as well as store.put({id: 3, foo: 'bar'}). Also, StorageJS only has the method set() instead of put() and add().

Easy, as well – you could write this as a one-liner. Now, add(). It’s close to put, except that it won’t overwrite data that already exists. So we just need to check if something with the given id already exists and throw an error if so.

What about remove()? Nothing to do there, it already has the desired signature and functionality.

That’s it, we’re done with basic compliance. Our ‘store’ object now implements the basic methods of the Dojo Object Store API and can be used by anything that can work with a Dojo Object Store – and it will keep it’s state persistent in the browser.

More compliance? Querying!

There’s one interesting thing in the API: the query() method. Uh, yeah, we want that! Lucky us, Dojo already provides a query engine as well as a method that makes sure there are iterative methods and a total property containing the number of hits available in our result (well, it also abstracts away the differences between sync and async results, but as StorageJS is purely synchronous, this is not important to us). First, define a queryEngine:

So, if we want our store to be able to run queries, all we need to do is to take the store’s data and hand it over to the query engine. We use StorageJS’ getAll() method to acquire the data. It will return an array of objects in the form { key: ‘theKey’, value: ‘theValue’}, but the queryEngine wants an array containing only the objects, so we need to do a conversion.

Now we can query our store for specific data, like so: var results = store.query({ prime: true }); This will give us all objects in the store that have a property called ‘prime’ and have that property set to true.

Hierarchie and Transactions

The API also mentions methods about data hierarchie and transactions, but I’m not going to cover these in this tutorial.

Convenience and Optimization

Currently, every time we run a query, we fetch all data again from our storage engine. So, if you don’t have a massive amount of data and want to query the store a lot, it might be wise to maintain a copy of all data in memory. There are other things you could optimize as well – or you could write your own queryEngine.

For your convenience,  here’s a wrapper that creates a Dojo Object Store like we did above and accepts a useMemory parameter during instantiation that denotes whether a copy of the data should be kept in memory or not. It uses the traditional require/provide instead of the AMD format. If you want to use it, make sure you use the right namespace in the declare() call! (Or, not recommended, copy it in the /dojo/store directory.)

If you want to fool around with it in the console first, you can go here (non-memory-using) or here (memory-using), check out the source and fire up the console and try things out.

Getting StorageJS

The above code uses the features base and getAll, so you could do a specific build of StorageJS with these features or just grab a full build of it for your desired storage engine (the storage-full-[engine].js files). If you don’t know about storage engines, all modern browser support localStorage, so this is the one what you might want.

As Firefox 4 is now stable, chances are that the async IDB (IndexedDB) API is not going to change anymore. But IDB is not localStorage – it can be a major pain to work with. So, here is a guide on how to use IDB in Firefox as a key-value store.

TL;DR

There is example code available at the end of the post. If you don‘t want to read this lengthy post and/or prefer to jump right in the middle of the action, grab the example implementation and play around with it.

Yes, it is. But I still argue that in WebApp context a key-value store will cover about 90% of use cases. So I won‘t cover in this post how to create fancy key ranges or how to create multiple indices. But I will cover the five basic methods that you will need to work with any store: get, set, remove, getAll and clear – this will enable you to work with IDB and give you an understanding of how it works. You‘re then of course heartly encouraged to explore it and it‘s features further!

As long as the sync API is not there, all operations are async. That means that you will make requests. All the time. These requests will have an onerror and an onsuccess property, where you can put your callbacks/handlers in. If your request was successful, you will get an event in your callback. That event contains a lot of useful and useless information, but what you will want to look at in most cases is event.target.result – this is where your result lives in. In most cases, you will have to start a transaction before; inside of the transaction, you can access the objectStore and make your request. However, there are different types of transactions, but you will get the details below.

Yep, it‘s not that you can just start away and store your data, there‘s some work to do ahead…

Open the Database

Easy one:

event.target.result then contains a reference to the database. Store it, you will need it.

Creating/Opening the ObjectStore

That‘s a little tougher. Some operations, such as creating and deleting object stores, require the database to be in a „mutation transaction state“. Wow. Well, there‘s only one way to put the db into this state, and that‘s via a setVersion request. If you are opening the db for the first time ever, you will need to do this anyway. You can, of course, change the version of the database anytime, which might be a useful thing if you, e.g., change the structure of how your objectStore saves the data. To check what version the database in the user‘s browser has, you can check the version property in the reference to the database anytime.

Inside of the callback function, you can now check if your store already exists, or if you need to create it. The reference to your db contains an object called objectStoreNames, which is a list of existing objectStores. As this is a DOMStringList, it features the handy contains() method:

If the store is not there, you need to create it via the createObjectStore() method. It takes two arguments: the name of the store, and an object with two properties: keyPath and autoIncrement. keyPath is… well… the path to the key. Think of it as the one column that is the primary key in your data table (you all did some MySQL at some time, I bet). You can do really fancy things with the keyPath in objectStores, but it‘s perfectly fine to just use something like „id“ as the keyPath. If you want to store an object and want to provide the keyPath, all you need to do is let the object you want to store have a property with the name of the keyPath, and a unique value. Uh, it‘s not that complicated, examples will follow. AutoIncrement is, well, autoIncrement. You have the option to let the objectStore take care for the uniqueness of it‘s data‘s keyPath properties, or do it yourself.

You immediately get a reference to the newly created objectStore, so save it.

If the store is already there, you need to open it. Um, well, you don‘t really have to open it, but you might want to obtain a reference to it, just to make sure that it really exists. To do this, you need to start a transaction via the transaction() method. It accepts three arguments: an array of store names (the ones which you want to work with), an integer denoting whether you want to read and/or write during that transaction, and a timeout. There are constants for read/write you can use for better readability in your code, you can find them at IDBTransaction.READ_ONLY and IDBTransaction.READ_WRITE. The timeout parameter is optional. That method returns a transaction object that contains a method called objectStore, which you can use to access your store. This is the „default“ way transactions work; the mutation transaction above is an eception to this, where you need to be in the callback of the transaction to modify the database.

In our case, we can start an „empty“ transaction:

There you have your reference to the store.

Now that the store is ready to use, it‘s time to store some data in it. To do so, you need to start a transaction, access the object store, and call the store‘s put method:

That‘s all. This type of transaction is what you also use for other data manipulation tasks, for example to retrieve data:

You will need to provide the key of the stored entry to get it; as you need to do when you want to remove it:

If you want to get all data objects from the store:

And, finally, if you want to clear (i.e., delete all data objects) the store:

It isn‘t. Yeah, well, it is, as you can store objects and don‘t have to stringify them before. But basically, to get the real advantages of IDB, you will need to dive deeper into it. Features like keyRanges remove the need to do map/reduce action when you search for items in your store. But now you have an idea of how the whole thing works. If you want to go further, browse in the spec or the MDC docs and try things out!

Example Code

There is an example page over here: http://static.uxebu.com/~jens/indexeddb/ff-idb.html.

It uses an ObjectStore wrapper I created using dojo that covers the basic methods mentioned above, the JavaScript is here: http://static.uxebu.com/~jens/indexeddb/idb-ff.js.

When you open the page, open the console and you will see some messages there. Check out the source to see what‘s going on; it‘s basically a round trip through all the basic methods.

Further Reading

• The spec: http://www.w3.org/TR/IndexedDB/
• The MDC article: https://developer.mozilla.org/en/IndexedDB

Last week, we tagged the current state of EmbedJS 0.1. This is a large step for us, and something we have been waiting for and wanting to do for a long time. And with doing so, the need arises to answer a lot of questions – and we better start sooner than later. So, here it is, the first part of „Explaining EmbedJS“.

What is it?

It‘s a JavaScript Toolkit, especially aimed at embedded devices. Embedded devices, that is phones, tablet devices, TVs, car dashboards and the like. Devices that are capable of running JavaScript. We are sure that this list is going to grow over time – and will hopefully one day also include devices like your fridge. EmbedJS is based on the Dojo Toolkit.

Why?

Because the toolkits and frameworks that exist focus on desktop browsers and therefore have a different approach – an approach we didn‘t find that suitable for embedded/mobile development. And the JS frameworks that target mobile devices are flawed, too: either they target only a specific device or platform, or they provide far more than we do need and want.

What we wanted was a toolkit that offered a unified API across different platforms, but with the best implementation that was available for each platform. And we didn‘t want to ship code that implemented a certain feature for, say, an old Blackberry device to an iPhone. Plus, we wanted that API to be small – enough to assist us in our daily work, but not that bloated that we would ship code to a phone that we wouldn‘t need in most of our projects.

The Dojo Toolkit

We at uxebu have a strong affinity to the Dojo Toolkit – most of us are committers – for a good reason: we believe that it has done many things right, and that it has an excellent code quality. Thus, finding a starting point for our venture to what was later to be EmbedJS was pretty easy: It was the very core of the Dojo Toolkit, dojo._base.

We examined the code and split it into its smallest parts (features, how we call them in EmbedJS). After having the features all nicely separated, we could look for the best implementation for each of the platforms we wanted to support, and put it into a separate file. Then we optimized our code, tweaking here and there, but also using the code from dojo._base as it was, because it already was the best implementation.

And we needed to find means to put all these tiny parts together again, respecting any given dependecies, to create a file that then contained all the feature‘s implementations for a given platform – so the EmbedJS Tools were born, including our highly flexible build system.

API

EmbedJS‘ API is a small subset on Dojo‘s API. It provides around 70 methods, from the fields language enhancement (like Array manipulation or JSON), OO (like classes and inheritance), transport (like JSONP or XHR), event system (like connecting to DOM events or methods), DOM manipulation (like style or node creation) and misc things like Promises or pub/sub.

We kept the method signatures intact so that you can import your existing knowledge from working with the Dojo Toolkit to EmbedJS. There are only a few minor changes where we

a) found a new name highly useful, e.g dojo.io.script.get() is embed.jsonp()

b) we needed to strip given functionality for the sake of code size.

All methods in EmbedJS are available under the embed namespace as well as under the dojo namespace. I.e. to call the connect method you can use dojo.connect() and embed.connect().

The full API documentations is here, though it‘s undergoing some updates right now.

Summary

I didn‘t want to dig too deep in the first article on EmbedJS, but I guess you got the point: EmbedJS is a highly optimized Dojo core, and offers you a specialized, optimal build for a given platform. To explain how the build system works, how we manage dependencies, how we separate our features and our implementations and how one can create a project-oriented custom build will need another blog post – but if you are too curious, you are invited to take a look at the source code at github; a lot is self-explenatory there. A further source of information is the EmbedJS project page, as well as the EmbedJS Tools source.

IndexedDB is all the buzz right now, but it is pretty hard to find information about it’s implementations. In addition to that, the impls change on a weekly basis. So I figured it would be nice to summarize every now and then what has been happening around IDB in the last time.

And as Mozilla released a new Firefox 4 beta these days, and introduces changes to it’s IDB impl, let’s start the series with a brief summary of the most important of these changes.

Changes in 4.09b

• The indexedDB object has been renamed: moz_indexedDB is now mozIndexedDB.
• When opening a database, the event handed over to the onsuccess callback no more has a property called “result”, where a reference to the databasse used to be; it is now available at event.target.result (event.target contains the IDBRequest).
• This applies to many (all?) cases, where you’d expect the result of a request to live in event.result, where it now is in event.target.result.
• When creating an object store, the second argument, keyPath, now has to be passed as a property inside of an object: {keyPath: "keyPathName"}
• When creating an objectStore, the third argument, autoIncrement, is dropped and now has to be part of the second argument: {autoIncrement: true}
• Methods that read from an objectStore now only seem to work inside of a transaction; before, it was allowed to call methods like get directly.
• keyRange support seems to have increased. However, I’m not really interested (right now) in keyRanges and queries, so I did not play around with these.

Note that the changes for the createObjectStore arguments are against the spec – the 4.08b was compliant, the 4.09b is no more. But, on the other hand, it is now closer to the Chrome impl, which also expects an object as second argument (but has no support for autoIncrement).

What it does *not* have (yet)

The sync API. Alas. To me, the snychronous implementation is exactly what makes indexedDB that attractive – I’m not a fan of databases and I don’t like to start transactions, iterate over resultSets and the like… I want a store with easy access. Seriously, in most real life scenarios, an async approach won’t get you anywhere but displaying a waiting message to your user, because you can’t go on until you have the data from the store.

More info?

If you want to stay in touch with what happens around the indexedDB impl, the best place to go ist the Mozilla bugtracker. Um, yeah, reading browser source code can be no fun at times, but it gets you as close as you can get. Or, check what happens in the test files.

Need working code?

The *only* place to get reliable, working code is the test files in the repository. Seriously, there’s so much code out there claiming to work, but it simply doesn’t. And even if it did one day, it might break the other day, after changes have been made to the impl. And there will be *lots* of changes in the future.

Where are these test files?

In the repo. To see them online, go here: http://hg.mozilla.org/mozilla-central/file/ and then navigate to dom/indexedDB/test.

Something else?

Nope, not really. There don’t seem to be any *major* changes for 4.09b – I hope you still found this short summary useful.

I hope to continue this series about IDB and post about different IDB topics every now and then  – so if you want to know more about indexedDB in general, or something specific, just drop a line in the comments!

Further reading

• The spec: http://www.w3.org/TR/IndexedDB/
• The MDC article: https://developer.mozilla.org/en/IndexedDB

Please grab all phones you have and navigate to http://jensarps.de/tests/storage-tests/.

There are four tiny tests there. Please do them all and report your results for each phone. If you like, include your twitter handle so I can say thank you!

Your help is greatly appreciated!! Thanks a ton!!

Among other tests, the main thing I wanted to know is how long does it take to read or write 1000 entries from/to localStorage. The keys were of the form “key000″, the values “val000″, with increasing numbers. So the tests were using minimal data – only 6  bytes – but the results are still useful to get an idea of how it goes – and how the browsers perform compared to each other.

Desktop

All test were run on a Mac Mini. Opera and IE were tested on a virtual machine, so there might be an additional performance penalty based on virtualization. However, here are the numbers (average time needed to run the respective method 1000 times):

Safari 4:
- getItem: 1.4ms
- setItem: 1.7ms
- key: 0.8ms

Chrome:
- getItem: 240ms
- setIem: 280ms
- key: 230ms

Firefox:
- getItem: 77ms
- setItem: 2100ms
- key: 25ms

Opera:
- getItem: 19ms
- setItem: 20ms
- key: 18ms

IE8:
- getItem: 5ms
- setItem: 72ms
- key: 3.6ms

Mobile

As we can see, Safari is lightening fast. That made me want to run the tests on a mobile phone that has a modern Safari browser. I took an HTC Desire, wich comes with Android 2.1, and thus has a Safari capable of localStorage. Here are the results:

HTC Desire:
- getItem: 16.4ms
- setItem: 19.5ms
- key: 8.7ms

This is pretty amazing, considering that it is a phone. It still beats most of the browseres on the desktop.

If I find that test page again (uh, yeah, it’s lost somehow), I’ll run the tests again with newer browser releases and publish the link here, so you can run the tests yourself.

So here you go: dojox.storage.encrypted, a Blowfish encrypted storage. It sits on top of dojox.storage, and you get all the dojo storage manager goodness, mainly the automatic selection of the best storage provider available. It exposes the complete API that dojox.storage does. If an attacker gains access to the storage area, he can still nuke the storage, but the data found within will be useless.

I’m not completely happy with it’s architecture, and I didn’t have the time yet to fully test it with all providers, but it works pretty well with localStorage and the performance is not too bad. If you have a spare minute and are interested, feel free to to test it yourself (ah I know, you don’t have the time yourself, but I’d be thankful!)…

Usage

Just use it as you would use dojox.storage, except that you need to set your passphrase before you start working with the storage:

dojo.require("dojox.storage.encrypted");
var sto = dojox.storage.encrypted;
sto.setPassphrase("my super secret passphrase");
sto.put('key','value');
var value = sto.get('key');

It uses/requires dojox.encoding.crypto.Blowfish but that’s been in dojo for ages, so no worries.

To use it, just put the encrypted.js file in your dojox/storage directory.

Files

the code (encrypted.js)
the test page