<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Jens Arps &#187; Storage</title>
	<atom:link href="http://jensarps.de/tag/storage/feed/" rel="self" type="application/rss+xml" />
	<link>http://jensarps.de</link>
	<description></description>
	<lastBuildDate>Fri, 25 Nov 2011 16:16:34 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
		<item>
		<title>Working with IDBWrapper, Part 1</title>
		<link>http://jensarps.de/2011/11/25/working-with-idbwrapper-part-1/</link>
		<comments>http://jensarps.de/2011/11/25/working-with-idbwrapper-part-1/#comments</comments>
		<pubDate>Fri, 25 Nov 2011 16:16:34 +0000</pubDate>
		<dc:creator>Jens Arps</dc:creator>
				<category><![CDATA[Goodies to go]]></category>
		<category><![CDATA[How to]]></category>
		<category><![CDATA[Storage]]></category>
		<category><![CDATA[client-side storage]]></category>
		<category><![CDATA[IDB]]></category>
		<category><![CDATA[IndexedDB]]></category>
		<category><![CDATA[persistence]]></category>

		<guid isPermaLink="false">http://jensarps.de/?p=405</guid>
		<description><![CDATA[A while ago I released IDBWrapper. If you don&#8217;t know it, it&#8217;s a wrapper for IndexedDB, a current specification (in draft status) for an in-browser object store. It&#8217;s implemented in Firefox and Chrome, and somehow (as a plugin of sorts) also in IE, but, honestly, I don&#8217;t care about that too much. It is mainly [...]]]></description>
			<content:encoded><![CDATA[<p>A while ago I released <a href="https://github.com/jensarps/IDBWrapper" target="_blank">IDBWrapper</a>. If you don&#8217;t know it, it&#8217;s a wrapper for <a href="http://www.w3.org/TR/IndexedDB/" target="_blank">IndexedDB</a>, a current specification (in draft status) for an in-browser object store. It&#8217;s implemented in Firefox and Chrome, and <a href="http://html5labs.interoperabilitybridges.com/prototypes/indexeddb/indexeddb/info" target="_blank">somehow</a> (as a plugin of sorts) also in IE, but, honestly, I don&#8217;t care about that too much.</p>
<p>It is mainly meant to serve as an example implementation, so that you could have a look at the code and see how to work with IndexedDB. But I figured that people are also interested in actually using it, as it abstracts away many of the tedious internals of IndexedDB (like transactions) – and it is perfectly fine to use IDBWrapper for all non-overly-complex scenarios.</p>
<p>So here&#8217;s a tutorial about how to work with IDBWrapper and add a little background info about IndexedDB internals every now and then, instead of writing Yet-Another-Super-Technical-IDB-Blargh. Part one will cover some info about what IndexedDB is, getting IDBWrapper to run and how to read and write data to a store. Part two will be about querying the store.</p>
<p><span id="more-405"></span></p>
<h1>What is IndexedDB – why is it special?</h1>
<p>IndexedDB really differs from other client-side storage engines. It&#8217;s an ObjectStore, whereas localStorage is a key-value store and Sqlite is a relational database. Think of it as a database like those SQL thingies, but with more freedom but without a query language. Uh, roughly, yeah. You store and retrieve JavaScript objects (unlike localStorage) and you don&#8217;t have fixed tables with predefined columns and types (unlike Sqlite).</p>
<h1>Getting IDBWrapper</h1>
<p>You can <a href="https://github.com/jensarps/IDBWrapper/" target="_blank">download/clone/fork IDBWrapper at GitHub</a>. To include it in your page/project, you have two options: You can either include the IDBStore.js file via a script tag, or you can require it using an AMD loader like RequireJS.</p>
<h1>Let&#8217;s go!</h1>
<p>In this tutorial, we&#8217;ll create a people store where we store information about humans – a group of customers, for example.</p>
<p>&nbsp;</p>
<p><strong>Step one: open the store</strong><br />
<script type="text/javascript" src="https://gist.github.com/1391434.js"></script>A note on the properties we pass to the constructor:</p>
<ul>
<li>dbName, dbDescription, storeName: just choose meaningful names</li>
<li>dbVersion: start with &#8217;1.0&#8242;, and only change this when the structure of your db/store changes</li>
<li>keyPath: the property you want as unique primary key</li>
<li>autoIncrement: set this to true if you do not want to take care of the uniqueness of the primary key yourself</li>
<li>onStoreReady: a function to be called when your store is ready to work with</li>
</ul>
<p>&nbsp;</p>
<p>IDBwrapper will open the database, check if a store with the given name exists and if not, it will create it using the above properties. You&#8217;ll then get an IDBStore instance back that you can use to work with the store. Once the store is created, it&#8217;s persistent. But, if you need to delete and re-create it (for example, if you happened to pick the wrong value for keyPath), you can delete the store using the <code>deleteObjectStore</code> method.</p>
<p><em>Inside IndexedDB</em>: IDBWrapper works with one store; when you do a <code>new IDBStore()</code> it represents one store in an IndexedDB. If you want multiple stores inside of the same IndexedDB, you&#8217;d have to do a <code>new IDBStore()</code> for each store. This is an artificial limitation of IDBWrapper; if you work with IndexedDB directly, you can lock multiple stores when starting a transaction. This be useful when you want to update one store depending on the information found in another store, and you want to prevent this other store&#8217;s contents to change during your update action.</p>
<p>&nbsp;</p>
<p><strong>Step 2: storing a customer</strong><br />
<script type="text/javascript" src="https://gist.github.com/1391487.js?file=turorial.js"></script></p>
<p>Note how we don&#8217;t have an <code>id</code> property on the dude object, as we specified autoIncrement as true, so the database will take care of this. In the <code>onsuccess</code> callback we&#8217;ll get the id of the newly inserted record back. Also, see how we attached an array of emails to the record? In a relational database you&#8217;d set up a different table for email addresses – no need to do that in IndexedDB.</p>
<p><em>Inside IndexedDB</em>: Make sure to wait until the store is ready before operating on it! The whole nature of IndexedDB is asynchronous. Whatever you do, you will have to work with callbacks. There is, however, also a draft for a specification of a synchronous API, but that is not implemented in any browser (and I don&#8217;t think this is going to happen anytime soon).</p>
<p>&nbsp;</p>
<p><strong>Step 3: reading the customer back from the store</strong><br />
<script type="text/javascript" src="https://gist.github.com/1391509.js?file=tutorial.js"></script></p>
<p>We just pass the id we got after inserting to the get method, and there we have our record from the store. It&#8217;s exactly the same thing we stored earlier, with one exception: the object now has the <code>id</code> property which has been attached to it by the database.</p>
<p><em>Inside IndexedDB</em>: What we pass as first parameter to the <code>get</code> method is the <em>keyPath value</em>. That means when we pass 1 to it, the database will look for an object that has 1 as value in it&#8217;s keyPath property, which we set to &#8216;id&#8217; when we created the store.</p>
<p>&nbsp;</p>
<p><strong>Step 4: updating data</strong><br />
<script src="https://gist.github.com/1393216.js?file=turorial.js"></script></p>
<p>To update a record, you also use the put method: If there already is an object in the store with the same keyPath value (the id), it will simply overwrite it. Note that it will really, really overwrite the existing object, which means you cannot simply put a modified property of it, but have to put the whole object, including all properties.</p>
<p><em><strong>Inside IndexedDB</strong></em>: keyPath values are also Type-sensitive. That means, if the id of our dude is 1 and of type Number, you have to pass numeric 1 to the get method to retrieve the dude, and for updates the value of the id property needs to be numeric 1. If you use &#8220;1&#8243; as String, IndexedDB will think of it as a different value.</p>
<p>&nbsp;</p>
<p><strong>Step 5: getting all items</strong><br />
<script src="https://gist.github.com/1393243.js?file=tutorial.js"></script></p>
<p>Nothing really to say about it. Just one note: IDBStore has default error handlers for every async method that prints potential errors to the console. So if you&#8217;re just playing around you don&#8217;t need to pass your own handler to the methods.</p>
<p>&nbsp;</p>
<p><strong>Step 6: deleting an item</strong><br />
<script src="https://gist.github.com/1393271.js?file=tutorial.js"></script></p>
<p>Different browsers and versions return different values to the success callback; that&#8217;s why there is an extra check in the success callback.</p>
<p>&nbsp;</p>
<p><strong>Step 7: clearing the store</strong><br />
<script src="https://gist.github.com/1393287.js?file=tutorial.js"></script></p>
<p>If you need to clear the store from all stored entries, you can use the clear method. Note that this won&#8217;t reset Chrome&#8217;s autoIncrement counter.</p>
<h1>That&#8217;s it, thanks for listening!</h1>
<p>Now you know everything to do basic IndexedDB data operations with IDBWrapper. Make sure to check out the examples and the documentation!</p>
<p>Part two of this tutorial about querying the store will follow soon.</p>
<p><strong>Resources</strong></p>
<ul>
<li><a href="https://github.com/jensarps/IDBWrapper" target="_blank">IDBWrapper on GitHub</a></li>
<li><a href="https://github.com/jensarps/IDBWrapper/blob/master/README.md" target="_blank">IDBWrapper Documentation</a></li>
<li><a href="http://jensarps.github.com/IDBWrapper/example/" target="_blank">IDBWrapper Examples</a></li>
</ul>
]]></content:encoded>
			<wfw:commentRss>http://jensarps.de/2011/11/25/working-with-idbwrapper-part-1/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
		<item>
		<title>Creating a persistent Dojo Object Store</title>
		<link>http://jensarps.de/2011/04/27/creating-a-persistent-dojo-object-store/</link>
		<comments>http://jensarps.de/2011/04/27/creating-a-persistent-dojo-object-store/#comments</comments>
		<pubDate>Wed, 27 Apr 2011 16:30:49 +0000</pubDate>
		<dc:creator>Jens Arps</dc:creator>
				<category><![CDATA[Dojo Love]]></category>
		<category><![CDATA[Goodies to go]]></category>
		<category><![CDATA[How to]]></category>
		<category><![CDATA[Storage]]></category>
		<category><![CDATA[client-side storage]]></category>
		<category><![CDATA[dojo]]></category>
		<category><![CDATA[StorageJS]]></category>

		<guid isPermaLink="false">http://jensarps.de/?p=288</guid>
		<description><![CDATA[[Note: This is a cross-post. I also published this on the uxebu blog.]As of version 1.6, dojo comes with the new Dojo Object Store API. This is an awesome thing, as it greatly simplifies the work with data stores in Dojo. Everybody who had to do with the traditional dojo.data API felt it was overly [...]]]></description>
			<content:encoded><![CDATA[<p>[<em>Note: This is a cross-post. I also published this on the <a href="http://uxebu.com/blog/2011/04/27/creating-a-persistent-dojo-object-store/" target="_blank">uxebu blog</a>.</em>]</p><p>As of version 1.6, dojo comes with the new <a href="http://docs.dojocampus.org/dojo/store" target="_blank">Dojo Object Store API</a>. This is an awesome thing, as it greatly simplifies the work with data stores in Dojo. Everybody who had to do with the traditional dojo.data API felt it was overly complex and hard to use – this has finally changed now. There are also wrappers from and to the old and new APIs, so that you can do stuff like using your traditional data-aware widgets with a new Object Store. And the goodness doesn’t end here; but more on this later. If you haven’t done so yet, you might want to read the <a href="https://www.sitepen.com/blog/2011/02/15/dojo-object-stores/" target="_blank">excellent post on the new Dojo Object Stores</a> by Kris Zyp where he explains all the awesomeness he created.</p>
<p><span id="more-288"></span><br />
Dojo also comes with two fresh implementations for the API, a non-persistent memory store and a JsonRest store that interacts with a server through RESTful HTTP requests. You can also observe changes made to objects in the store. What is missing is a store that uses a client-side persistent backend, which would be useful for a couple of reasons (e.g., as one commenter in that post asks, to use it as chache store for dojo.store.Cache). As Kris already mentions, it’s a piece of cake to create one, so let’s go ahead and do it!</p>
<h2>Choosing the Backend</h2>
<p>The post mentions two possible candidates: dojox.storage and <a href="https://github.com/jensarps/StorageJS" target="_blank">StorageJS</a>, both already having a similar API, so making them compliant to the Dojo Object Store API is fairly easy. For this tutorial I choose StorageJS – because it is very lightweight and still provides all we need (and, well, for a couple of other reasons that don’t belong in this post). So, what is the exact API we need to comply to? Check it out <a href="http://docs.dojocampus.org/dojo/store#id2" target="_blank">in the docs</a>. See the first sentence in that paragraph? <em>“Every method in the API is optional, it’s presence indicating support for that feature.”</em> Wow, that’s nice!</p>
<h2>Basic compliance</h2>
<p>Let’s start with the methods get(), put(), add() and remove(). That will make our store pretty usable for most store scenarios.</p>
<p>First, create a ‘store’ object we can work with (StorageJS creates a global variable called <code class="codecolorer text mac-classic"><span class="text">storage</span></code> where all it’s methods reside):</p>
<p>‘store’ now contains everything that ‘storage’ does, but we can leave the original ‘storage’ object alone and modify our ‘store’ object if needed. Time for the first method, let’s start with <code class="codecolorer text mac-classic"><span class="text">get()</span></code>. StorageJS works with Strings as keys and values only, so we need to parse what we get from it.</p>
<p>Easy one, now move on to <code class="codecolorer text mac-classic"><span class="text">put()</span></code>. The Object Store API allows user to do stuff like this: <code class="codecolorer text mac-classic"><span class="text">store.put({foo: 'bar'}, {id: 3})</span></code> as well as <code class="codecolorer text mac-classic"><span class="text">store.put({id: 3, foo: 'bar'})</span></code>. Also, StorageJS only has the method set() instead of put() and add().</p>
<p>Easy, as well – you could write this as a one-liner. Now, <code class="codecolorer text mac-classic"><span class="text">add()</span></code>. It’s close to put, except that it won’t overwrite data that already exists. So we just need to check if something with the given id already exists and throw an error if so.</p>
<p>What about <code class="codecolorer text mac-classic"><span class="text">remove()</span></code>? Nothing to do there, it already has the desired signature and functionality.</p>
<p>That’s it, we’re done with basic compliance. Our ‘store’ object now implements the basic methods of the Dojo Object Store API and can be used by anything that can work with a Dojo Object Store – and it will keep it’s state persistent in the browser.</p>
<h2>More compliance? Querying!</h2>
<p>There’s one interesting thing in the API: the <code class="codecolorer text mac-classic"><span class="text">query()</span></code> method. Uh, yeah, we want that! Lucky us, Dojo already provides a query engine as well as a method that makes sure there are iterative methods and a <code class="codecolorer text mac-classic"><span class="text">total</span></code> property containing the number of hits available in our result (well, it also abstracts away the differences between sync and async results, but as StorageJS is purely synchronous, this is not important to us). First, define a queryEngine:</p>
<p>So, if we want our store to be able to run queries, all we need to do is to take the store’s data and hand it over to the query engine. We use StorageJS’ getAll() method to acquire the data. It will return an array of objects in the form { key: ‘theKey’, value: ‘theValue’}, but the queryEngine wants an array containing only the objects, so we need to do a conversion.</p>
<p>Now we can query our store for specific data, like so: <code class="codecolorer text mac-classic"><span class="text">var results = store.query({ prime: true });</span></code> This will give us all objects in the store that have a property called ‘prime’ and have that property set to true.</p>
<h2>Hierarchie and Transactions</h2>
<p>The API also mentions methods about data hierarchie and transactions, but I’m not going to cover these in this tutorial.</p>
<h2>Convenience and Optimization</h2>
<p>Currently, every time we run a query, we fetch all data again from our storage engine. So, if you don’t have a massive amount of data and want to query the store a lot, it might be wise to maintain a copy of all data in memory. There are other things you could optimize as well – or you could write your own queryEngine.</p>
<p>For your convenience,  <a href="http://statis.uxebu.com/~jens/dojo.store/PersistentLocal.js" target="_blank">here’s a wrapper</a> that creates a Dojo Object Store like we did above and accepts a useMemory parameter during instantiation that denotes whether a copy of the data should be kept in memory or not. It uses the traditional require/provide instead of the AMD format. If you want to use it, make sure you use the right namespace in the declare() call! (Or, not recommended, copy it in the /dojo/store directory.)</p>
<p>If you want to fool around with it in the console first, you can go <a href="http://static.uxebu.com/~jens/dojo.store/index-persistent-nomemory.html" target="_blank">here (non-memory-using)</a> or <a href="http://static.uxebu.com/~jens/dojo.store/index-persistent-usememory.html" target="_blank">here (memory-using)</a>, check out the source and fire up the console and try things out.</p>
<h2>Getting StorageJS</h2>
<p>The above code uses the features base and getAll, so you could do a specific build of StorageJS with these features or just <a href="https://github.com/jensarps/StorageJS/tree/master/builds?raw=true" target="_blank">grab a full build</a> of it for your desired storage engine (the storage-full-[engine].js files). If you don’t know about storage engines, all modern browser support localStorage, so <a href="https://github.com/jensarps/StorageJS/raw/master/builds/storage-full-localStorage.js" target="_blank">this is the one</a> what you might want.</p>
<p>&nbsp;</p>
]]></content:encoded>
			<wfw:commentRss>http://jensarps.de/2011/04/27/creating-a-persistent-dojo-object-store/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Using IndexedDB on Firefox</title>
		<link>http://jensarps.de/2011/03/30/using-indexeddb-on-firefox/</link>
		<comments>http://jensarps.de/2011/03/30/using-indexeddb-on-firefox/#comments</comments>
		<pubDate>Wed, 30 Mar 2011 14:20:26 +0000</pubDate>
		<dc:creator>Jens Arps</dc:creator>
				<category><![CDATA[Storage]]></category>
		<category><![CDATA[js]]></category>
		<category><![CDATA[persistence]]></category>

		<guid isPermaLink="false">http://jensarps.de/?p=289</guid>
		<description><![CDATA[[Note: This is a cross-post. I also published this on the uxebu blog.]As Firefox 4 is now stable, chances are that the async IDB (IndexedDB) API is not going to change anymore. But IDB is not localStorage – it can be a major pain to work with. So, here is a guide on how to use [...]]]></description>
			<content:encoded><![CDATA[<p>[<em>Note: This is a cross-post. I also published this on the <a href="http://uxebu.com/blog/2011/03/30/using-indexeddb-on-firefox/" target="_blank">uxebu blog</a>.</em>]</p><p>As Firefox 4 is now stable, chances are that the async IDB (IndexedDB) API is not going to change anymore. But IDB is not localStorage – it can be a major pain to work with. So, here is a guide on how to use IDB in Firefox as a key-value store.</p>
<p><span id="more-289"></span><br />
<em><strong>TL;DR</strong></em></p>
<p>There is example code available at the end of the post. If you don‘t want to read this lengthy post and/or prefer to jump right in the middle of the action, grab the example implementation and play around with it.</p>
<p>Yes, it is. But I still argue that in WebApp context a key-value store will cover about 90% of use cases. So I won‘t cover in this post how to create fancy key ranges or how to create multiple indices. But I will cover the five basic methods that you will need to work with any store: get, set, remove, getAll and clear – this will enable you to work with IDB and give you an understanding of how it works. You‘re then of course heartly encouraged to explore it and it‘s features further!</p>
<p>As long as the sync API is not there, all operations are async. That means that you will make requests. All the time. These requests will have an onerror and an onsuccess property, where you can put your callbacks/handlers in. If your request was successful, you will get an event in your callback. That event contains a lot of useful and useless information, but what you will want to look at in most cases is event.target.result – this is where your result lives in. In most cases, you will have to start a transaction before; inside of the transaction, you can access the objectStore and make your request. However, there are different types of transactions, but you will get the details below.</p>
<p>Yep, it‘s not that you can just start away and store your data, there‘s some work to do ahead…</p>
<h2>Open the Database</h2>
<p>Easy one:</p>
<p><code class="codecolorer text mac-classic"><span class="text">event.target.result</span></code> then contains a reference to the database. Store it, you will need it.</p>
<h2>Creating/Opening the ObjectStore</h2>
<p>That‘s a little tougher. Some operations, such as creating and deleting object stores, require the database to be in a „mutation transaction state“. Wow. Well, there‘s only one way to put the db into this state, and that‘s via a <code class="codecolorer text mac-classic"><span class="text">setVersion</span></code> request. If you are opening the db for the first time ever, you will need to do this anyway. You can, of course, change the version of the database anytime, which might be a useful thing if you, e.g., change the structure of how your objectStore saves the data. To check what version the database in the user‘s browser has, you can check the <code class="codecolorer text mac-classic"><span class="text">version</span></code> property in the reference to the database anytime.</p>
<p>Inside of the callback function, you can now check if your store already exists, or if you need to create it. The reference to your db contains an object called <code class="codecolorer text mac-classic"><span class="text">objectStoreNames</span></code>, which is a list of existing objectStores. As this is a DOMStringList, it features the handy <code class="codecolorer text mac-classic"><span class="text">contains()</span></code> method:</p>
<p>If the store is not there, you need to create it via the <code class="codecolorer text mac-classic"><span class="text">createObjectStore()</span></code> method. It takes two arguments: the name of the store, and an object with two properties: <code class="codecolorer text mac-classic"><span class="text">keyPath</span></code> and <code class="codecolorer text mac-classic"><span class="text">autoIncrement</span></code>. keyPath is… well… the path to the key. Think of it as the one column that is the primary key in your data table (you all did some MySQL at some time, I bet). You can do really fancy things with the keyPath in objectStores, but it‘s perfectly fine to just use something like „id“ as the keyPath. If you want to store an object and want to provide the keyPath, all you need to do is let the object you want to store have a property with the name of the keyPath, and a unique value. Uh, it‘s not that complicated, examples will follow. AutoIncrement is, well, autoIncrement. You have the option to let the objectStore take care for the uniqueness of it‘s data‘s keyPath properties, or do it yourself.</p>
<p>You immediately get a reference to the newly created objectStore, so save it.</p>
<p>If the store is already there, you need to open it. Um, well, you don‘t really have to open it, but you might want to obtain a reference to it, just to make sure that it really exists. To do this, you need to start a transaction via the <code class="codecolorer text mac-classic"><span class="text">transaction()</span></code> method. It accepts three arguments: an array of store names (the ones which you want to work with), an integer denoting whether you want to read and/or write during that transaction, and a timeout. There are constants for read/write you can use for better readability in your code, you can find them at <code class="codecolorer text mac-classic"><span class="text">IDBTransaction.READ_ONLY</span></code> and <code class="codecolorer text mac-classic"><span class="text">IDBTransaction.READ_WRITE</span></code>. The timeout parameter is optional. That method returns a transaction object that contains a method called <code class="codecolorer text mac-classic"><span class="text">objectStore</span></code>, which you can use to access your store. This is the „default“ way transactions work; the mutation transaction above is an eception to this, where you need to be in the callback of the transaction to modify the database.</p>
<p>In our case, we can start an „empty“ transaction:</p>
<p>There you have your reference to the store.</p>
<p>Now that the store is ready to use, it‘s time to store some data in it. To do so, you need to start a transaction, access the object store, and call the store‘s <code class="codecolorer text mac-classic"><span class="text">put</span></code> method:</p>
<p>That‘s all. This type of transaction is what you also use for other data manipulation tasks, for example to retrieve data:</p>
<p>You will need to provide the key of the stored entry to get it; as you need to do when you want to remove it:</p>
<p>If you want to get all data objects from the store:</p>
<p>And, finally, if you want to clear (i.e., delete all data objects) the store:</p>
<p>It isn‘t. Yeah, well, it is, as you can store objects and don‘t have to stringify them before. But basically, to get the real advantages of IDB, you will need to dive deeper into it. Features like keyRanges remove the need to do map/reduce action when you search for items in your store. But now you have an idea of how the whole thing works. If you want to go further, browse in the spec or the MDC docs and try things out!</p>
<p><strong>Example Code</strong></p>
<p>There is an example page over here: <a href="http://static.uxebu.com/~jens/indexeddb/ff-idb.html" target="_blank">http://static.uxebu.com/~jens/indexeddb/ff-idb.html</a>.</p>
<p>It uses an ObjectStore wrapper I created using dojo that covers the basic methods mentioned above, the JavaScript is here: <a href="http://static.uxebu.com/~jens/indexeddb/idb-ff.js" target="_blank">http://static.uxebu.com/~jens/indexeddb/idb-ff.js</a>.</p>
<p>When you open the page, open the console and you will see some messages there. Check out the source to see what‘s going on; it‘s basically a round trip through all the basic methods.</p>
<p><strong>Further Reading</strong></p>
<ul>
<li>The spec: <a href="http://www.w3.org/TR/IndexedDB/" target="_blank">http://www.w3.org/TR/IndexedDB/</a></li>
<li>The MDC article: <a href="https://developer.mozilla.org/en/IndexedDB" target="_blank">https://developer.mozilla.org/en/IndexedDB</a></li>
</ul>
<p>&nbsp;</p>
]]></content:encoded>
			<wfw:commentRss>http://jensarps.de/2011/03/30/using-indexeddb-on-firefox/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>localStorage Performance Test Results</title>
		<link>http://jensarps.de/2010/09/07/localstorage-performance-test-results/</link>
		<comments>http://jensarps.de/2010/09/07/localstorage-performance-test-results/#comments</comments>
		<pubDate>Tue, 07 Sep 2010 11:17:50 +0000</pubDate>
		<dc:creator>Jens Arps</dc:creator>
				<category><![CDATA[Experiments in Web]]></category>
		<category><![CDATA[Storage]]></category>
		<category><![CDATA[client-side storage]]></category>
		<category><![CDATA[js]]></category>
		<category><![CDATA[localStorage]]></category>
		<category><![CDATA[persistence]]></category>

		<guid isPermaLink="false">http://jensarps.de/?p=260</guid>
		<description><![CDATA[It&#8217;s been some time since I last updated this blog, mostly because there&#8217;s plenty going on these days. However, there&#8217;s something I&#8217;ve been wanting to publish for quite some time now: The results of the localStorage performance tests I ran several weeks ago. As I am currently working on performance tests for Mozilla&#8217;s IndexedDB implementation, [...]]]></description>
			<content:encoded><![CDATA[<p>It&#8217;s been some time since I last updated this blog, mostly because there&#8217;s plenty going on these days. However, there&#8217;s something I&#8217;ve been wanting to publish for quite some time now: The results of the localStorage performance tests I ran several weeks ago. As I am currently working on performance tests for Mozilla&#8217;s IndexedDB implementation, which is available in latest Minefield releases, I got reminded that there are still other results to publish – so, here we go:</p>
<p><span id="more-260"></span></p>
<p>Among other tests, the main thing I wanted to know is how long does it take to read or write 1000 entries from/to localStorage. The keys were of the form &#8220;key000&#8243;, the values &#8220;val000&#8243;, with increasing numbers. So the tests were using minimal data – only 6  bytes – but the results are still useful to get an idea of how it goes – and how the browsers perform compared to each other.</p>
<p><strong>Desktop</strong></p>
<p>All test were run on a Mac Mini. Opera and IE were tested on a virtual machine, so there might be an additional performance penalty based on virtualization. However, here are the numbers (average time needed to run the respective method 1000 times):</p>
<p>Safari 4:<br />
- getItem: 1.4ms<br />
- setItem: 1.7ms<br />
- key: 0.8ms</p>
<p>Chrome:<br />
- getItem: 240ms<br />
- setIem: 280ms<br />
- key: 230ms</p>
<p>Firefox:<br />
- getItem: 77ms<br />
- setItem: 2100ms<br />
- key: 25ms</p>
<p>Opera:<br />
- getItem: 19ms<br />
- setItem: 20ms<br />
- key: 18ms</p>
<p>IE8:<br />
- getItem: 5ms<br />
- setItem: 72ms<br />
- key: 3.6ms</p>
<p><strong>Mobile</strong></p>
<p>As we can see, Safari is lightening fast. That made me want to run the tests on a mobile phone that has a modern Safari browser. I took an HTC Desire, wich comes with Android 2.1, and thus has a Safari capable of localStorage. Here are the results:</p>
<p>HTC Desire:<br />
- getItem: 16.4ms<br />
- setItem: 19.5ms<br />
- key: 8.7ms</p>
<p>This is pretty amazing, considering that it is a phone. It still beats most of the browseres on the desktop.</p>
<p>If I find that test page again (uh, yeah, it&#8217;s lost somehow), I&#8217;ll run the tests again with newer browser releases and publish the link here, so you can run the tests yourself.</p>
]]></content:encoded>
			<wfw:commentRss>http://jensarps.de/2010/09/07/localstorage-performance-test-results/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Don&#8217;t use Cookies</title>
		<link>http://jensarps.de/2010/04/10/dont-use-cookies-use-localstorage/</link>
		<comments>http://jensarps.de/2010/04/10/dont-use-cookies-use-localstorage/#comments</comments>
		<pubDate>Sat, 10 Apr 2010 17:54:01 +0000</pubDate>
		<dc:creator>Jens Arps</dc:creator>
				<category><![CDATA[Browser Struggle]]></category>
		<category><![CDATA[How to]]></category>
		<category><![CDATA[Storage]]></category>
		<category><![CDATA[client-side storage]]></category>
		<category><![CDATA[cookies]]></category>
		<category><![CDATA[localStorage]]></category>
		<category><![CDATA[persistence]]></category>

		<guid isPermaLink="false">http://jensarps.de/?p=216</guid>
		<description><![CDATA[– or: How to persist data in the 21st century. The common way to persist data on the client side – application state, offline data, whatever –  still is to use cookies. But times have changed, and so have browsers, and there are better ways to do it today. But why are cookies that bad? [...]]]></description>
			<content:encoded><![CDATA[<p>– or: <strong>How to persist data in the 21st century</strong>.</p>
<p>The common way to persist data on the client side – application state, offline data, whatever –  still is to use cookies. But times have changed, and so have browsers, and there are better ways to do it today.</p>
<p>But why are cookies that bad? Well, here are the top three reasons:</p>
<ol>
<li>Of all client side storage mechanisms, cookies have the worst limitation in size (4k if you want to stay IE-safe)</li>
<li>Cookies are sent to the server on every requests that matches the cookie domain – inlcuding XHR calls (aka. <em>How to slow down your AJAX app</em>)</li>
<li>Cookies perform bad, can be easily disabled, and, oh well, they are sooo 1995…</li>
</ol>
<p>What else to use? There are several options, let&#8217;s start with the best:<br />
<span id="more-216"></span></p>
<h2><strong>localStorage</strong></h2>
<p>This is the storage mechanism proposed by the HTML5 draft. It offers you a fast, reliable key-value store with a whopping 5M (!!) of storage space. And here&#8217;s the kicker: It&#8217;s implemented in all current major browser releases. That is, FF, Safari, Opera, IE and Chrome/Chromium. Yes, you name it, all of them, the full list. To read more about localStorage, check the <a href="http://dev.w3.org/html5/webstorage/" target="_blank">HTML5 draft</a>. But here&#8217;s all you need to know in short:</p>
<p><strong>Storage Methods</strong></p>
<p>To test if the user agent supports localStorage, just test for the existence of <code>window.localStorage</code>. Working with the storage object is trivial:</p>
<p>Setting a value:</p>
<pre>localStorage.setItem('key','value');</pre>
<p>Though the draft says the user agent should write a &#8220;structured clone&#8221; of the value to store, don&#8217;t think that you can store objects – you can only store strings. To store objects, you need to convert them to JSON before.</p>
<p>To retrieve a value, there are two options. First, you can get a value by it&#8217;s key:</p>
<pre>var value = localStorage.getItem('key');</pre>
<p>The user agent will return the value for the key, or, if it doesn&#8217;t find a matching record, <code>null</code>. Also, you can retrieve a key by it&#8217;s index:</p>
<pre>var value = localStorage.key(n);</pre>
<p>The storage object has a <code>length</code> property, and n is a number between zero and length-1. You do not get the index of a value when inserting it, but you can use the <code>key()</code> method when iterating over all items in the store using the <code>length</code> attribute.</p>
<p>There is another method to retrieve a value, but as this not documented (at least I didn&#8217;t find anything about it), I do not recommend it:</p>
<pre>var value = localStorage.key;</pre>
<p>Deleting an item:</p>
<pre>localStorage.removeItem('key');</pre>
<p>Deleting the whole storage:</p>
<pre>localStorage.clear();</pre>
<p>This clears only the storage objects of the same storage area. That is, storage objects of the current domain.</p>
<p>So far, everything is pretty trivial, but let&#8217;s look at</p>
<p><strong>Storage Exceptions</strong></p>
<p>When accessing the storage object, a user agent may raise an <code>SECURITY_ERR</code> exception – but I have never managed to trigger one. When setting an item, a user agent may raise an <code>QUOTA_EXCEEDED_ERR</code> exception. This is not only for the case when storage space is at it&#8217;s limit, but may be raised for other reasons as well. Again, I have never managed to trigger it, but it may be wise to wrap a <code>setItem</code> call in a try/catch block.</p>
<p><strong>Storage Event</strong></p>
<p>Now things get tricky, and the different browsers have different (if any) implementations. Ok, first, the theory: methods that modify the storage object (<code>setItem()</code>,<code>removeItem()</code>,<code>clear()</code>) do not report if – or when – the changes are in effect. To get notified about this, you need to connect to the storage event:</p>
<pre>window.addEventListener("storage", function(evt){
    // a change to the storage object has been attempted.
}, false);</pre>
<p>The event should have the following attributes:</p>
<ul>
<li>key</li>
<li>oldValue</li>
<li>newValue</li>
<li>url (the address of document object whose storage object was modified)</li>
<li>storageArea (the affected storage object)</li>
</ul>
<p>With this information, you can easily determine to what change the storage event belongs. But, alas, that event is not fully implemented everywhere. Here&#8217;s what I found out:</p>
<p><strong>FF 3.6.3 </strong>lets you connect to the event, and fires it according to the draft. But it does not contain *any* of the fields mentioned above.</p>
<p><strong>Safari 4.0.5</strong> lets you connect to the event, and fires it according to the draft. It&#8217;s event contains all attributes except url.</p>
<p>on <strong>Chrome 5 &amp; Chromium</strong> I couldn&#8217;t connect to the event at all or it wouldn&#8217;t let my execute code in my callback function – I have no idea if it&#8217;s even fired.</p>
<p>Latest <strong>WebKit nightly</strong> wouldn&#8217;t report anything to me, like  Chrome.</p>
<p><strong>Opera 10 </strong>lets you connect to the event, fires it accordingly, and the event contains all attributes (woot!)</p>
<p><strong>IE 8</strong> didn&#8217;t report anything back to me, too.</p>
<p>I used <a href="http://www.jensarps.de/tests/storage.html" target="_blank">this simple test file</a> to check for the event. Feel free to test other browsers and report what you found!</p>
<h2><strong>Other options</strong></h2>
<p>If the browser is too old to have localStorage, there are still other options to persist data: Firefox 2 had globalStorage, which behaves similar to localStorage. IE has userData beahvior since version 5.5. Safari has it&#8217;s built-in Sqlite database, but I don&#8217;t know since when. Plus, there&#8217;s also plugin-based storage via Flash or Gears. Only if they all fail, you should get back to cookies.</p>
<p>Of course, it would be a lot of work to write wrappers for all these storage mechanisms – but, hey, you don&#8217;t have to, others have already done it. There are several toolkits that do the work for you, and I will present three of them.</p>
<p><strong>Dojo</strong></p>
<p>The dojo toolkit has dojox.storage. It currently offers support for Flash, Gears and globalStorage. But with the files you can grab at my <a href="http://jensarps.de/2010/01/04/persistent-local-storage-with-dojo/" target="_blank">previous post on storage</a>, you gain access to localStorage, userData behavior and cookies. Working with dojox.storage is pretty simple, and you don&#8217;t have to worry about anything: dojo&#8217;s storage manager will pick the best available provider. Here&#8217;s a simple example:</p>
<pre>dojo.require("dojox.storage");
dojox.storage.put('key','value');
var value = dojox.storage.get('key');</pre>
<p>That&#8217;s it. Simple as that. You do have more options, though; you can also specify a namespace for your keys – so you can simulate working with different &#8220;tables&#8221; in one store. You can also specify a callback function for the <code>put()</code> method, to get notified when the changes are in effect and if the change was successful.</p>
<p><strong>Lawnchair</strong></p>
<p>Lawnchair has been created by <a href="http://twitter.com/brianleroux" target="_blank">Brian Leroux</a> from Nitobi. Working with it is dead simple, too. It supports localStorage, userData behavior, Gears, Webkit&#8217;s Sqlite and Cookies. It works more like a document store (you can save an object w/o specifying a key, for example), but you can also use it like a straight key/value store. In opposition to dojo&#8217;s storage system, Lawnchair works completely asnc, so you have to pass a callback function to a get() method to retrieve a value (being async is not a drawback, and it is neccessary to work with WebKit&#8217;s Sqlite). You can find some docs <a href="http://brianleroux.github.com/lawnchair/" target="_blank">here</a> and get it over <a href="http://github.com/brianleroux/lawnchair" target="_blank">at Github</a>.</p>
<p><strong>Persist.js</strong></p>
<p>Persist.js has support for localStorage, globalStorage, Gears, Flash, userData behavior and Cookies. It also works completely async. You can get it <a href="http://github.com/lloyd/persist-js" target="_blank">at Github</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://jensarps.de/2010/04/10/dont-use-cookies-use-localstorage/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
		<item>
		<title>persistent local storage with dojo</title>
		<link>http://jensarps.de/2010/01/04/persistent-local-storage-with-dojo/</link>
		<comments>http://jensarps.de/2010/01/04/persistent-local-storage-with-dojo/#comments</comments>
		<pubDate>Mon, 04 Jan 2010 21:46:40 +0000</pubDate>
		<dc:creator>Jens Arps</dc:creator>
				<category><![CDATA[Dojo Love]]></category>
		<category><![CDATA[Experiments in Web]]></category>
		<category><![CDATA[Goodies to go]]></category>
		<category><![CDATA[Storage]]></category>
		<category><![CDATA[client-side storage]]></category>
		<category><![CDATA[dojo]]></category>
		<category><![CDATA[dojox]]></category>
		<category><![CDATA[js]]></category>
		<category><![CDATA[local]]></category>
		<category><![CDATA[localStorage]]></category>
		<category><![CDATA[persistence]]></category>

		<guid isPermaLink="false">http://jensarps.de/?p=140</guid>
		<description><![CDATA[Last year, Brian Leroux&#8216;s Lawnchair caught my interest – it is an easy and fast way to access local persistant local storage. Uh, persistant local storage? Ahm, yeah, in case you don&#8217;t know, that means a client-side storage, be it in a browser, or in an Air app. It&#8217;s not very popular, but that concept [...]]]></description>
			<content:encoded><![CDATA[<p>Last year, <a href="http://twitter.com/brianleroux" target="_blank">Brian Leroux</a>&#8216;s <a href="http://brianleroux.github.com/lawnchair/" target="_blank">Lawnchair</a> caught my interest – it is an easy and fast way to access local persistant local storage.</p>
<p><strong>Uh, persistant local storage?</strong></p>
<p>Ahm, yeah, in case you don&#8217;t know, that means a client-side storage, be it in a browser, or in an Air app. It&#8217;s not very popular, but that concept has been around for a long time. So why local? There are two major reasons for it: First, we need this for apps and tools that work offline – and apps and tools that work online but need an offline backup and sync later. <a href="http://twitter.com/kriszyp" target="_blank">Kris Zyp</a> wrote a <a href="http://www.sitepen.com/blog/2008/09/23/effortless-offline-with-offlinerest/" target="_blank">post about the JsonRestStore and OfflineRest</a> back in 2008, he goes a little into detail there. Secondly, we need it for apps and tools that rely on persistence for other reasons, no matter if online or offline – like it happended to me. When I ran into Lawnchair, I had the idea to build a tool that was monitoring CSS selector usage on websites and apps (you know how you sometimes lose control about CSS selectors in webapps…). To achieve this, I needed to store data locally, and persistent.<br />
<span id="more-140"></span><br />
<strong>What dojo offers<br />
</strong></p>
<p>As I was using dojo&#8217;s excellent CSSRuleStore to find out what selectors where used, I wanted to switch from Lawnchair to a dojo solution. Dojo has dojox.storage (the above mentioned OfflineRest uses dojox.storage), which provides wrappers for the following backends: Gears, Flash, globalStorage and Adobe Air (globalStorage is Firefox&#8217;s early implementation of the early WhatWG storage draft). Hm. Not that much. Gears and Flash are 3rd party solutions, globalStorage is a Firefox-only-thingy (and old, though it&#8217;s still supported in current Versions) and Air is non-browser. That means: No IE without plugins. No Safari when Flash is not available.</p>
<p><strong>What dojo does not offer (but should)</strong></p>
<p>Guaranteed, reliable support for *any* browser. Without depending on 3rd party plugins. As of now, to have this, you&#8217;d need to use Lawnchair or <a href="http://github.com/lloyd/persist-js" target="_blank">persist,js</a>, which is pretty complete. Speaking of it, let me quote from it&#8217;s readme (it&#8217;s a good read on local persistent storage in general):</p>
<blockquote><p>The most notable attempt at addressing this problem [storing client-side persistent data] is probably Dojo Storage.  Unfortunately, Dojo Storage does not support Internet Explorer without Flash, and it does not support Safari or other WebKit-based<br />
browsers at all (at least, not without Flash).  Also, Dojo Storage is<br />
not standalone; it requires a several other Dojo components in order to<br />
operate.</p></blockquote>
<p>Whoa, even other JS tools mention the lack of support in dojo…</p>
<p><strong>New providers</strong></p>
<p>Attached to this post are wrappers for the following mechanisms:</p>
<ul>
<li>localStorage: this is the current spec, and it is supported by IE8, Firefox 3, Safari 4, Opera 10, Chrome 5.</li>
<li>userData behavior: this is Microsofts client-side storage for IE &lt; 8.</li>
<li>cookie: worst case fallback solution…</li>
</ul>
<p>With these three, there&#8217;s an almost 100% coverage of browser situations (see storage overview). There are still no-go scenarios, of course, such as IE7 with userData disabled and cookies disabled, but there&#8217;s no way to address them all.</p>
<p><strong>Notes on BehaviorStorageProvider</strong></p>
<p>This one uses the <a href="http://msdn.microsoft.com/en-us/library/ms531424%28VS.85%29.aspx" target="_blank">userData behavior</a>, present in IE from version 5 up. It works with DOM elements that have a special behavior assigned. You can then use  set/get/removeAttribute() methods to store data. It has some limitations though – of course…</p>
<ul>
<li>Maximum size varies from 64k to 1M – depending on the zone the site is in. So, you can&#8217;t rely on more than 64k.</li>
<li>UserData can be switched off in security settings.</li>
<li>I couldn&#8217;t find a way to read out stored keys if you don&#8217;t know them. Maybe this is some security thingy or just a bug – but it means that some methods in the provider don&#8217;t work. (that is, getNamespaces, getKeys and clear). To work around this, we have to take some of the precious storage space and store the keys and namespaces seperately.</li>
</ul>
<p><strong>Usage</strong></p>
<p>Copy the the providers into your <code>/dojox/storage</code> directory and add the needed <code>dojo.require</code>s to <code>/dojox/storage/_common.js</code>:</p>
<pre>dojo.require("dojox.storage.LocalStorageProvider");
dojo.require("dojox.storage.GearsStorageProvider");
dojo.require("dojox.storage.BehaviorStorageProvider");
//&gt;&gt;excludeStart("offlineProfileExclude", kwArgs.dojoxStorageBuildOption == "offline");
dojo.require("dojox.storage.WhatWGStorageProvider");
dojo.require("dojox.storage.FlashStorageProvider");
//&gt;&gt;excludeEnd("offlineProfileExclude");
dojo.require("dojox.storage.CookieStorageProvider");</pre>
<p><strong>What&#8217;s left?</strong></p>
<p>Well, it would be pretty cool to have a dojo.data API compliant store that uses dojox.storage as backend, so that you could write your apps using dojo.data calls and have a reliable offline backup. I started working on that, but then came Christmas, parents, vacation, new year and a lot of beer. But hey, it&#8217;s 2010 now, and there&#8217;s plenty of time left!</p>
<p><strong>Storage Overview</strong><br />
<!-- .overviewTable td {font-size: 10px; padding:2px 5px;} --></p>
<table class="overviewTable" border="0">
<tbody>
<tr>
<td>IE 6 / IE7</td>
<td>BehaviorStorageProvider<br />
FlashStorageProvider<br />
GearsStorageProvider<br />
CookieStorageProvider</td>
</tr>
<tr>
<td>IE 8</td>
<td>LocalStorageProvider<br />
GearsStorageProvider<br />
FlashStorageProvider<br />
CookieStorageProvider</td>
</tr>
<tr>
<td>Safari 3</td>
<td>FlashStorageProvider<br />
CookieStorageProvider</td>
</tr>
<tr>
<td>Safari 4</td>
<td>LocalStorageProvider<br />
FlashStorageProvider<br />
CookieStorageProvider</td>
</tr>
<tr>
<td>Chrome 5</td>
<td>LocalStorageProvider<br />
FlashStorageProvider<br />
CookieStorageProvider</td>
</tr>
<tr>
<td>Firefox 2</td>
<td>WhatWGStorageProvider (= globalStorage)<br />
GearsStorageProvider<br />
FlashStorageProvider<br />
CookieStorageProvider</td>
</tr>
<tr>
<td>Firefox 3</td>
<td>LocalStorageProvider<br />
GearsStorageProvider<br />
FlashStorageProvider<br />
CookieStorageProvider</td>
</tr>
<tr>
<td>Opera 10</td>
<td>LocalStorageProvider<br />
FlashStorageProvider (?)<br />
CookieStorageProvider</td>
</tr>
</tbody>
</table>
<p><strong>Files</strong></p>
<p><a href="http://jensarps.de/tests/dojo_tests/dojo-release-1.4.0-src/dojox/storage/LocalStorageProvider.js" target="_blank">LocalStorageProvider.js</a><br />
<a href="http://jensarps.de/tests/dojo_tests/dojo-release-1.4.0-src/dojox/storage/BehaviorStorageProvider.js" target="_blank">BehaviorStorageProvider.js</a><br />
<a href="http://jensarps.de/tests/dojo_tests/dojo-release-1.4.0-src/dojox/storage/CookieStorageProvider.js" target="_blank">CookieStorageProvider.js</a><br />
<a href="http://jensarps.de/tests/dojo_tests/dojo-release-1.4.0-src/dojox/storage/tests/test_storage.html" target="_blank">Storage Test Page</a></p>
]]></content:encoded>
			<wfw:commentRss>http://jensarps.de/2010/01/04/persistent-local-storage-with-dojo/feed/</wfw:commentRss>
		<slash:comments>8</slash:comments>
		</item>
	</channel>
</rss>

