Author Archive

With more and more browsers adding support for the
HTML5 Web Storage API,
the future is looking good for web applications that need persistent
client-side data storage. Unfortunately, maintaining compatibility with
older browsers that don’t yet support Web Storage can be a challenge, and
dropping support for older browsers is rarely an acceptable option.

The Storage Lite
YUI 3 Gallery module, which was developed for use on
Yahoo! Search, aims to solve this
problem by providing a simple, lightweight API wrapper for a variety of
persistent client-side storage mechanisms with no external plugin
dependencies of any kind (not even Flash). It weighs in at about 2.6KB after
minification and before gzip, is similar to the HTML5 localStorage API, and
works in IE6+, Firefox 2+, Safari 3.1+, Chrome 4+, and Opera 10.5+.

Apart from being based on YUI 3, Storage Lite differs from the existing
YUI 2 Storage Utility
in several ways. YUI 2 Storage is a robust and extensible implementation
that stays more faithful to the HTML5 localStorage and sessionStorage APIs
(for example, it provides a change event and a key() method,
which Storage Lite does not). YUI 2 Storage also falls back on Flash or
Gears storage for older browsers, which can in some cases provide better
persistence and more storage capacity. Storage Lite trades extensibility and
plugin-based fallbacks for lighter weight and better performance, and
specifically focuses on emulating only localStorage.

Take a look at this
persistent notepad example to see Storage Lite in action, or read through
the following non-interactive code sample for a quick tour of the API.

First, include the script:

<script src="http://yui.yahooapis.com/combo?3.0.0/build/yui/yui-min.js&gallery-2010.02.22-22/build/gallery-storage-lite/gallery-storage-lite-min.js"></script>

In your implementation code, listen for the storage-lite:ready
event, which is fired when the storage mechanism is ready for use. After the
storage mechanism is ready, you can use the API:

YUI().use('gallery-storage-lite', function (Y) {

    // For full compatibility with IE 6-7 and Safari 3.x, listen for the
    // storage-lite:ready event before making storage calls. If you're not
    // targeting those browsers, you can safely ignore this step.
    Y.StorageLite.on('storage-lite:ready', function () {

        // To store an item, pass a key and a value (both strings) to setItem().
        Y.StorageLite.setItem('kittens', 'fluffy and cute');

        // If you set the optional third parameter to true, you can use any
        // serializable object as the value and it will automatically be stored
        // as a JSON string.
        Y.StorageLite.setItem('pies', ['apple', 'pumpkin', 'pecan'], true);

        // To retrieve an item, pass the key to getItem().
        Y.StorageLite.getItem('kittens');    // => 'fluffy and cute'

        // To retrieve and automatically parse a JSON value, set the optional
        // second parameter to true.
        Y.StorageLite.getItem('pies', true); // => ['apple', 'pumpkin', 'pecan']

        // The length() method returns a count of how many items are currently
        // stored.
        Y.StorageLite.length(); // => 2

        // To remove a single item, pass its key to removeItem().
        Y.StorageLite.removeItem('kittens');

        // To remove all items in storage, call clear().
        Y.StorageLite.clear();

    });

});

Data stored using Storage Lite is persisted across pageviews and browser
restarts^*, and is accessible only from the same domain in which
it was stored. Behind the scenes, Storage Lite uses the following storage
mechanisms, automatically choosing the best one that’s supported:

• Firefox 3.5+, Chrome 4+, Safari 4+, IE8, Opera 10.5+: HTML5 localStorage — these modern browsers all support the core localStorage functionality defined in the HTML5 draft.
• Firefox 2.x and 3.0.x: Gecko globalStorage, an early API similar to HTML5’s localStorage.
• Safari 3.1 and 3.2: HTML5 Database Storage, because Safari 3.1 and 3.2 don’t support HTML5 localStorage.
• IE6, IE7: userData persistence, a rarely used IE feature for associating string data with an element on a web page and persisting it between pageviews.

For more details, see the
Storage Lite GitHub project
and the API
reference.

* Caveat: IE6 and IE7 persist data across pageviews, but not across
browser restarts.

Update (July 12, 2010): While the results described in this article are accurate for HTML pages, new tests have revealed very different cache limits for CSS and JS resources. The updated results are described in Mobile Browser Cache Limits, Revisited.

In early 2008, Wayne Shea and Tenni Theurer wrote a YUI Blog post on iPhone Cacheability in which they shared the results of research into various characteristics and limitations of Mobile Safari’s cache in iPhone OS 1.x. Among other things, they found that individual components larger than 25KB were not cached, and that there was a maximum total cache size of between 475KB and 500KB.

Much has changed since then. We’ve seen two new major releases and many minor releases of the iPhone OS (now iOS), and several other mobile devices with highly capable browsers have appeared to challenge the iPhone. Stoyan Stefanov found, in late 2009, that the iPhone’s cache limits had changed (sadly, for the worse). But where do things stand now? And what about those non-iOS browsers?

Background

Browsers have two types of caches that we’re concerned with for the purposes of these tests:

• The component cache, or object cache, stores individual files. HTML, CSS, JavaScript, and images all go into the component cache. Whenever it needs one of these components, the browser first checks the cache before making a network request.
• The page cache, also known as the back/forward cache, stores an entire page and all of its components, as well as their current state. When you use the back or forward button, the browser will load the page from the page cache if possible.

The HTML5 application cache is another type of cache that’s widely supported by mobile browsers. Browser makers already do a good job of documenting the limits of the application cache, so I didn’t include it in my testing. More on the application cache later.

Devices Tested

I tested the following mobile browser/platform combinations:

• Android 2.1 (Nexus One)
• Mobile Safari on iOS 3.1.3 (1st-gen iPhone)
• Mobile Safari on iOS 3.2 (iPad)
• Mobile Safari on iOS 4.0 (iPhone 3GS)
• Mobile Safari on iOS 4.0 (iPhone 4)
• webOS 1.4.1 (Palm Pre Plus)

Note: With the exception of Mobile Safari on iOS 4.0, I tested only one device in each category. If there are variations between individual devices or differences based on installed software beyond the OS, my tests would not detect those variations. These particular devices were tested because they’re the ones I had access to, not because I consider them to be more important than other devices.

Methodology

Cache testing is tedious, but relatively simple.

I wrote a tiny Sinatra app (fork it on GitHub!) that generates a response consisting of a requested number of pseudorandom alphanumeric and whitespace bytes. The responses can be served either gzipped or uncompressed. The following far-future expiration response headers are sent to ensure that all responses are considered cacheable:

Cache-Control: max-age=315360000
Expires: Fri, 01 May 2020 03:47:24 GMT

Over my local network, I then manually performed the following steps on each device to test the component cache:

1. Load the cache test index page.
2. Tap on a link to a component of a particular size, ranging from 5KB to 20MB, and wait for it to finish loading.
3. Tap the back button.
4. Tap the same link again. Observe whether the random characters are the same, and whether the server console prints a log entry for a request, to determine whether the component was cached in step 2.
5. Repeat and adjust component sizes as necessary to determine the maximum component size that will be cached.

To test the page cache, I performed essentially the same steps except that instead of tapping the link again in step 4, I tapped the browser’s forward button, causing it to use the page cache rather than the component cache.

Support for ETag and Last-Modified was determined by tweaking the server to send the appropriate ETag or Last-Modified response headers (in separate tests) and to omit the far-future expiration headers. I then inspected the request headers received by the server to verify that the browser sent the expected If-None-Match or If-Modified-Since headers on step 4.

Results

I tested with gzip both enabled and disabled, but I found that gzip had no effect on cacheability on any device. The uncompressed component size is what matters in all cases, regardless of whether or not that component is served gzipped. As such, all component sizes mentioned here are uncompressed sizes.

The table below illustrates my overall findings.

Table: Mobile browser cache characteristics

Browser/OS/Device	Single Component Limit	Total Component Limit	Page Cache Size Limit	Supports Last-Modified	Supports ETag	Survives Power Cycle
Android 2.1 (Nexus One)	~2MB (~2,048,000b)	~2MB (~2,048,000b)	∞ 2	Yes	Yes	Yes
Mobile Safari, iOS 3.1.3 (1st-gen iPhone)	0b 1	0b 1	∞ 2	No	No	No
Mobile Safari, iOS 3.2 (iPad)	25.6KB (26,214b)	~281.6KB (~288,354b)	25.6KB (26,214b)	Yes	Yes	No
Mobile Safari, iOS 4.0 (iPhone 3GS)	51.199KB (52,428b)	~1.05MB (~1,100,988b)	∞ 2	Yes	Yes	No
Mobile Safari, iOS 4.0 (iPhone 4)	102.399KB (104,857b)	~1.9MB (~1,992,283b)	∞ 2	Yes	Yes	No
webOS 1.4.1 (Palm Pre Plus) 3	~1MB (~1,048,576)	?	~1MB (~1,048,576)	No	No	Yes

Android

The Android browser exhibited the best cache behavior of all devices tested. While it appears to impose no limit on the size of individual components, the total cache size seems to be limited to approximately 2MB, which means that individual components are effectively limited to 2MB as well.

The page cache appeared to impose no limit on the size of individual pages, happily caching every byte I threw at it until the available RAM was exhausted and the browser crashed.

I was pleasantly surprised to find that Android’s component cache survived both browser restarts and power cycles, a feat none of the iOS devices was able to match.

Possible caveat: A review of Android’s WebKit source tree leads me to believe that its cache limits may adapt based on the amount of RAM and/or flash memory available on the particular device on which it’s running. If true, these numbers may only be applicable to the Nexus One. In fact, if the cache size adapts based on unused memory rather than total memory, these numbers may only be applicable to my Nexus One.

I could be mistaken, but the differences in the iOS 4.0 test results on the iPhone 3GS and iPhone 4 support this theory. (Android and Mobile Safari are both WebKit-based browsers, so they may have this behavior in common.) If you’re familiar with the WebKit source and can shed more light on this, please get in touch with me.

iOS

Results varied wildly across the three most recent versions of iOS. Astonishingly, Mobile Safari on iOS 3.1.3 did not cache components of any size, despite having an apparently unlimited page cache size. This is troubling since it means iOS 3.1.3 users are likely getting a suboptimal browsing experience, especially if they aren’t using wifi. The unlimited page cache size does little good here, since it only comes into play for back/forward navigations. This behavior is a significant change from what others observed in previous iOS releases and I can’t imagine any good reason for it, so I suspect this may be a bug.

Mobile Safari on iOS 3.2 (which is only available on the iPad) does not exhibit this bug. Its 25.6KB component limit and ~281.6KB total cache limit are better than nothing, but they still seem paltry compared to the other devices tested. Uniquely among iOS devices, the iPad appears to limit the size of pages in the page cache to 25.6KB, the same as its component size limit.

Mobile Safari on iOS 4.0 exhibited different limits on the iPhone 3GS and on the iPhone 4, which implies that the limits adapt based on available RAM (the iPhone 3GS has 256MB while the iPhone 4 has 512MB; both devices tested had 32GB of flash memory). On the iPhone 3GS, iOS 4.0 has a 51.199KB component size limit and a ~1.05MB total component cache size.

On the iPhone 4, the component size limit was almost exactly two times the limit on the iPhone 3GS, at 102.399KB. The total component cache size was approximately 1.9MB. Perhaps because iOS 3.2 and iOS 4.0 were developed separately but branched from a common ancestor, the iOS 4.0 page cache size appears to be limited only by available RAM on both devices tested, just like iOS 3.1.3.

None of the iOS devices preserved the contents of the cache across forced browser restarts or device power cycles, although they did preserve the cache when merely switching applications without actually killing the browser.

webOS

My test results on webOS were so inconsistent that I have little confidence in them. I’ve included what little data I managed to gather purely for the sake of completeness. Please take it with a hefty grain of salt.

As near as I was able to determine, webOS might have an individual component size limit of about 1MB, with a matching page size limit in the page cache. I was unable to coax If-None-Match or If-Modified-Since request headers from webOS, which implies that it does not support ETag and Last-Modified.

On some tests, it appeared that webOS’s maximum component size was greater than 1MB, but this was inconsistent. As far as I can tell, webOS appears to have a nasty bug where, after a certain point—possibly when the maximum total cache size is reached—the cache just completely stops working altogether until the phone is power-cycled. In some cases even power cycling didn’t fix the cache breakage, so I eventually gave up trying to establish the exact cause of the problem and the exact limits of the webOS cache.

Recommendations

Based on these results, I offer the following recommendations to anyone developing web applications for the tested devices:

• Use far-future cache expiration headers. This will prevent the browser from having to send a conditional GET request and will improve cacheability in webOS, which doesn’t support ETag or Last-Modified.
• At least until iOS 4.0 arrives on the iPad, try to limit individual component sizes to 25.6KB or less, uncompressed. And urge your iPhone users to upgrade to iOS 4.0 as soon as possible.
• If your website must support iOS 3.1.3 users (which is likely), if it requires components larger than 25.6KB, or if the total size of all your components is larger than 281.6KB, consider using the HTML5 application cache, localStorage, or database storage to store your components. Alex Kessinger’s recent YUI Blog post, An Introduction to Using YUI 3 in Offline Applications, might be of interest for YUI 3 developers considering this approach.
• Do your own testing. Don’t assume that these results apply to any future version of any of the tested browsers or devices. Use these results as a starting point, but verify them yourself before you make major decisions based on assumptions about mobile cache limitations. The mobile browser world changes at a lightning pace, so this research will have a very short shelf life.

I’ve made my test code available on GitHub and I encourage you to use it, fork it, and share what you learn.

Call for Documentation

Browser makers, please consider documenting and publishing your browser’s cache limits. In the desktop world where these limits are typically so high as to be a non-issue, documentation wasn’t needed. In the mobile world, browser cache limits are vital information that web developers must have in order to create performant websites with compelling features.

The limits of new features like localStorage and the application cache are typically well-documented. Please extend this level of documentation to the component cache as well.

In Mobile Browser Cache Limits: Android, iOS, and webOS, I shared the results of my attempts to determine browser cache limits on Android, iOS, and webOS devices. At the end of the article, I wrote:

Use these results as a starting point, but verify them yourself before you make major decisions based on assumptions about mobile cache limitations. The mobile browser world changes at a lightning pace, so this research will have a very short shelf life.

As it turns out, that was good advice: the day after the article was posted, Steve Souders commented that he had run tests using a different methodology that was more representative of a real-world web workflow and had gotten different results.

New Methodology

My original methodology involved navigating directly to a randomly generated page of a certain size, served with a text/html content type. The results using this methodology were reliably reproducible (except on webOS), but as Steve pointed out, users don’t navigate directly to CSS and JavaScript files. My assumption that the limits for direct navigation to an HTML resource were the same as the limits for external CSS and JavaScript was incorrect, so even though the results of my tests were valid, they weren’t widely applicable.

Over the course of many IM sessions, several emails, and a couple of phone calls, Steve and I worked out a new testing methodology. I implemented a version of it on top of my cache testing framework, then Steve implemented a version capable of publishing results to Browserscope.

In the new tests, we load an HTML page that refers to a randomly-generated CSS or JavaScript component of a certain size. Then we navigate to a second HTML page that loads the same component and checks whether or not it was loaded from the cache. To determine whether a component was loaded from the cache, we store a timestamp in a cookie on each request; if the timestamp is updated the second time we load the component, we know the request hit the server, which means the component was not loaded from the cache.

New Results

We found that all the mobile browsers we tested had significantly higher cache limits for external resources loaded by a page than they did for an HTML page itself. This is excellent news for mobile web developers.

The table below illustrates our findings:

Table: Mobile browser external resource cache characteristics

Browser/OS/Device	Single Component Limit	Survives Power Cycle
Android 2.2 (Nexus One)	2MB	Yes
Mobile Safari, iOS 3.1.3 (1st-gen iPhone)	4MB+	No
Mobile Safari, iOS 3.2 (iPad)	4MB+	No
Mobile Safari, iOS 4.0 (iPhone 3GS)	4MB+	No
Mobile Safari, iOS 4.0 (iPhone 4)	4MB+	No
webOS 1.4.1 (Palm Pre Plus)	~0.99MB (1,023KB)	Yes

Note that 4MB was the largest size we tested, and all the iOS devices cached 4MB components. The actual cache limit for those devices may be larger than 4MB. Also, webOS on the Palm Pre Plus gave consistent results in this test, whereas it had some problems in the previous test.

It’s possible that the much lower limits my previous test showed for HTML components on iOS may indicate the use of a RAM cache for those components, while the much higher limits for CSS/JS components in this test may indicate the use of a disk cache, but this is just conjecture. Android, at least, does appear to use a disk cache in both cases, since its cache survives power cycles.

New Recommendations

Based on these new results, coupled with the results from my previous tests, I offer the following updated set of recommendations:

• Use far-future cache expiration headers. This will prevent the browser from having to send a conditional GET request.
• Try to limit HTML pages to 25.6KB or less if you want them to be cached, since the previous tests showed that this limit—imposed by iOS 3.2 on the iPad—was the lowest HTML resource limit of the devices tested.
• Keep CSS and JS components under 1MB. Of course, 1MB is enormous and your components should be much smaller than this, but don’t bother splitting a component into separate requests for the sake of cacheability unless its size approaches 1MB.
• Consider using the HTML5 application cache if it’s important that your components persist in the cache for a long time, or across power cycles.
• Do your own testing. I stressed the importance of this in my previous article and I’ll stress it again here. Use these results as a starting point, but verify them yourself before you make important decisions based on them.

Ever notice the semi-transparent gray background that appears sometimes when you tap on something in Mobile Safari? That’s the default tap highlight color, which Mobile Safari displays when you tap on an element with a JavaScript click handler.

Unfortunately, Mobile Safari has no way to distinguish between normal click subscribers and delegated click subscribers, which is when the click event is attached to a container rather than to each child of that container. As a result, a delegated click will result in the entire container being highlighted rather than just the item that was tapped, and this can be both ugly and confusing for the user.

The good news is that Mobile Safari supports a CSS extension called -webkit-tap-highlight-color, which you can use to customize the color of the tap highlight or hide it completely.

The -webkit-tap-highlight-color property accepts any standard CSS color value, but you’ll probably want to provide an rgba value in order to control the alpha transparency. Disabling the tap highlight is as simple as setting the alpha value to 0, like so:

#container {
  -webkit-tap-highlight-color: rgba(0,0,0,0);
}

Fire up your favorite iOS device and view this simple demo to see what a delegated click looks like both with and without a tap highlight. For more details on -webkit-tap-highlight-color and other useful Mobile Safari CSS extensions, see the Safari CSS Reference.

At YUIConf 2011 last week, we set up a video camera and invited attendees to tell us how they use YUI and why they chose it for their projects. We were thrilled to hear their stories, and we’d love to hear yours as well! After checking out the video, leave a comment and tell us how you use YUI.

In one of the most popular talks at YUIConf 2011, YUI engineer Eric Ferraiuolo discusses the MVC components of the YUI App Framework and some of the new features coming in YUI 3.5.0 and beyond, including Handlebars-based templating and a new Y.App component that makes the creation of MVC apps easier than ever on desktop, mobile, and the server.

Links

• View in HD on YouTube
• Slides
• Photos Near Me demo app
• Photos Near Me source code on GitHub
• Full YUIConf 2011 Playlist on YouTube

Mozilla Labs engineer and TC39 representative Dave Herman joined us at YUIConf 2011 to give this keynote talk on the future of JavaScript, covering many of the new features currently under consideration for ES6, the next edition of the ECMAScript standard.

Links

• View in HD on YouTube
• Full YUIConf 2011 Playlist on YouTube

Pat Cavit (@tivac), a frontend engineer at ArenaNet and an active YUI contributor and community member, joined us at YUIConf 2011 to give this talk on automating build-time website optimizations such as file concatenation, minification, renaming, and more using the Ant build tool.

Links

• View in HD on YouTube
• Full YUIConf 2011 Playlist on YouTube

In this talk from YUIConf 2011, YUI engineer Allen Rabinovich (@allenr) shares the process he used to architect and build the new Calendar widget in YUI 3, and explains how you can use a similar process to build your own widgets. He also shows off the new component and reveals a clever performance trick used to speed up the rendering of multiple calendars.

Links

• View in HD on YouTube
• Full YUIConf 2011 Playlist on YouTube

Since the initial release of the App Framework in YUI 3.4.0, we’ve been astonished by how quickly it’s been adopted. In projects large and small, both long-time YUI users and those who are completely new to the library have been enthusiastically using the App Framework’s MVC components and providing great feedback and bug reports. Thank you!

In YUI 3.5.0, the App Framework will receive lots of bug fixes and some major enhancements. Eric Ferraiuolo covered many of the upcoming changes in his fantastic YUIConf talk, but we want to highlight them here as well so you’ll know what’s coming and what you should prepare for if you plan to upgrade App Framework-based code from 3.4.x to 3.5.0. These changes are already in YUI 3.5.0 PR1, which was released today, so now’s a great time to start testing them.

Y.Controller is now Y.Router

“Controller” was a silly and confusing name for a component that really concerns itself more with URL-based routing, especially given the more traditionally controller-like role that Y.View plays. We’ve decided to bite the bullet and rename the Y.Controller class to Y.Router in 3.5.0. Y.Controller will become an alias to preserve backwards compatibility, but this alias will eventually be removed, so you should update your code to refer to the new name.

New route handler signature

The method signature for route handler functions in Y.Router has changed slightly to make it more similar to Express and to make Router’s API more natural when used on the server (a feature we’re currently working on for 3.5.0).

Previously, a route handler function received two arguments: req (a request object) and next (a function). In 3.5.0, route handlers will receive three arguments: req, res (a response object), and then next.

For the sake of backwards compatibility, the new res argument is also a function that behaves exactly like next, so old-style route handlers that expect next as the second arg will continue to work fine in 3.5.0. However, this compatibility shim will eventually be removed, so don’t wait too long to update your code.

Some properties are now attributes

We experimented with a not-entirely-YUI-like style of using properties for configurable options in the App Framework components in 3.4.0, but this turned out to be a little confusing and more than a little limiting, since properties don’t benefit from change events, setters, and validators like attributes do. So in 3.5.0, we’re converting many of these properties to attributes.

Unfortunately, this change is not backwards compatible, so existing code that uses Y.Controller (now Y.Router) or Y.View may need to be updated. Specifically, Y.Router’s html5, root, and routes properties are now attributes, and Y.View’s container, model, and modelList properties are now attributes as well.

In addition to this, Y.View’s container attribute now treats string values as CSS selectors used to find nodes on the page. In 3.4.x, it assumed a string value represented raw HTML that should be converted into a node. To get the old behavior, just change your existing HTML string values from '<div>foo</div>' to Y.Node.create('<div>foo</div>').

Documentation for 3.5.0 PR1

Work-in-progress documentation for these changes and other changes in YUI 3.5.0 PR1 can be found on our staging website. Here are some links to relevant staging docs that include information on App Framework deprecations in 3.5.0 and details about how to upgrade your code:

• Router
• View

Note that the content at stage.yuilibrary.com reflects ongoing work in progress and may be incomplete or even occasionally broken as we test new stuff. You’ll always find the docs for the latest stable release at our production site, yuilibrary.com.

What else is new?

In this blog post I’ve summarized the important deprecations coming to the App Framework in 3.5.0, but there are also lots of feature enhancements and under-the-hood bug fixes. For a complete list of App Framework changes in 3.5.0 PR1, consult the HISTORY file.

Also, look for a blog post from Eric soon about Y.App, an awesome new high-level component of the App Framework that wraps up URL-based routing and view management into a single easy-to-use API that’ll get you from zero to a working application in no time.

We hope you love the preview release, and we’d love to hear from you! You can send us feedback in the forums, in a bug report, on Twitter, on the #yui IRC channel on Freenode, or just chime in here with a comment.