Mobile use is one of the biggest challenges now facing many websites. It’s also important for some intranets, particularly in companies with many traveling employees.
Usability Research
To discover what makes websites easy or difficult to use on mobile devices, we combined three usability methods:
- Diary study. 14 participants from 6 countries (Australia, The Netherlands, Romania, Singapore, UK, and US) logged everything they did with their mobile device—aside from making phone calls—for about a week. For each activity, they sent us a Twitter message on the spot; at the end of each day, we sent them a questionnaire to collect more in-depth information.
- User testing. 48 people participated in usability studies using their own phones, and we recorded sessions with a document camera. Half of the participants were men and half were women. The age distribution was fairly even across the 20-49 year range; a smaller number of users were age 50 and above. Of the 48 participants, 33 were in the US (two locations) and 15 were in London.
- Cross-platform review. We conducted a design review of 20 sites, using 6 phones: one feature phone, three different smartphones, and two different touch phones.
In the usability tests, we asked participants to perform typical tasks with their phones. In testing mobile.winespectator.com, for example, we showed people a bottle of wine and asked them to find information about it on the site. For m.lufthansa.com, one of the tasks was:
“Your friend is scheduled to arrive in London today around 12pm from Munich, Germany. Find out if her flight is on time.”
In all, we tested 36 websites and asked users to attempt particular tasks on each. These site-specific tasks let us systematically observe several users with different phones trying to do the same thing. We also tested 34 Web-wide tasks where participants could use any site they wanted. One such task was:
“You and your vegetarian friend want to find a good Indian restaurant nearby. Use the Web to locate one that you may want to go to and that serves vegetarian food.”
These tasks gave us usability insights into hundreds of additional sites as well as an understanding of how people decide which sites to visit on their mobiles.
The test tasks were inspired by the user activities recorded in the diary studies. The diaries also let us follow user behavior over a longer time period in more naturalistic settings than is feasible for lab studies.
The Mobile User Experience Is Miserable
The phrase “mobile usability“ is pretty much an oxymoron. It’s neither easy nor pleasant to use the Web on mobile devices. Observing user suffering during our sessions reminded us of the very first usability studies we did with traditional websites in 1994. It was that bad.
view Mobile Web 2009 = Desktop Web 1998
In our mobile studies, the average success rate was 59%, which is admittedly higher than success rates in the 1990s, but substantially lower than the roughly 80% success rate when testing websites on a regular PC today.
view Success Rate: The Simplest Usability Metric
view IA Task Failures Remain Costly
Before the study, we had expected to get better results in London because the UK has a stronger tradition for mobile services than the US. However, the actual sessions didn’t bear this out: the British sites were just as bad as the American sites, and users struggled about as much to get things done.
Main Mobile Problems
Mobile users face four main usability hurdles:
- Small screens. For something to be mobile, it must be easy to carry and thus relatively small. Small screens mean fewer visible options at any given time, requiring users to rely on their short-term memory to build an understanding of an online information space. This makes almost all interactions harder. It’s also difficult to find room for multiple windows or other interface solutions that support advanced behaviors, such as comparative product research.
- Awkward input, especially for typing. It’s hard to operate GUI widgets without a mouse: menus, buttons, hypertext links, and scrolling all take longer time and are more error-prone, whether they’re touch-activated or manipulated with a teensy trackball. Text entry is particularly slow and littered with typos, even on devices with dedicated mini-keyboards.
- Download delays. Getting the next screen takes forever—often longer than it would on dial-up, even with a supposedly faster 3G service.
- Mis-designed sites. Because websites are typically optimized for desktop usability, they don’t follow the guidelines necessary for usable mobile access.
The first two problems seem fundamental. Yes, such problems impact newer phones less than older phones (as I discuss below), but still: mobile devices will never offer screens as big or input devices as good as a full-fledged PC.
Connectivity problems will hopefully diminish in the future, but it will take many years until mobile connections are as fast as even a modest cable modem—let alone as fast as the broadband connections promised by wireline improvements.
Mobile will never be the same as desktop. So, we’re left with the hope that websites will redesign for better mobile usability.
Mobile Sites Beat Full Sites
When our test participants used sites that were designed specifically for mobile devices, their success rate averaged 64%, which is substantially higher than the 53% recorded for using “full” sites—that is, the same sites that desktop users see.
Improving user performance by 1/5 is reason enough to create mobile-optimized sites. Such sites were also more pleasant to use and thus received higher subjective satisfaction ratings. This fact offers an additional rationale: When users are successful and satisfied, they’re likely to come back. So, if mobile use is important to your Internet strategy, it’s smart to build a dedicated mobile site.
Still, users often had trouble getting to mobile sites, even when companies offered them. The best approach is to auto-sense users’ devices and auto-forward mobile users to the mobile site (even if they’re using a high-end phone). You should also offer clear links from the desktop site to the mobile site, as well as a link back to the full site. As for link labels, we recommend “Mobile Site” and “Full Site,” respectively.
Linking to the full site supports users who want advanced features that the mobile site doesn’t support. Given this fallback solution, you should scale back the mobile site’s functionality and focus on features that people are actually likely to use in a mobile scenario. Users repeatedly told us that they don’t want to do tasks on their mobiles that involve heavy interaction or in-depth information perusal.
Better Phones Perform Better
There are 3 distinct classes of mobile user experience, and they’re mainly defined by screen size:
- Feature phones (regular cellphones) with a tiny screen and a numeric keypad. These devices account for the vast majority of the market (at least 85% in some statistics).
- Smartphones, in a range of form factors, typically with a mid-sized screen and a full A-Z keypad.
- Touch-screen phones (such as the iPhone) with a nearly device-sized screen and a true GUI driven by direct manipulation and touch gestures.
Unsurprisingly, the bigger the screen, the better the user experience when accessing websites. Average success rates were:
| Feature phones | 38% |
| Smartphones | 55% |
| Touch phones | 75% |
With these numbers, the consumer advice is easy: buy a touch phone if using websites is important to you.
The advice for Internet managers is harder. Considering the horrible usability of feature phones, should you even support them? Alternatively, should you focus on smartphone and touch phone users who are more likely to use your site extensively? There’s no single answer.
For services highly suited for mobile use—such as news or social networking—you should probably create a dedicated feature-phone site, as well as a site optimized for higher-end phones. Most other websites might be better off concentrating their investment on a single mobile site optimized for smartphones and touch phones. Finally, if you focus on complex transactions or in-depth content, you’ll probably have too few mobile users to justify a separate site.
No Progress Since 2000?
In our London sessions, we repeated two tasks from our study of WAP usability in 2000. We expected to find reasonable improvements in task performance, but the results contradicted those expectations (which is obviously why we bother doing research). The mean task times from the two studies were:
view WAP Field Study Findings
| Tasks | WAP phones (2000) | Current Phones |
| Find the local weather for tonight | 164 sec. | 247 sec. |
| Find what’s on BBC TV 1 tonight at 8 pm | 159 sec. | 199 sec. |
Amazingly, users spent 38% more time on these two tasks now than they did in 2000. Are modern mobile devices really worse than the horrible WAP phones of times past? Has site usability declined that much? The answer is no on both counts; phones and sites are definitely better now.
What has changed is the usage environment. In 2000, users were restricted to the “walled garden” supplied by their mobile carrier. WAP phones came with a built-in “deck” that supplied direct access to a few selected services. While this approach limited users’ freedom and restricted them to only the simplest of tasks, they could get to the information with just a few key-presses.
Today’s mobile users are highly search-dominant. When we don’t specify which site they should use (and often even when we do), they turn first to their favorite search engine. Again, this means plenty of typing, which is slow, awkward, and error-prone on mobile devices.
Today, mobile users can do anything. The fact that doing most things takes so long further emphasizes the need for scaled-back mobile site designs.
In our current study, one user did really well—an iPhone user who had a weather application installed on the phone and used it to get the weather forecast in only 18 seconds (1/3 of the fastest speed from 2000). If any additional evidence were needed for mobile-dedicated design’s benefits, this example should surely suffice.
Mobile Usability Is Hard
All of our new research findings support a single conclusion: designing for mobile is hard. Technical accessibility is very far from providing an acceptable user experience. It’s not enough that your site will display on a phone. Even touch phones that offer “full-featured” browsers don’t offer PC-level usability in terms of users’ ability to actually get things done on a website.
When designing for mobile, there’s a tension between (a) making content and navigation salient so that people do not work too hard to get there, and (b) designing for a small screen and for slow downloading speeds. That’s why almost every design decision must be made in the context of the site being designed, and what works for a site may not work for another.
Unless websites are redesigned for the special circumstances of mobile use, the mobile Web will remain a mirage. Users won’t realize the benefits promised by mobile vendors, and site owners won’t reap the profits that would follow from gathering hordes of loyal mobile customers.
Learn More
132-page report on Mobile Usability with 85 design guidelines is available for download.
In-depth course on mobile usability, including many video clips of test participants using a wide range of sites on many different phones, at the Usability Week 2009 conference in Las Vegas and Berlin.
view Designing for Mobile Devices