Wednesday, November 8, 2006

4-Second Rule?

 It looks like Juniper Research has finally done away with the 8-second rule in favor of a 4-second rule. I want to point something out right up front... This new "rule" is based on a survey that asks the question...
Question: Typically, how long are you willing to wait for a single Web page to load before leaving the Web site? (Select one.)
A. More than 6 seconds.
B. 5-6 seconds.
C. 3-4 seconds.
D. 1-2 seconds.
E. Less than 1 second.
Sorry Juniper - I promise that if we sat down with your respondents and asked them to identify how many seconds various pages took to load that MOST of them would not get it right and that MOST of the wrong ones *think* a page takes longer to load than it actually does. Reviewing the report for yourself here:
Scott Barber
Chief Technologist, PerfTestPlus, Inc.

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."


Charlie Audritsh said...

Damn straight. People really have no concept of time. Personally, I've noticed I can't even count "one one thousand, two one thousand, three one thousand..." with any accuracy... ;-) I'm usually a few seconds off one way or the other by the time I get to 20 or 30. And of course no one even counts, not to mention looking at a watch, while their pages load...

While I'm rambling in this direction, when something seems to be taking a long time, I mean not something I'm testing but something I need to do and I'm wondering if my machine has "locked up", I literally make myself look at my watch and wait for 30 seconds. Whatever it is usually gets done in that time. So 30 seconds is both longer and shorter than you'd think. But faster than rebooting and trying again...

Human beings really have no idea how long things take at all. Not with any accuracy...

Roland Stens said...

In this article about Time Perception by Edward Willett, the author talks about the following test:

Dr. Anthony Chaston and Dr. Alan Kingstone of the University of Alberta's Department of Psychology gave subjects tests that required them to find specific items in various images. Before they began, they were told that once the test was over, they would be asked to estimate how long it took. In the simplest of the seven different levels of test, the items sought were a different colour than everything else, or hardly hidden at all. In the more difficult levels, the items were placed among many similar-looking items--or weren't even present.

It showed something that most people are familiar with:

The harder the search task, the smaller the estimate of how long the test had taken. In other words, the more engaged the mind on a task, the faster time seems to go by.

I have lost track of time while being immersed in some heavy thinking. But this is an interesting observation. Because if an application keeps on engaging a user's brain even during a refresh of a screen or retrieval of information, the perceived time spent waiting will be lower than the actual time spent. I experience that myself with AJAX -based Web applications where a lot of the retrieval is done in the background and the applications keep the user engaged. These application just feel a lot quicker and they probable are not. Another interesting quote:

... because people are generally better at predicting how long a task will take than estimating how long it took.

I think this is directly relevant to the whole response times issue. The moment that people complain about the response time is when their estimation/expectation is not met. Fascinating mix of psychology and technology.

Roland Stens

Cem Kaner said...

That context variables influence time perception is not surprising. They influence visual perception too. (Maybe that means we're all blind, or that it is pointless trying to estimate anything about visual experience?) While we're at it we can use context variables to prove that people don't know how loud things are, what they smell, taste or feel like, how heavy or cold they are.

That one can know when to start swinging a bat (in order to connect with the ball at the right time) shows a remarkable temporal ability.

Whether the in the survey gets at that temporal system is a different question.

Put a different way. Decades ago, people studied performance of computer systems and found that delays of a second disrupted the task flow and increased error rates.

I learned in the 80's that a delay longer than 1/4 second degrades human performance, increasing error rate and the probability of task switching. This is more general than software-usability. Delays in a wide array of human tasks interfere with the task and annoy the human doing the task. For many tasks I studied in the 70's a delay of over 1/10th second was disruptive.

The conclusion I suggest is that people say a performance delay of 4 seconds is "OK" today because they guess that 4 seconds is normal (common, typical from good websites). As they get a bit more used to shorter delays, the tolerable result will be shorter, until we reach 1/4 second. Early work on telecom system design was often optimized to hit this same target. For example, air flight reservation system codes (those cryptic 3-letter codes) were designed with the goal of reducing the high rate of errors caused by slow transmission.

Until we reach a level that no longer triggers a sense of delay, "acceptable" delay estimates are fairly arbitrary.

I did a very quick search, here are a few of the vast set of relevant references:

Miller, R.B. 1968 'Response Time in Man-Computer Conversational Transactions', Fall Joint Comp. Conf. U.S.A., pp 267-277.

The effect of System Response Time on interactive computer aided problem solving (1978) "The effects of System Response Time, SRT, on interactive graphical problem solving were investigated for fixed SRT's of 0.16, 0.72 and 1.49 seconds. The object was to demonstrate the importance or otherwise of even small SRT values for interactive graphical problem solving of a type which often occurs in Computer Aided Design. A SRT of 1.49 sec was found to degrade performance by about 50%, measured by problem solution time, compared with that found for 0.16 and 0.72 seconds."

Ben Shneiderman, Response time and display rate in human performance with computers, ACM Computing Surveys (CSUR), v.16 n.3, p.265-285, Sept. 1984 "The pace of human-computer interaction is an important issue to computer scientists and computer users alike. Experimental results have begun to shed some light on this complex, controversial, and vital subject. This paper reviews the theory and reports on experimental results concerning display rates, response time expectations and attitudes, user productivity, and variability. The decomposition of concerns and tasks helps to clarify the issues, but substantial effort remains before a predictive model can emerge.
In general, the results indicate that frequent users prefer response times of less than a second for most tasks, and that productivity does increase as response time decreases.
However, error rates increase with too short or too long a response time. Users pick up the pace of the system, but the profile of commands may change with the speed of the system"

<a href="”>The Cognitive Effects of Delayed Visual Feedback: Working Memory Disruption While Driving in Virtual Environments</a>

Unknown said...

This is the kind of stuff that I try to explain to people when they start talking about in "seconds", but it's a lot more powerful with references!

Somehow, I feel like we just keep fighting the same old fights with new weapons. Are we ever actually going to *win* this fight?!? ::sigh::

Sorry, been reading some fiction... I've got battle scenes on the brain. ;)