Working Slower with More Powerful Computers

by Lorenz M. Hilty, Andreas Köhler, Fabian van Schéele and Rainer Zah

An empirical study conducted by the Swiss Federal Laboratories for Materials Testing and Research showed that using a more powerful PC can significantly slow down office workers in performing every-day office tasks.

The average service life of a personal computer used for business is 2-3 years. After this short service life, it is usually replaced by a more powerful PC running new versions of system and application software. Therefore, it is worth considering whether the high replacement costs of PCs bring a comparable benefit. Whereas the benefits from having additional functionality (eg, in multimedia applications) are self-evident, it is not a priori clear whether pre-existing functionality can be accessed and used as efficiently as before when the user goes to a new machine. The study reported here addresses only the latter issue.

Our laboratory experiment was based on PC configurations typical of the years 1997, 2000 and 2003. As Table 1 shows, the hardware used differed greatly in clock speed, working memory and hard disk capacity. Since the test only involved file handling and text editing, it was sufficient to install the operating system and a word processor. The configurations A, B and C are reconstructions of PC configurations used widely in business in the respective year. All software components were installed taking the default parameter settings. The appearance of the desktops was kept similar on all computers to optimally satisfy blind test conditions. A switch box made it possible to connect the same mouse, keyboard, and monitor to each of the three computers.

We defined two tasks for testing user performance on the three systems: a file handling and a text editing task. The file handling task included locating, copying, moving and deleting files and groups of files varying in size from 24 kByte to 42 MByte. The text editing task focused on locating, inserting and deleting text, changing fonts and formats in a large document of 150 pages as well as copying, pasting and positioning photographs between two documents.

The laboratory experiment was performed on 42 subjects recruited from a service organization with 180 employees. Subjects had to execute each of the tasks twice on each computer system in randomized order. The whole test took 45-90 minutes per subject. We measured the time the subjects needed to complete each task, logged user interactions and processor workload on the computer and videotaped the screen signal.

The test revealed a statistically significant decrease in user performance on Windows XP as compared to Windows 2000 for the file handling task, despite the fact that Windows XP was run on considerably faster hardware. This result was totally unexpected because Windows 2000 had never been used in the organization in which the subjects were employed. Windows NT systems had been in use for five years. Approximately five months before the test, a migration from Windows NT to Windows XP had been initiated. By the time of the test, virtually every employee was working with Windows XP. Only four subjects reported that they usually worked with Windows 2000, and another four were still working with Windows NT.

In order to find explanations for our result, we examined the effort required from the user and the machine to complete the tasks in detail. Since most user actions during task execution were mouse positionings (usually followed by a single mouse click, less often by no click or a double click), we used the number of mouse positionings as an indicator of the user’s effort. The computer’s work was measured in CPU time, which we calculated by integrating processor utilization over time.

As Figure 1 (top) shows, the file handling task required much less CPU time on system B (year 2000) than on system A (1997), but again much more on system C (2003), and the same is true for the number of mouse positionings the users performed (significant difference, P=0.016). Whereas they managed with roughly 50 mouse positionings on Windows 2000 on the average, they needed more than 80 positionings for the same task on Windows XP.

The text editing task (Figure 1, bottom) shows a decrease in mouse positionings from A to B to C, but an increase in CPU time from B to C. Given the fact that the C hardware executes more than twice as much many instructions per second than the B hardware, one would instead expect a significant decrease.

The main conclusion from this experiment is that changing over to faster a computer running newer software does not necessarily lead to better performance, at least for the types of tasks we used in our experiment. On the contrary, it is even possible that both – the machine and the user – need to work significantly longer in order to replicate a given task with the new system.

Acknowledgements
The work reported here was conducted in cooperation with the Swedish Royal Technical University (KTH), Stockholm, and co-funded funded by the Board of the Swiss Federal Institutes of Technology (ETH board) as a part of the ‘Sustainability in the Information Society’ research program at Empa.

Links:
Empa: http://www.empa.ch
Technology and Society Lab at Empa: http://www.empa.ch/TSL
Sustainability in the Information Society Program: http://www.empa.ch/SIS

Please contact:
Lorenz M. Hilty, Technology and Society Lab, Empa, Swiss Federal Laboratories for Materials Testing and Research, Switzerland
Tel: +41 71 2747 500
E-mail: lorenz.hiltyempa.ch