The Use of Logging Data in the Design of a New Text Editor

Michael Good

Digital Equipment Corporation
Nashua, New Hampshire, USA

Originally published in Proceedings of CHI ’85 Human Factors in Computing Systems (San Francisco, April 14-18, 1985), ACM, New York, pp. 93-97. Included here with permission. Copyright © 1985 by ACM, Inc.

Abstract

Many different human factors techniques are available to the designer of a new computer system. This case study examines how one technique, the use of logging data, was used throughout the design of a new text editor which is measurably easy to learn and easy to use. Logging data was used in four areas: keyboard design, the initial design of the editor’s command set, refinements made later in the design cycle, and the construction of a system performance benchmark.

1. Introduction

One of the leading requests of VAX/VMS operating system users has been for a faster, more extensible editor.¹ These requests led to the development of the VAX Text Processing Utility (VAXTPU). VAXTPU provides a programming language which can be used to build text editors and other text processing application programs.

The VAXTPU project provided the opportunity to design a new text editor interface, implemented entirely in the VAXTPU programming language. This interface, called Eve, was to be measurably easy to learn and easy to use for both novice and experienced computer users. It would also package some of the more advanced features of VAXTPU, such as multiple windows, adjustable margins, and learn sequences, so that they could be used without any knowledge of the VAXTPU programming language. Eve would also take advantage of the keyboard layout in Digital’s VT200 series terminals.

A variety of human factors techniques were used throughout the editor design process, including experimentation, rapid prototyping, iterative design, and logging. This paper describes how logging data collected from a variety of text editors was used throughout the design of Eve. Benchmark tests show that the resulting editor is measurably easy to learn and easy to use.

2. Logging Data

Existing logging data from five different editors was used during the design of Eve. These editors were Computer Corporation of America’s Emacs editor (CCA Emacs), Digital’s EDT and WPS editors, Yale’s Z editor, and the experimental EPT editor.

All of the editor data included totals for either keystrokes or command frequencies, as described below. The WPS and EPT data also included the frequencies of transitions between keystrokes.

The interface to CCA Emacs closely resembles the interface to the original Emacs (Stallman, 1981). When the logging data was collected, CCA Emacs contained 206 commands, but did not include an editor extension language. The logging sample included almost 700,000 command invocations, and was collected from one and a half weeks of usage by programmers, managers, and secretaries (Zimmerman, 1982).

Z was intended to provide support both for programming and for standard document editing (Wood, 1981). The editor contained 74 commands, and the logging sample contained over 22,000,000 keystrokes collected over three weeks. Users included computer science department students, faculty, and staff, as well as users from other academic departments (Wood, 1982).

EDT is the standard text editor currently provided with the VMS operating system (Digital, 1980). The logging sample recorded frequencies for the 49 EDT keypad commands. The sample was collected from nine members of a technical work group over a five week period, and included over 500,000 keystrokes.

WPS is a commercial word processing system with 64 keyboard commands, many of them involving sophisticated text formatting functions. The logging sample contained over 400,000 keystrokes, collected from eight secretaries over an eleven-day period (Whiteside et al., 1982).

EPT (Editor Prototyping Tool) is a small editor with 29 commands, designed specifically for research purposes. The logging sample of over 500,000 keystrokes was collected from six members of a human factors research group over a two-month period, measuring 212 person-hours of use (Whiteside et al., 1982).

3. Keyboard Design

Eve was designed to take advantage of the keyboard layout of Digital’s VT200 series terminals. Figure 1 shows the portion of the VT200 series keyboard used by Eve for basic editing commands. Seventeen keys are used outside of the main typing array, arranged in small groups.

Figure 1: Eve Keypad for VT200 Series Terminals

One feature of this keyboard is its arrangement of cursor keys into an inverted-T layout. This arrangement was based in part on logging data from the EPT editor. Table 1 shows the frequencies of transitions between arrow keys in the EPT editor.

The transition from the down-arrow key to the left-arrow key occurred more than twice as often as any other transition between arrow keys. The inverted T arrangement puts these keys next to each other. Three fingers of the user’s hand can rest on the three most frequently used arrow keys. From this position, it is also easier to reach up to the up-arrow key than it would be to reach down to a down-arrow key.

The arrow keys and the keys in the two groups immediately above it have their legends engraved on the keycaps. The Find command searches for text within the current file. Select, Remove, and Insert Here are used to move or copy regions of text from one place to another. The Previous Screen and Next Screen keys move the cursor through the file a screenfull at a time. The Help key provides online help for the keypad interface. The Do key is pressed to enter commands through the command-line interface.

Table 1: Arrow Key Transitions in the EPT Editor

Although these commands are not necessarily high-frequency commands in the logging data, they are basic commands which can be used to perform most core editing tasks. All but three of the core editing tasks in Roberts’ editing function list (Roberts, 1979) can be performed using the Eve keypad commands. The exceptions are the hardcopy and intermediate input/output tasks.

The five keys to the left of the Help key are not engraved, but can be labeled by means of a label strip inserted immediately above the keycaps. The Exit key leaves the editor, saving any modified files. The Forward/Reverse key toggles the current direction of the editor, affecting the operation of the Find and Move By Line keys. The Move By Line key moves to the start or the end of the current line, depending on the current direction. The Erase Word key erases the current word and moves the cursor to the start of the next word. The Insert/Overstrike key toggles the current mode of the editor.

With the exception of the Forward/Reverse key, each of these five labeled keys is used in a similar way by the VMS operating system when a user is editing the current command line. Compatibility with existing keyboard engravings and existing software usage were powerful constraints in the design of the keyboard interface. Since these earlier key definitions were chosen for text-editing applications, it appeared that in this case compatibility would enhance ease of learning and ease of use. Eight novice users were run in informal tests early in the design cycle as part of iterative testing of the keypad interface. In general, the keypad interface appeared easy to learn.

4. Command Set Design

Eve contains many commands that are accessible through a command-line interface. The user presses the Do key, types the command, and presses the Return key to issue the command. Commands and their arguments may be abbreviated, if the abbreviations are not unique, the user is prompted with the possible choices. All commands are saved in a buffer, where they can be recalled, edited, and reissued. Pressing the Do key twice in a row repeats the previously issued command.

Usage frequencies for all five editors were studied to determine which commands should be included in the Eve command-line interface. We wanted to include commands that were used moderately frequently (the most frequently used commands should also be on the keyboard), as well as powerful commands that would be difficult or impossible to build out of a series of more elementary commands, even if these powerful commands are not so frequently used. The goal was to develop a powerful but modestly sized command set, designed for both ease of learning and ease of use. We wanted to avoid the problems associated with overly large and complex computer systems, such as more error-prone implementations, more forbidding documentation, and harder to remember language forms (Ledgard et al., 1981).

The editor usage data identified high-frequency commands which were included in the command set. For example, word motion commands were very frequently used in the CCA Emacs, WPS, and EDT editors. In each of these editors the commands were among the top 20% as ranked by frequency of use, and accounted for more than 1% of all commands, including typing characters. Word motion was used less frequently in the EPT and Z editors, but still ranked within the top 50% of commands based on frequency of use. Commands such as Erase Character and Erase Line were also frequently used across different editors.

The usage data also identified less powerful, low frequency commands which were not included in the command set. For example, cursor motion by sentence was included in the CCA Emacs, WPS, and EPT editors. In none of these editors did the commands account for more than 0.03% of all keystrokes. WPS users used the sentence command most frequently (101 out of 406,102 commands), but the command was still in the 50% least frequently used commands despite the fact that all cursor motion in WPS is by entities, rather than arrow keys.

The data for CCA Emacs uncovered a general trend which guided the design of several commands in Eve. Commands which worked on a standard unit of text were used much more frequently than commands which required the user to manually select a range of text before using the command. For example, the Fill Paragraph command was used 96 times compared to 1 use of the Fill Region command. The Lowercase Word command was used 116 times compared to 1 use of the Lowercase Region command. The Eve command set includes Fill Paragraph and Lowercase Word commands, not the Fill Region and Lowercase Region commands.

During an iterative design phase, users may often ask for new commands and features, especially if the command was present in their previous system. Usage data helps the designer to determine the value of the request, flagging unnecessary “one-plus” requests which would detract from the simplicity and ease of use of the system. During Eve’s iterative design, users asked for Start of Line and End of Line commands, which were later added to the command set, and a Move by Sentence command, which was not added.

5. System Development

Logging was added to Eve during the development and iterative testing process. Typing keys, arrow keys, and the delete, tab, and return keys were not recorded, because we already knew that these keys were used extremely frequently. Frequencies for the remaining 64 commands were logged over a period of sixteen weeks for twelve different users. These users included researchers, programmers, and managers. We collected data on 1020 person-hours of Eve usage, including over 394,000 command invocations.

Usage statistics for other editors often show that there are many commands accessible through the keyboard which are not frequently used. This is not the case with Eve. With the exception of the Help command, all of the Eve keypad commands ranked in the top third of frequently used commands. All of the keypad commands, except for the Forward/Reverse key, were used by all 12 users (Forward/Reverse was used by 9 of the 12 users). In contrast, CCA Emacs and WPS include keypad commands which were not used by any of the users in the logging sample.

The Fill Paragraph command was one of the most frequently used of the non-keypad commands, confirming the earlier decision to include the Fill Paragraph command rather than the Fill Region command

Logging data combined with user comments led to some small changes in the Eve command set. Two infrequently used commands, Next Line and Open Line, were removed from the editor. They were replaced by Start of Line and End of Line commands. These additions also improved compatibility with the command-line editor included with the VMS operating system.

6. Benchmark Development

VAXTPU was designed to give fast response time, especially in frequently used commands such as cursor motion and character deletion. Eve was run on an existing editing system performance benchmark to measure whether this goal was met. Eve response time was generally two to three times faster than EDT response time.

The system performance benchmark was based on the EPT and WPS logging data. The EDT usage data was collected in part to help verify the existing data for constructing this benchmark.

7. Results of Usability Benchmarks

The specification for Eve stated that Eve’s ease of use would be measured by running Roberts’ core learning and core use benchmark tasks (Roberts, 1979). Eve was to rank with the best performance recorded on these benchmark tasks, based on data published on nine different text editors (Roberts and Moran, 1983).

Eve’s score on the core learning benchmark was 4.9 ± .16 minutes/task; on the average, it took people 1 hour and 42 minutes to complete the standard learning sequence. Eve’s score on the core use benchmark was 26 ± .07 seconds/task; on the average, it took people 22 minutes of error-free time to complete the timed tasks. Eve ranked second out of ten editors on the ease of learning benchmark and tied for third out of ten editors on the ease of use benchmark, though most of the differences between editors on these benchmarks are not statistically significant.

Eve performed 54% of the tasks on Roberts’ functionality benchmark, which is the mean functionality for the ten editors. Gypsy, the editor which scored highest on the ease of learning and ease of use benchmarks, was also the lowest functionality editor, scoring only 37% on the functionality benchmark. (For completeness, Eve’s error score was 15 ± .19 % total time. None of the differences between editors on this score are statistically significant.)

8. Discussion

Logging data was used throughout the design of the Eve editor. Before Eve was designed, logging data contributed to the design of the keyboard for the editor’s target computer terminal and to the development of a benchmark for measuring editing system performance. Logging data was then used to guide the composition of the Eve command set. Logging data served as a check for user comments during the iterative design stage. Logging data for Eve itself was used in changing the Eve command set later in the design cycle.

Much logging data is available for text editors, as well as for traditional computer operating systems (Hanson et al., 1984). However, we lack logging data for many other computer application areas, such as graphics and spreadsheet systems. Such data would be of great value to anyone designing or refining such an application system.

Most computer systems designers claim that their system is easy to use. These claims are often meaningless because they give no indication of how ease of use is to be measured. The entire Eve design process was driven by measurable goals for ease of learning and ease of use, specified at the beginning of the design process. It is difficult to create concrete usability goals, especially in areas other than text editors where no benchmark usability tasks are available. Logging data can be used to develop new benchmark tasks which reflect actual usage of the systems being measured. This data can be used to develop both system performance and user performance benchmarks.

Acknowledgments

Steve Long, Sharon Burlingame, and Terrell Mitchell designed and implemented the VAX Text Processing Utility. Michael Greenstein collected the EDT editor usage data. Paula Levy ran the ease of learning benchmark tests. Mary Ellen Zurko ran the system performance benchmark tasks.

Footnotes

¹VAX, VMS, and VT are trademarks of Digital Equipment Corporation.

References

Digital Equipment Corporation. EDT Editor Manual. Order No. AA-J726A-TC, Maynard, Mass., October 1980.

Hanson, Stephen José, Kraut, Robert E. and Farber, James M. Interface design and multivariate analysis of UNIX command use. ACM Transactions on Office Information Systems 2, 1 (January 1984), 42-57.

Ledgard, Henry, Singer, Andrew and Whiteside, John. Directions in Human Factors for Interactive Systems. Springer-Verlag, New York, 1981.

Roberts, Teresa L. Evaluation of computer text editors. Xerox PARC Report SSL-79-9, November 1979. Stanford Ph.D. dissertation.

Roberts, Teresa L. and Moran, Thomas P. The evaluation of text editors: methodology and empirical results. Communications of the ACM 26, 4 (April 1983), 265-283.

Stallman, Richard M. EMACS, the extensible, customizable self-documenting display editor. SIGPLAN Notices 16, 6 (June 1981), 147-156.

Whiteside, John, Archer, Norman, Wixon, Dennis and Good, Michael. How do people really use text editors? SIGOA Newsletter 3, 1 (June 1982), pp. 29-40.

Wood, Steven R. Z – the 95% program editor. SIGPLAN Notices 16, 6 (June 1981), pp. 1-7.

Wood, Steven R. Data distributed over ARPANET, October 24, 1982.

Zimmerman, Steve. Data distributed over ARPANET, May 27, 1982.

Copyright © 1985 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept, ACM Inc., fax +1 (212) 869-0481, or permissions@acm.org.

This is a digitized copy derived from an ACM copyrighted work. ACM did not prepare this copy and does not guarantee that is it an accurate copy of the author’s original work.