INTRODUCTION

This paper proposes the use of a disciplining methodology based on multilinear events sequencing to document and display human decision events associated with accidents. That methodology, Sequentially Timed Events Plotting (STEP), has been used extensively in the investigation and analysis of aviation and other of accidents. Its application to the understanding and improvement of decision events in aviation is discussed here.

Reported accident data, especially those reporting human factors information, tend to be abstract, to reflect judgments and disciplinary jargon, and to lack specificity. These attributes of accident data have impeded significant advancements in the understanding, prediction and control of safety-related problems in aeronautical and other fields of human endeavor.

ABSTRACT VERSUS CONCRETE TERMINOLOGY IN DOCUMENTATION OF ACCIDENT DATA

The ladder of abstraction shown below provides a useful way of visualizing the relationship between 'abstract and "concrete" words.

The concrete words at the bottom of the ladder substitute for pointing to specific, unique and observable phenomenon. Words found on higher rungs are called 'abstract" since they abstract only the "meanings-in-common" from all the words in their supporting ladders. The term "human factors" is relatively high on the ladder of abstraction. So, too, is "human error." These illustrate why an attempt to develop a ladder of abstraction, using human factor terms was frustrated. The ladder was essentially without a base, without concrete. "point-to-able" events.

Relevance to Understanding Accidents

The relevance of the ladder of abstraction to accident investigation and subsequent research is that the higher the level of abstraction, the lesser the specificity of the accident descriptions and the greater the difficulty in communicating problems and describing solutions to decision-makers. Without a concrete description of the accident process, one from which the accident scenario can be visualized, the probability of reaching agreement about what should be done to produce better human decisions, especially in the cockpit of airplanes, is significantly reduced. Without agreement among researchers and "investigating strangers" on the referents of words and terms, each may be looking at a different "elephant," as in committee folklore.

PROBLEMS ENCOUNTERED BY ONE RESEARCH EFFORT

Overview of the Research

At a meeting sponsored by the NTSB in 1984, one attendee made an impassioned plea for human factors information that could be used to make decisions, for data specific enough to be used by a pilot walking out to the flight line in making the decision to fly or turn around. As a result of this plea. in 1985 a research team undertook a search for a new approach to the Understanding and teaching of pilot decision making. The purpose of the research was to develop the teaching materials that would respond to that plea. A recent paper by Russell Lawton of AOPA describes the research.[1] The research was based, in part, on the DECIDE decision making model developed for the hazardous materials transportation field. Certain modifications to the original DECIDE model were made, using work done during refinement of the STEP methodology. The research resulted in more successful simulator flight outcomes by subjects who had been instructed in the new approach than by others.

Development of the Decision Making Model

Relation of DECIDE to other Models. Although the DECIDE model closely paralleled Rockwell's aviation-based model,[2] DECIDE' s focus on "change" as the trigger for a "decision set" represented a major difference. More importantly for training, DECIDE viewed decision making as a continuous, adaptive process made up of serial and parallel decision sets. This process perception parallels STEP which defines accidents as processes.

Problems Encountered in Development.

To develop realistic decision- making training, the researchers selected STEP' s "mental movie" methodology as the vehicle for helping aircraft pilots improve their understanding and use of the decision process. Ideally, the mental movies used for decision making training would replicate critical, real-world in-flight scenarios.

In the quest for scenarios, aviation accident reports were selected which had an apparently above average quantity and quality of concrete pilot decision making information. However, as attempts were made to analyze those reports with STEP's "mental movie' approach, the researchers quickly realized that the pilot decision steps required to produce the outcome were rarely reported in concrete, "pointed to" terms or words. The table below displays the presence (Y) or absence (N) of such data in a report where 10 potential pilot decision sets were identified.

These results were not exceptional. In five accident cases. a 51 potential pilot decision sets were identified. Yet analysis showed that only 3 of those 51 decision sets contained fully reported decision events. The data base of specific, concrete accident (or near miss) decision events needed to develop the kind of requested pilot decision making training was clearly not available.

Project Completion. The project was completed, assisted by informed analysis and speculation, using the DECIDE model steps and the data available in the reports. The DECIDE step descriptions were refined and used in the training. The final model is shown below and is described more fully in the project report to AOPA.[3]

CONSEQUENCES OF THE LACK OF CONCRETE DATA

This research project highlights the lack of concrete accident event data. Even when accident reports are selected because of their apparent richness in concrete decision data, the reporting reflects an "attributes" view of decision making. Unfortunately, most accident data is reported in relatively abstract terminology, as attributes rather than as actions. Consider the two statements, "X was inattentive," and "X looked at wrong gauge." The first reflects a "state of being" or an attribute; the second, an observable, dynamic living action, an event. And it is in the real world of specific, concrete events that pilots make decisions and sometimes have accidents.

When the accident reports favoring attribute information are analyzed and plotted (using the STEP analytical procedures), the consequences of this attribute view become evident. One result is that the pilot's decision making process cannot be visualized or assessed in concrete terms. A second result is that ambiguities and gaps are found in accident event data. These two consequences combine to severely cripple attempts to provide concrete and specific pilot decision making training.

OVERVIEW OF STEP METHODOLOGY

The STEP methodology[4] used to overcome the ambiguity of the abstract human factors data in this research seems generally applicable.

Documentation of Accident Events.

When STEP methodology in used in accident investigation, each specific "event," each pointed to occurrence, is documented in a disciplined Building Block format consisting, as minimum, of a uniquely named actor, a single observable action, and measurable. concrete descriptors.

Such documentation is replicable in that words on or near the bottom rungs of the ladder of abstraction are used, words whose specific meanings permit visualization of the actions. STEP' s disciplining methodology assures compatibility of event documentation, even when several investigators are attempting to describe the same accident process.

Displaying of Accident Events

After several Building Blocks have been documented, they are displayed on a worksheet, linked by arrows to indicate the precede follow relationships among them. Before a new Building Block is placed in the individual actor's row on the worksheet, it is tested against previously displayed Building Blocks and placed in the appropriate time column indicating whether it occurred before, at the same time as, or after each previously displayed Building Block. Thus event sets are developed and documented. Each event set consists of at least two Building Blocks. In some cases, the event set may involve several Building Blocks, depending on whether the preceding event triggered several following events or whether more than one preceding event was necessary to trigger the following event. These are indicated by arrow links.

Testing Accident Documentation.

As a way of testing the logical consistency of events, linking arrows connections are examined to determine if (1) all the preceding events which were necessary to trigger the following event have been displayed and (2) if the preceding events were sufficient, by themselves, to trigger the following event. Finally, the event set is reviewed to determine whether too many events have been displayed. i. e. whether one or more of the following events could have been triggered by one fewer preceding event.

Identifying Gaps in Understanding of the Accident Process.

Gaps in the flow of events provide the investigators with hypotheses to aid the search for data. STEP' s BackSTEP technique is used to postulate events. In essence, backSTEP is a sequential logic tree diagram, placed horizontally to call attention to the importance of the relative timing of the displayed events. Using backSTEP, the investigator continually asks, "What could have triggered the event on the right?", until one of the events postulated as preceding the event on the right resembles one of the previously documented preceding events. The evidence of the specific intervening postulated events is then sought.

APPLICATION OF STEP TO PILOT DECISION MAKING RESEARCH

STEP Enables Visualization of the Decision Making Process

The objective, of the STEP methodology is to enable the researcher and anyone else to produce a mental movie of the scenario of interest. That mental movie should be complete and satisfying as well as capable of communicating the scenario being analyzed. Thus a STEP description of a pilot's actions during a critical time period should enable visualization of each decision set in terms of its constituent event(s). This, in turn, makes it possible to replicate and control the process outcomes in the future. STEP Can Define Hypotheses for Investigation and Research

STEP' s requirements for concrete terminology and disciplined documentation of reported accident data enable development of a data base of decision sets. Direct comparison of decision sets would be possible because of the use of concrete words in the documentation of events, eliminating the need for interpretation of data.

Hypotheses in Investigations. Investigators develop hypotheses during investigations to identify areas where "research" is needed to produce accident data. STEP assists in the development of such hypotheses, first, by disciplining the display of decision events on STEP worksheets and, second, through the use of backSTEP.

Hypotheses in Research. As accident investigators report specific data in STEP format, the grist for the generation of more encompassing hypotheses by researchers becomes available, Individual displays of decision sets in the STEP format could he overlaid. Such overlaying could be expected to produce hypotheses in the form of "When event 1 occurs, what event(s) follow?" As a data base of decision sets is built, the ability to compare and analyze decision sets becomes increasingly reliable. Graphically, this analysis can be illustrated with this three dimensional figure which shows the way in which decision sets can be "stacked" to facilitate comparison of the processes.