Position: Specific incidents, analyzed scientifically, will immediately identify successful interventions which have averted accidents, from which effective prevention strategies can be directly derived.

Problem: The objective is to reduce the incidence and severity of aviation accidents and incidents by formulating changes to human behavior which contribute to those adverse outcomes. Historical approaches have failed to achieve enduring modification of errant behavior. Traditional paradigms must be altered to overcome the inherent inadequacies of prevailing mishap origination concepts and investigation methodologies.

"If you do what you did, you're gonna get what you got!"

- Yogi Berra

I. Background

A. Current mishap investigation approaches, objectives and methodologies:

• Originated in 1930's by the U S Civil Aeronautics Board, based on legalistic concepts of culpability and liability.
• Evolved to fulfill statutory and regulatory demands rather than scientifically-based accident prevention requirements.
• Mandate finding of "probable cause(s)" as the primary goal for mishap investigation, which diminishes prevention initiatives.
• Focus on problems rather than on actions which overcame problems, failing both to prevent recurrence of old causes or emergence of new problems.
• Define "accident" or "incident" in terms of relative outcomes, which are irrelevant for identifying data significance or analyzing processes leading to the outcomes.
• Trivialize mishaps in which surviving participants could describe the process by which they prevented catastrophic outcomes.
• Lack investigative standardization, therefore lack the capability to measure or control either internal quality or inter-investigation consistency.
• Accept uncritically all data sources and formats without applying standards for verification of individual data elements, corroboration among data elements, or validation by scientific testing.
• Discourage candid, objective contributions by subjecting surviving participants to negative judgmental evaluation.
• Lack methodology for recasting anecdotal or episodic data into scientific format, therefore lack ability to discriminate among variables with sufficient certainty to verify statistical inferences.

B. Current analytic approaches, objectives and methodologies:

• Do not require variables to be normalized.
• Accept data at least four levels abstracted from observation as statistically valid:
• Investigator's descriptions of observed data
• Report format and statistical category designers' assumptions
• Investigator's interpretations of descriptive data to fit undefined format criteria ("Fill out the Form")
• Data analyst's reinterpretations of formatted data to fit undefined statistical category criteria
  
  - Render reliable statistical inference and productive safety recommendations virtually impossible because of high level of abstraction of safety data.

II. Human Factors/Human Error Models

A. Traditional

1. Prospective

Probabilistic Risk Assessment
Fault Tree
Error-tolerance Limit

2. Retrospective
.a. Qualitative

MORT
"Root Cause"

b. Quantitative

Epidemiology

B. Innovative

1. Reason: Latent vs. Active failures

• "Windows of opportunity"

2. Perrow: "Normal" accidents

• System complexity vs. coupling density

3. Ratner: Enhanced Safety Net

• Facilitating Operational Situations/Predisposing Underlying Factors

4. Benner: Multilinear Events Sequencing

• Event Pairs and Logic Testing

C. Common requirement of all models: A scientifically accurate description of

WHAT HAPPENED? - what people did to produce the outcome - is seldom formulated by current post-facto investigations.

Investigations must provide data from incidents or accidents to determine:

• Who did what, when, where, how and why?
• What specific behavior increased the likelihood of an undesired outcome?
• What specific behavior mitigated or prevented the undesired outcome?
• What behavior is preferred within the operational context?
• How can the preferred behavior be achieved?

III. Real-world problems of data relevance, quality, quantity and accessibility

A. Relevance

Investigation outputs must, at minimum, include compatible descriptions of:

• What was supposed to happen?
• What did happen? -- yet frequently do not.

B. Quality and quantity

• Post-accident investigation data rely principally on secondhand (or more abstract) sources, inferences or opinions, and rarely exist in sufficient quantity or quality to fulfill requirements without interposing abundant unverifiable conjecture.

• Incident investigation affords opportunities to obtain first-hand data directly from participants, including how they perceived deviations from plans, diagnosed effects of deviations, evaluated and selected among alternatives, and evaluated consequent intervention actions and their results; in short, the entire process by which the incipient accident was mitigated to achieve an acceptable outcome. [ref. "DECIDE" Model - see Hendrick, Benner & Lawton (1987) and Lawton, Benner, Clarke, et al (1987)]

C. Accessibility.

Two mutually exclusive problem areas exist:

1. Accessibility of data possessed by mishap participants:

• Current "Attitude" paradigm focuses on error, blame and liability, providing little motivation for survivors' willing cooperation. "Parties" to post-accident investigations commonly restrict participant employees from candid interviews, and interpose strict legal oversight to protect their perceived vulnerability as potential litigants.

2. Accessibility of information/data possessed by investigation agencies:

• Judgmental "Attitude" paradigm leads to restrictive covenants between investigation agencies and participants/witnesses which inhibit flow of data needed to fulfill current statistically-driven database approach to problem definition.

As a result of both problem areas, numerous independent incident-derived data bases have been established and are maintained by manufacturers and operators for "proprietary" internal risk management programs, which are rarely shared for prevention purposes.

IV. Alternative approach, objective and methodology

A. The aviation community's quest for a reality-based initiative designed to reduce the incidence and severity of aviation accidents involving human factors will be better served by:

1. Adjusting the "Opportunity" paradigm from "Post-accident" to "Pre-accident"; i.e., redefine "accident" and "incident" from mere outcome attributes to functional process descriptors: e.g., ".ane incident is an incipient accident which failed to attain its full potential because of successful intervention by persons, things or fortuity within the system."

• Redirect data acquisition concentration from accidents (which identify "causes" or operational failures) to incidents (which identify both operational failures and successful recoveries therefrom.)

2. Adjusting the "Attitude" paradigm from Negative: "What Went Wrong to 'Cause' the accident?" to Positive: "What Went Right to Prevent it?"

• Acknowledge both the ubiquity of human error, and human capability to recover from errors. Redirect resources toward successful intervention processes which thwart accident progression, thereby focusing on adaptation to error rather than error perpetuation.

• Expand investigations' focus to include positive factors. Encourage witnesses to provide accurate data for constructing effective prevention strategies, in contrast to defensive "CYA" strategies fostered by current judgmental perspectives which emphasize failures and errors.

3. Adjusting the "Investigation Methodology" paradigm from "Investigator's Option" to standardized investigation techniques which generate scientific data formats, permit critical data analyses, and allow data to be evaluated against measurable quality standards.

• Institute data input, process and product quality standards and controls.

  <

• Solicit, document and collect actions during actual successful anecdotal and episodic experiences to obtain timely source data for defining prevention initiatives which actually worked.

4. Adjusting the "Data" paradigm from unscientifically-defined statistically-driven data base fulfillment to direct observation, recasting the raw data into scientific formats accessible to quality control, verification and confidence testing procedures.

• Use observed data both to devise and validate models of successful error compensation. Current theoretical models employing abstract data categories or forms as templates drive investigation data selection and analysis, and are rarely (if ever) subjected to validation or verification testing.
  
• Compile participant-sourced data from intervention-focused investigations which identify successful accident mitigation tactics immediately and accurately, and are self verifying. Current error-focused investigation data deficiencies beget unverifiable assumption and conjecture about opportunities for error avoidance or response.
  
• Require accurate process timing data to support time-sensitive modeling methodologies; e.g., Perrow's "coupling density" and Benner's STEP/Multilinear Events Sequencing.

5. Adjusting the "Analysis" paradigm from current styles which are incapable of identifying defective data and rely on normal statistical methods, to techniques which enable investigators to define and describe interactions and interfaces accurately and reliably.

• Recognize necessity for "Rare Event" statistical treatment, including techniques for recasting anecdotal and episodic data to permit valid statistical treatment.
  
• Measure "What happened?" against "What was supposed to happen?" to establish disciplinary relevance boundaries, and identify when incipient accident processes begin, not merely where they end.

V. Recommendations

1. Redirect FAA investigative resources to develop more useful data about successful intervention actions during incidents, rather than investigating relatively data-poor NTSB-delegated mishaps.
2. Standardize investigation methodology to facilitate exercising objective quality controls on data and analyses.
3. Require that investigation methodology include accurate chronological data to define events and relational logic within the mishap process.
4. Establish a reporting process which
   1. encourages operational level persons to submit their incident experiences and anecdotes directly and immediately, and
   2. captures their data in a format that enhances the data's accuracy andutility with minimal need for editing or manipulation.

   (This could easily be an enhancement of the current NASA/ASRS.)

5. Establish a centralized repository through which participants worldwide can contribute and extract data and information.
6. Conduct prompt triage on anecdotal and episodic data to identify incidents which hold high potential for producing useful intervention actions. Analyze those incidents and provide priority feedback to operators who can benefit therefrom.
7. Enhance the accuracy and quality of scientific investigations to improve the efficacy of their outputs by establishing a single independent government-sponsored National Investigative Agency to replace the myriad organizations currently conducting unstandardized investigations which fail to achieve established quality management objectives. A dedicated national agency, staffed by professionally educated and trained investigators intimately acquainted with quality management, could overcome parochial bias and enhance investigations' effectiveness in preventing recurrence of mishaps of national influence. Core investigative expertise would be reinforced on a case-by-case basis by technical specialists from appropriate regulatory and operational agencies.

References

1. Benner, Ludwig Jr.; Four Accident Investigation Games: Simulations of The Accident Investigation Process. Events Analysis, Inc., Oakton, VA 1979. 2nd Edition 1982
2. Benner, Ludwig Jr.; "Rating Accident Models and Investigation Methodologies." J. Safety Research 16:3, Fall 1985, Chicago, IL.
3. Benner, Ludwig Jr. & Ira J. Rimson; "Quality Management for Accident Investigators". forum 24:3, October 1991 (Part 1); 25:1, March 1992 (Part 2). International Society of Air Safety Investigators (ISASI), Sterling, VA.
4. Feil, John A. & C. L. Lewis; "Accident/Incident Prevention through Airport Safety (An Airline Program for Airfield Inspection and Audits)." forum 23:4, February 1991. ISASI, Sterling, VA.
5. Ganse, Larry R.; "Integrating Accident Prevention and Investigation." forum 27:4, December 1994. ISASI, Sterling, VA.
6. Hendrick, Kingsley & L. Benner, Jr.; Investigating Accidents with STEP. New York, Marcel Dekker, 1987.
7. Hendrick, Kingsley, L. Benner & R. Lawton; "A Methodological Approach to the Search for Indirect Human Elements in Accident Investigations." Proceedings of the Fourth International Symposium on Aviation Psychology. (R. S. Jensen, Ed.) The Ohio State University, Columbus; April 29, 1987.
8. Lawton, Russell L., L. Benner, R. Clarke, R. Jensen, A. Adrion & J. Maresh; The DECIDE Approach to Training Pilots & Pilot Decision Making. AOPA Air Safety Foundation, Frederick, MD; June 1987.
9. Perrow, Charles; Normal Accidents. New York, Basic Books, 1984.
10. Rare Event/Accident Research Methodology; NBS Special Publication 482. Washington DC. U.S. Department of Commerce, National Bureau of Standards, 1977.
11. Ratner, Robert S. & J. Guselli; "Techniques for Identifying Indirect Causal Factors in Aviation Incidents and Accidents." forum 28:2, June 1995. ISASI, Sterling, VA
12. Reason, James T.; Human Error. Cambridge, Cambridge University Press, 1990.
13. Rimson, Ira J., & J. B. Galipault; "Incident Analysis: A Key to Accident Prevention". forum 12:3, Winter 1979. ISASI, Sterling, VA.
14. Rimson, Ira J.; "Standards for the Conduct of Aircraft Accident Investigations". forum 23:4, February 1991. ISASI, Sterling, VA.

---