• INTRODUCTION TO BackSTEP
• OBJECTIVE
• APPLICABILITY
• GENERAL GUIDANCE
  • DATA REQUIRED
  • DATA SOURCES
  • DATA PRESENTATION
• STEP-MES BackSTEP TREE DEVELOPMENT PROCEDURE
• CONFIRMING THE LOGIC TREE
• QUALITY CONTROL
• COMMENTS

Go to Guide: 0 1 2 3 4 5 6 7 8 9 10 11

During the development of STEP-MES-based matrix arrays, unlinked BBs indicate a gap in the flow of the actions that produced the outcome that investigators need to fill or explain. Gaps define data still needed to complete the investigation. Investigators can fill these gaps by adding new input data BBs from continuing observations of available sources, but when that option is exhausted, another option is needed. That option is to generate hypothesized scenarios and data BBs. Hypothesized scenarios can be developed using BackSTEP bounded logic trees.. BackSTEP displays of scenarios composed of hypothesized data BBs help investigators find plausible scenarios and data BBs, and by systemizing their development, disciplining investigator's speculations. BackSTEP(s) displays also provide a way to enlist the knowledge and experiences of experts or others familiar with the people or system behaviors or equipment or facilities to help develop hypothetical scenarios in an efficient, focused, intellectually challenging and rewarding procedure.

The hypothesized scenarios displayed on BackSTEP trees then point investigators to data needed to verify or discard each proposed BB in the hypothesized scenario. When the BBs in a hypothesized BackSTEP scenario are verified with data from additional observations, simulations or other means, they can be considered a feasible "array gap filler" and the BBs integrated directly into the STEP-MES matrix array, efficiently and quickly.

If no senario can be confirmed, the effort expended to develop the BackSTEP tree can demonstrate a "best efforts" attempt to resolve uncertainties. Additionally, the work will prepare the investigator to deal with the "second guessers" and "experienced experts" who always seem to be around to challenge investigator's work products.

OBJECTIVE

The objective of this Guide is to provide investigators with a disciplined investigator hypothesis generation and validation procedure for filiing gaps in their understanding of episodes, as represented by unlinked BBs in STEP-MES-based matrix arrays.

APPLICABILITY

Most investigators find themselves with aspects of an episode that they don't yet understand (unknowns) at some times during an investigation. Some methods like STEP-MES reveal those unknowns and also, sometimes, unknown unknowns (unk-unks) to investigators. The unknowns are revealed as gaps in linked STEP-MES data BB arrays. BackSTEP trees can be used to resolve unknowns during investigations by developing hypothesized scenarios and validating their BBs for integration into the matrix array to complete it. BackSTEP tree scenarios requie further data acquisition or testing, to determine the most verifiable BB gap-filling scenarios during a specific episode.

Logic trees are sometime used to organize speculations to describe gaps in understanding episodes. Logic trees are displays of converging events linked with cause-effect arrows and logic "gates" to show the flows of different events that will produce the "top event." Logic trees have various "stop rules" for bottom events on the tree. As a general rule, conventional deductively reasoned logic trees do not lend themselves well to the disciplined display of scenarios during an investigation. Their greatest value is in structuring the results of the search for different ways something can happen. They lacak the ability to represent the timing relationships among a tree's converging events, and lack of consensus for criteria for their "event" building block content. "Event time lines" are aso used in investigations. Event time lines are event lists sorted by their timing; they do not show coupled interactions among the various kids of data items used on the lists.

Additionally, BBs usually can not be recast adequately into the "fault tree" logic structure, required for a conventional logic trees, without introducing undesired biases into an investigation. Logic trees deal with one top event, as does BackSTEP, but the matrix array defines all "top events" and "bottom events" for BackSTEP tree type scrutiny. Finally, except for STEP-MES-based matrices, the relative timing of BBs can not be displayed efficiently in conventional logic trees. See for an illustration of the difference between STEP-MES-Tree and Fault Tree approach to matrix array gaps.

GENERAL GUIDANCE

BackSTEP trees in general provide a structured way to organize speculations about possible BBs to fill each specific gap in a STEP-MES matrix array. The difference between a BackSTEP used during an investigation, and a conventional logic tree is the horizontal tree configuration, the "bottom event" limiting the tree is specified, and the actor/action format for all entries in the blocks constituting the BackSTEP tree.

Figure 3-a The General BackSTEP Process

DATA REQUIRED

• The BBs on each side of a gap in a STEPMES matrix array.

• Procedures for constructing and confirming hypotheses with the BackSTEP tree.

• Candidate BB sequences or scenarios to enter into the BackSTEP tree during its construction.

DATA SOURCES

1. A BackSTEP tree is dependent on the investigator's or experts' knowledge and background for data that can be used to develop candidate scenarios and BBs displayed in the tree.

2. Tentative new data is generated as the tree is developed, by repeatedly asking the questions:
   • What prior BBs must have occurred to make the later BB occur (necessary logic)? and

   • Were any of these scenario input BBs sufficient to make the next BB happen by themselves (sufficient logic)?

   • Were all the input BBs necessary and sufficient to produce the next BB?

   • (The answers determine the configuration of the BackSTEP tree.)

3. After a BackSTEP tree scenario description is completed, the scenario BB entries dictate the additional data that need to be gathered, and the possible sources for such data.

DATA PRESENTATION

For investigations, a modified logic tree format is used to display hypotheses on a BackSTEP tree The BB on the left of the gap on the array becomes an anchor BB on the left end of the Tree, and the BB on the right of the gap) becomes the anchor on the right end of the tree. Typically, the flow of the BBs in a BackSTEP tree diverge from the left BB toward the center, and then converge again from the center toward the right BB.

STEP-MES -TREE DEVELOPMENT PROCEDURE

POST-IT notes can be used to record the hypothesized BBs to create a hypothetical scenario. Erasures and correction of entries are encouraged until the tree defines BBs to fill the gap. All BBs on a BackSTEP tree should be in the same "actor/action" format as used in STEP-MES-based matrix arrays. The tree development procedure follows.

1. Select a gap on a STEP-MES matrix to fill with data BBs. insetFig03a

2. Select the BBs on the right and left of the gap on the Matrix.
   

3. Prepare a BackSTEP tree workspace, such as a 24 x 36 inch sheet of paper from a classroom easel. Place the two BBs from the gap along the right and left margins of the BackSTEP tree.

4. Starting with the right BB, which is the equivalent to the top event on a fault tree, postulate what BBs might have led to the occurrence of that BB. Remember the "change maker" concept, and keep the initial speculation within the bounds of the known system and its operation. Display the first "tier" of BBs on the BackSTEP tree as shown in Figure 3b, which has 3 hypothesized BBs leading to the right BB of the gap.

   Figure 3-b Starting BackSTEP development
   
   

5. Note how the BackSTEP tree builds up from the right to the left. A1 to C1 are the "first tier" BBs. In this example, either A1, B1 or C1 could have produced the right BB. This reflects an "OR" gate array in a BackSTEP tree. See Figure 3-1b BB C2 and D1 for an example of an "AND" gate array in a BackSTEP tree. Necessary and sufficient tests for BB pairs are useful as each tier is added to the BackSTEP tree.

   Figure 3-c Expanding BackSTEP hypothesized BBs
   
   

6. Next, try to hypothesize at least one and probably two or more BBs that would have led to the BBs in the first tier to the left of the right end BB. Remember, use the actor/action format for every BB. At this time do not ignore any possibilities that you can reasonably visualize from your knowledge or other experts' knowledge of your system. Sometimes it is helpful to use the Change Tracking method, or the Energy Trace and Barrier Analysis method to help stimulate hypothesis development. These are both "paper" methods and thus relatively economical to use, when compared to simulations, tests, tear downs or other commonly used investigation techniques. Another convenient way to develop hypotheses is to try to picture the BBs that you think might have occurred, using the mental movie technique.

   Figure 3-d Building BackSTEP scenarios
   
   

7. Note that in the sample has two BBs leading into the first tier C1 in scenario 3 BB (C2 and D1.) If both had to occur before C1 occurred, you have an "AND" gate logical relationship. This kind of array arises from testing the links with necessary and sufficient logic as BBs are added to the tree.

   NOTE: When laying out hypothesized BBs, using an optional timeline and actor/action matrix to array the tentative BBs like on a MES/STEP matrix will enable the transfer of validated BBs directly to the gap in the existing matrix array.

   
   

8. Proceed step-by-step with your hypothesized BBs sideways to the left through each BB until you have reached the left-most BB, i.e., the BB on the left side of the STEP-MES Matrix array gap. This is where a major deviation from the conventional fault tree procedure occurs: the hypothesized scenario must tie into the BBs on BOTH ends of the tree. It is permissible to tie a scenario into other BBs such as BB A4 in Figure 3e below, to satisfy the necessary and sufficient tests. However, each scenario MUST tie into both end BBs to be a credible candidate "gap filler" scenario.

   Figure 3-e How a verified BackSTEP array might look
   
   
   In Figure 3.e, scenario 1 would require both the left BB and BB A4 inputs to produce A3, after which the scenario progresses to the right BB linearly through A2 and A1. Scenario B is all linear as often occurs when only one or a few BBs are required to fill a gap. But note that there is no link from the left BB to B4, which means it is not a valid candidate input to B right. Scenario C converges at C1 because actions D3,2 and1 and C4,3 and 2 and the left BB are all necessary before C1 is the input for the right BB, as might be experienced with a more complex "gap filler" scenario. This is the sort of necessary/sufficient thinking for BackSTEP.

9. Remember, this is a MATRIX and should be regarded as a work-in-process document until one or more of the scenarios displayed are confirmed with additional data from the investigation. It is OK to make changes and refinements to the matrix as often as necessary as the work progresses.

10. When a scenario uses more than one pathway, it is permissible to prepare that scenario on another BackSTEP tree, and use transfer symbols (a "copy here" sign post) to move from one matrix to another, so all the scenarios are indicated on a single matrix for investigation administrative purposes.

11. Check your BB flows for their logic, for consistency with the system design or operation, and for their input/output links before proceeding to the next task.

CONFIRMING BackSTEP TREES

1. Review the BBs in each path through the tree and try to determine how that BB might have been observed or how it changed something. Look for data to confirm each BB as well as data that would rule out the BB. Continue this process until

   only one path has only supporting data and no contradicting data, and

   all the other paths have no evidence or evidence showing they could not have happened.

2. By a process of elimination, identify the most likely path through the tree, if one can be found.

3. Alternatively, if data are lost or destroyed, or no data can be found to support any BBs along any path from the left BB to the right BB, three choices are available. Either

   acknowledge that data needed to support a choice are not available, and report the gap,

   perform a simulation of the episode, bounded by the BBs already on the STEP-MES Matrix, or

   subjectively identify the most likely path(s) using expert judgments, the "Delphi" method or some similar rationale to arrive at your conclusion.

4. If a reasonably verifiable scenario is found, take the most likely path and transfer those BBs from the BackSTEP tree onto your STEP-MES Matrix.

As an intermediate quality assurance step, check the logic against other BBs on the Matrix after the new BBs have been added. If no contradictory evidence or illogical, non-sequential relationships can be identified, you have probably done as much as can reasonably be expected to fill the gap in your understanding of the occurrence.

QUALITY CONTROL

BackSTEP-Tree quality control is exercised during the development process by insisting on the use of BB development quality controls and on the use of linked BBs as the tree develops. The final QC check occurs after the placement of the most likely scenario on the Matrix, and the subsequent QC testing of the Matrix in the area of the former gap.

COMMENTS

The BackSTEP tree should be viewed at first only as a "best guess" about what happened. It should not be relied on as the most likely scenario without substantiating data or testing. While the method will indicate the most likely scenario for that portion of the episode, the most rigorous validating method is the STEP-MES Matrix. Proposed actions based on these speculations should not be given great credence without further verification, using the STEP-MES Matrix. Possible exceptions: where no other information can be obtained or data didn't survive the episode.

Go to Guide: 0 1 2 3 4 5 6 7 8 9 10 11