|
Beth Veinott, Gary L. Klein, & Sterling Wiggins. (2010). Evaluating the effectiveness of the PreMortem technique on plan confidence. In C. Zobel B. T. S. French (Ed.), ISCRAM 2010 – 7th International Conference on Information Systems for Crisis Response and Management: Defining Crisis Management 3.0, Proceedings. Seattle, WA: Information Systems for Crisis Response and Management, ISCRAM.
Abstract: One problem affecting crisis management planning teams is overconfidence- An inflated belief that a plan will be successful. In this paper we compared the effect of several different methods for reducing individual team member confidence levels and compared each to a baseline control condition. One hundred and seventy-eight people participated in one of five conditions to evaluate an H1N1 flu epidemic plan in a university context. Over the course of evaluating the plan, participants provided several ratings of confidence in the plan's success and their understanding. We compared several techniques commonly used, such as critique, Pro/Cons generation, Cons only generation and a newer technique, PreMortem, to a baseline condition. The Pro/Cons generation, Cons only generation and the PreMortem technique all reliably reduced confidence levels more than baseline condition. Furthermore, the Premortem method, imagining that a plan has failed and then generating reasons to explain why, reliably reduced confidence more than each of the other conditions, and therefore can be a useful tool for combating overconfidence in crisis management planning. We discuss the results in the context of sensemaking and decision making theory.
|
|
|
Jennifer Mathieu, Mark Pfaff, Gary L. Klein, Jill L. Drury, Michael Geodecke, John James, et al. (2010). Tactical robust decision-making methodology: Effect of disease spread model fidelity on option awareness. In C. Zobel B. T. S. French (Ed.), ISCRAM 2010 – 7th International Conference on Information Systems for Crisis Response and Management: Defining Crisis Management 3.0, Proceedings. Seattle, WA: Information Systems for Crisis Response and Management, ISCRAM.
Abstract: We demonstrate a method of validating the utility of simpler, more agile models for supporting tactical robust decision making. The key is a focus on the decision space rather than the situation space in decision making under deep uncertainty. Whereas the situation space is characterized by facts about the operational environment, the decision space is characterized by a comparison of the options for action. To visualize the range of options available, we can use computer models to generate the distribution of plausible consequences for each decision option. If we can avoid needless detail in these models, we can save computational time and enable more tactical decision-making, which will in turn contribute to more efficient Information Technology systems. We show how simpler low fidelity, low precision models can be proved to be sufficient to support the decision maker. This is a pioneering application of exploratory modeling to address the human-computer integration requirements of tactical robust decision making.
|
|
|
Jill L. Drury, Gary L. Klein, Jennifer Mathieu, Yikun Liu, & Mark Pfaff. (2013). Sympathetic decisions: Incorporating impacts on others into emergency response decision spaces. In J. Geldermann and T. Müller S. Fortier F. F. T. Comes (Ed.), ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management (pp. 199–209). KIT; Baden-Baden: Karlsruher Institut fur Technologie.
Abstract: We designed two decision support tools and employed them during a one-week, simulation-driven experiment that included emergency responders acting in their real-life roles. Each tool visualized a “decision space”: A diagrammatic depiction of the relative desirability of one option versus another, including the inherent uncertainty in the potential outcomes. One requirement was to develop a tool accounting for the impacts of decisions on others, so that emergency responders can make “sympathetic decisions.” For example, one decision space enabled responders to request resources from surrounding jurisdictions while also considering the potential negative effects on the lending organizations. Another decision space enabled responders to engage in a strategic dialogue with the public: “listening” to the public's greatest concerns by mining social media to measure emotion, and thereby suggesting strategic communications addressing those concerns. We report how we designed the decision spaces and the qualitative results of using these spaces during the experiment.
|
|
|
Jill L. Drury, Gary L. Klein, Mark Pfaff, & Steven O. Entezari. (2012). Establishing collaborative option awareness during crisis management. In Z.Franco J. R. L. Rothkrantz (Ed.), ISCRAM 2012 Conference Proceedings – 9th International Conference on Information Systems for Crisis Response and Management. Vancouver, BC: Simon Fraser University.
Abstract: This paper presents empirical results of the use of a novel decision support prototype for emergency response situations, which was designed to enhance the understanding of the relative desirability of one potential course of action versus another. We have termed this understanding “option awareness.” In particular, this paper describes the process employed by pairs of experiment participants while performing emergency responder roles using different types of “decision space” visualizations to help them collaborate on decisions. We examined the decision making process via a detailed analysis of the communication between the cooperating team members. The results yield implications for design approaches for visualizing option awareness. © 2012 ISCRAM.
|
|
|
Jill L. Drury, Loretta More, Mark Pfaff, & Gary L. Klein. (2009). A principled method of scenario design for testing emergency response decision-making. In S. J. J. Landgren (Ed.), ISCRAM 2009 – 6th International Conference on Information Systems for Crisis Response and Management: Boundary Spanning Initiatives and New Perspectives. Gothenburg: Information Systems for Crisis Response and Management, ISCRAM.
Abstract: We are investigating decision aids that present potential courses of action available to emergency responders. To determine whether these aids improve decision quality, however, we first developed test scenarios that were challenging in well-understood ways to ensure testing under the full breadth of representative decision-making situations. We devised a three-step method of developing scenarios: define the decision space, determine the cost components of each decision's potential consequences based on the principles of Robust Decision Making, then choose conflicting pairs of cost components (e.g., a small fire, implying low property damage, in a densely inhabited area, which implies high personal injury). In a validation of this approach, experiment participants made decisions faster in non-ambiguous cases versus cases that included this principled introduction of ambiguity. Our Principled Ambiguity Method of scenario design is also appropriate for other domains as long as they can be analyzed in terms of costs of decision alternatives.
|
|
|
Louise K. Comfort, Brian Colella, Mark Voortman, Scott Connelly, Jill L. Drury, Gary L. Klein, et al. (2013). Real-time decision making in urgent events: Modeling options for action. In J. Geldermann and T. Müller S. Fortier F. F. T. Comes (Ed.), ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management (pp. 571–580). KIT; Baden-Baden: Karlsruher Institut fur Technologie.
Abstract: Decision making in extreme events presents a difficult challenge to emergency managers who are legally responsible for protecting life, property, and maintaining continuity of operations for their respective organizations or communities. Prior research has identified the benefits of gaining situation awareness in rapidly changing disaster contexts, but situation awareness is not always sufficient. We have investigated “option awareness” and the decision space to provide cognitive support for emergency managers to simulate computationally possible outcomes of different options before they make a decision. Employing a user-centered design process, we developed a computational model that rapidly generates ranges of likely outcomes for different options and displays them visually through a prototype decision-space interface that allows rapid comparison of the options. Feedback from emergency managers suggests that decision spaces may enable emergency managers to consider a wider range of options for decisions and may facilitate more targeted, effective decision making under uncertain conditions.
|
|
|
Yikun Liu, Sung Pil Moon, Mark Pfaff, Jill L. Drury, & Gary L. Klein. (2011). Collaborative option awareness for emergency response decision making. In E. Portela L. S. M.A. Santos (Ed.), 8th International Conference on Information Systems for Crisis Response and Management: From Early-Warning Systems to Preparedness and Training, ISCRAM 2011. Lisbon: Information Systems for Crisis Response and Management, ISCRAM.
Abstract: We have been using exploratory modeling to forecast multiple plausible outcomes for a set of decision options situated in the emergency response domain. Results were displayed as a set of box-plots illustrating outcome frequencies distributed across an evaluative dimension (e.g., cost, score, or utility). Our previous research showed that such displays provide what we termed “option awareness” – an ability to determine robust options that will have good outcomes across the broadest number of plausible futures. This paper describes an investigation into extending this approach to collaborative decision making by providing a visualization of both collaborative and individual decision spaces. We believe that providing such visualizations will be particularly important when each individuals decision space does not account for the synergy that may emerge from collaboration. We describe how providing collaborative decision spaces improves the robustness of joint decisions and engenders high confidence in these decisions.
|
|