|
Peter Berggren, Björn J.E. Johansson, Nicoletta Baroutsi, Isabelle Turcotte, & Sébastien Tremblay. (2014). Assessing team focused behaviors in emergency response teams using the shared priorities measure. In and P.C. Shih. L. Plotnick M. S. P. S.R. Hiltz (Ed.), ISCRAM 2014 Conference Proceedings – 11th International Conference on Information Systems for Crisis Response and Management (pp. 130–134). University Park, PA: The Pennsylvania State University.
Abstract: The purpose of this work in progress paper is to report on the method development of the Shared Priorities measure to include content analysis, as a way of gaining a deeper understanding of team work in crisis/emergency response. An experiment is reported where the performance of six trained teams is compared with the performance of six non-trained teams. The experiment was performed using an emergency response microworld simulation with a forest fire scenario. Dependent measures were simulation performance, the Crew Awareness Rating Scale (CARS), and content analysis. Trained teams performed better and scored higher on measures of team behaviors.
|
|
|
Nitesh Bharosa, Marijn Janssen, & Harry Bouwman. (2010). Ex-ante evaluation of disaster information systems: A gaming-simulation approach. In C. Zobel B. T. S. French (Ed.), ISCRAM 2010 – 7th International Conference on Information Systems for Crisis Response and Management: Defining Crisis Management 3.0, Proceedings. Seattle, WA: Information Systems for Crisis Response and Management, ISCRAM.
Abstract: Failures in coordination and information sharing between relief agencies have increased the number of calls for innovative information system (IS) designs. While both the academic and the industrial communities have proposed many IS designs, methodologies for the ex-ante evaluation of such IS designs are scarce. Consequently, disaster IS architects are offered little guidance in the ex-ante evaluation process. Not only is it difficult to evaluate IS designs in practice, it is also difficult to include the conditions of disaster situations in the evaluation process. This paper explores the difficulties of ex-ante evaluation and discusses the suitability of the gaming-simulation methodology for the evaluation of principle-based IS designs. Gaming-simulation entails the use of professionals, scenarios and prototypes and can be adapted to a quasi-experimental form enabling researchers to control contextual interferences and rule out alternative explanations. This paper concludes with some discussions on the advantages and pitfalls of employing gaming-simulation for IS evaluation.
|
|
|
Jill L. Drury, Loretta More, Mark Pfaff, & Gary L. Klein. (2009). A principled method of scenario design for testing emergency response decision-making. In S. J. J. Landgren (Ed.), ISCRAM 2009 – 6th International Conference on Information Systems for Crisis Response and Management: Boundary Spanning Initiatives and New Perspectives. Gothenburg: Information Systems for Crisis Response and Management, ISCRAM.
Abstract: We are investigating decision aids that present potential courses of action available to emergency responders. To determine whether these aids improve decision quality, however, we first developed test scenarios that were challenging in well-understood ways to ensure testing under the full breadth of representative decision-making situations. We devised a three-step method of developing scenarios: define the decision space, determine the cost components of each decision's potential consequences based on the principles of Robust Decision Making, then choose conflicting pairs of cost components (e.g., a small fire, implying low property damage, in a densely inhabited area, which implies high personal injury). In a validation of this approach, experiment participants made decisions faster in non-ambiguous cases versus cases that included this principled introduction of ambiguity. Our Principled Ambiguity Method of scenario design is also appropriate for other domains as long as they can be analyzed in terms of costs of decision alternatives.
|
|
|
Rianne Gouman, Masja Kempen, & Niek Wijngaards. (2010). Actor-agent team experimentation in the context of incident management. In C. Zobel B. T. S. French (Ed.), ISCRAM 2010 – 7th International Conference on Information Systems for Crisis Response and Management: Defining Crisis Management 3.0, Proceedings. Seattle, WA: Information Systems for Crisis Response and Management, ISCRAM.
Abstract: The collaboration between humans (actors) and artificial entities (agents) can be a potential performance boost. Agents, as complementary artificial intelligent entities, can alleviate actors from certain activities, while enlarging the collective effectiveness. This paper describes our approach for experimentation with actors, agents and their interaction. This approach is based on a principled combination of existing empirical research methods and is illustrated by a small experiment which assesses the performance of a specific actor-agent team in comparison with an actor-only team in an incident management context. The REsearch and Simulation toolKit (RESK) is instrumental for controlled and repeatable experimentation. The indicative findings show that the approach is viable and forms a basis for further data collection and comparative experiments. The approach supports applied actor-agent research to show its (dis)advantages as compared to actor-only solutions.
|
|
|
Rianne Gouman, Masja Kempen, Philip De Vree, Toon Capello, Eddy Van Der Heijden, & Niek Wijngaards. (2007). The borsele files: The challenge of acquiring usable data under chaotic circumstances. In K. Nieuwenhuis P. B. B. Van de Walle (Ed.), Intelligent Human Computer Systems for Crisis Response and Management, ISCRAM 2007 Academic Proceedings Papers (pp. 93–103). Delft: Information Systems for Crisis Response and Management, ISCRAM.
Abstract: Conducting empirical research involves a balancing act between scientific rigor and real-life pragmatics. DECIS Lab researches systems-of-systems, consisting of humans and artificial systems involved in collaborative decision making under chaotic circumstances. An important objective is the usefulness of our results to our major application domain: crisis management. DECIS Lab was involved to set up a crisis management exercise experiment and according measurements regarding an improvement in internal communication at Gemeente (Municipality) Borsele. In this paper the empirical research regarding this experiment, the methodology and its results are briefly outlined. Our main lessons learned concern the interrelationship between scenario, experiment and measurements; the problem of acquiring usable data; and the challenges of conducting grounded research.
|
|
|
Tim J. Grant, F.L.E. Geugies, & Peter A. Jongejan. (2013). Social media in command & control: A proof-of principle experiment. In J. Geldermann and T. Müller S. Fortier F. F. T. Comes (Ed.), ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management (pp. 52–61). KIT; Baden-Baden: Karlsruher Institut fur Technologie.
Abstract: The literature on the organizational use of social media in crisis response and management is largely concerned with communication between organizations and the general public (“citizens”). By contrast, there are few papers on the use of social media within organizations for operational purposes. One essential operational capability in such organizations is Command & Control (C2) or its equivalent. Our research focuses on the use of social media in C2 for crisis management at the operational and tactical levels. To enable the use of social media in C2, Jongejan and Grant (2012) extended Reuter, Marx and Pipek's (2011) theoretical framework. In the research reported in this paper, the extended framework was tested by performing a proof-of-principle experiment for a famine relief scenario with human subjects in the laboratory. The results show that more extensive concept demonstrations and field experimentation are justified.
|
|
|
Guido Te Brake, Tjerk De Greef, Jasper Lindenberg, Jouke Rypkema, & Nanja Smets. (2006). Developing adaptive user interfaces using a game-based simulation environment. In M. T. B. Van de Walle (Ed.), Proceedings of ISCRAM 2006 – 3rd International Conference on Information Systems for Crisis Response and Management (pp. 6–10). Newark, NJ: Royal Flemish Academy of Belgium.
Abstract: In dynamic settings, user interfaces can provide more optimal support if they adapt to the context of use. Providing adaptive user interfaces to first responders may therefore be fruitful. A cognitive engineering method that incorporates development iterations in both a simulated and a real-world environment is used to develop new adaptive concepts. In a simulated 3D-world, created with the Unreal Tournament game-engine, a team of emergency personnel have to rescue people and develop an understanding of the situation. We believe a game-based simulation environment can provide an effective platform for experiments in which crisis management situations can be created under controlled circumstances. Using this simulation, support concepts based on adaptive user interfaces can be developed and evaluated before they are implemented in a real-world setting. This paper describes the work that has been done, and presents the design of the planned experiments.
|
|
|
Erich Heumüller, Sebastian Richter, & Ulrike Lechner. (2013). Training, test and experimentation: A classification of command post exercises. In J. Geldermann and T. Müller S. Fortier F. F. T. Comes (Ed.), ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management (pp. 110–114). KIT; Baden-Baden: Karlsruher Institut fur Technologie.
Abstract: Based on a state of the art analysis of exercise classifications and empirical findings from an Action Research approach this paper presents a classification for command post exercises, that addresses identified shortcomings concerning practical support for goal-achievement and evaluation in exercise design. The authors' classification distinguishes between Training, Test and Experimentation Exercises, which are characterized by the following aspects: Goal and purpose of exercise, participants, evaluation content and output, evaluation methodology, role of observers, scenario complexity, potential for organizational innovation and results. The classification was developed in an Action Research approach with an empirical basis of four command post exercises. Results indicate high benefit for exercises and the approach is perceived rather simple, easy to understand and to apply.
|
|
|
Corine H.G. Horsch, Nanja J. J. M. Smets, Mark A. Neerincx, & Raymond H. Cuijpers. (2013). Comparing performance and situation awareness in USAR unit tasks in a virtual and real environment. In J. Geldermann and T. Müller S. Fortier F. F. T. Comes (Ed.), ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management (pp. 556–560). KIT; Baden-Baden: Karlsruher Institut fur Technologie.
Abstract: A convenient way to test Urban Search And Rescue (USAR) robots would be in virtual environments (VEs). Evaluations in VEs are generally accepted as alternative for real scenarios. There are obvious differences between operation in a real and virtual environment. Nonetheless, the current experiment showed no significant differences in situation awareness (SA) and performance during several elementary tasks (e.g. slalom) between a virtual world and a previous experiment in reality (Mioch, Smets, & Neerincx, 2012). Only small dependencies between the unit tasks were found. The effect of individual differences (like gender, km driven per year, and gaming experience), were significant for certain elementary tasks. Testing robots in virtual environments could still be useful even if differences between VE and reality exist, since comparisons of different conditions in VE seems to have the same results as the same comparison in the field (Bishop & Rohrmann, 2003; Van Diggelen, Looije, Mioch, Neerincx, & Smets, 2012).
|
|
|
Yasir Javed, & Tony Norris. (2013). Computerized system to enhance situation awareness: Key challenges associated with the design, evaluation, and extension of a prototype. In J. Geldermann and T. Müller S. Fortier F. F. T. Comes (Ed.), ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management (pp. 951–961). KIT; Baden-Baden: Karlsruher Institut fur Technologie.
Abstract: Successful decision making and task execution in emergency management require appropriate levels of situation awareness (SA). This paper proposes an ontology-based model for the design of a computer-based system, Situation Aware Vigilant Emergency Reasoner (SAVER) that supports the individual, shared and team SA of managers in emergency situations. SAVER is evaluated in simulated experiments that demonstrate the improvements in SA performance. The paper provides a complete description of the SAVER design, implementation, evaluation and its proposed extension from a proof-of-concept to a production environment.
|
|
|
Jill L. Drury, Gary L. Klein, Jennifer Mathieu, Yikun Liu, & Mark Pfaff. (2013). Sympathetic decisions: Incorporating impacts on others into emergency response decision spaces. In J. Geldermann and T. Müller S. Fortier F. F. T. Comes (Ed.), ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management (pp. 199–209). KIT; Baden-Baden: Karlsruher Institut fur Technologie.
Abstract: We designed two decision support tools and employed them during a one-week, simulation-driven experiment that included emergency responders acting in their real-life roles. Each tool visualized a “decision space”: A diagrammatic depiction of the relative desirability of one option versus another, including the inherent uncertainty in the potential outcomes. One requirement was to develop a tool accounting for the impacts of decisions on others, so that emergency responders can make “sympathetic decisions.” For example, one decision space enabled responders to request resources from surrounding jurisdictions while also considering the potential negative effects on the lending organizations. Another decision space enabled responders to engage in a strategic dialogue with the public: “listening” to the public's greatest concerns by mining social media to measure emotion, and thereby suggesting strategic communications addressing those concerns. We report how we designed the decision spaces and the qualitative results of using these spaces during the experiment.
|
|
|
Michael E. Stiso, Aslak Wegner Eide, & Antoine Pultier. (2015). A foray into the use of serious games in controlled research on crisis management. In L. Palen, M. Buscher, T. Comes, & A. Hughes (Eds.), ISCRAM 2015 Conference Proceedings ? 12th International Conference on Information Systems for Crisis Response and Management. Kristiansand, Norway: University of Agder (UiA).
Abstract: Controlled experiments on crisis management could provide many insights into the human factors that lead to effective performance in the area. However, the challenge of establishing a controlled environment directly relevant to the chaotic settings in which crisis management occurs means that such experiments are scarce. Here, we describe our attempt to use a videogame (ARMA III) as a realistic but controllable environment for research in this domain. We successfully developed a testbed linking the game world to the front-end of a prototype command-and-control system, so that one can use the latter to monitor events in the former. However, when it came to developing controlled scenarios for the experiment, we discovered that too much realism can be a problem. This paper outlines the challenges we encountered and provides recommendations for researchers and game designers interested in the use of serious games in scientific research.
|
|
|
Niels Netten, & Maarten Van Someren. (2006). Automated support for dynamic information distribution in incident management. In M. T. B. Van de Walle (Ed.), Proceedings of ISCRAM 2006 – 3rd International Conference on Information Systems for Crisis Response and Management (pp. 230–237). Newark, NJ: Royal Flemish Academy of Belgium.
Abstract: For all emergency response personnel involved in crisis situations it is essential to timely acquire all information critical to their task performance. However, in practice errors occur in the distribution of information between these collaborating actors leading to mistakes and subsequently more damage to the situation. In this paper we present a prototype system for dynamic information distribution able to support the information flow between collaborating crisis actors. The system has been evaluated by means of simulated experiments that use data from a real incident scenario. The results indicate that automated support by means of Machine Learning method works well. Especially, when actor work context features are included, then the performance on selecting and distributing relevant information is high. Furthermore, actors acquire relevant information much faster making group communication much more efficient.
|
|
|
Jan Maarten Schraagen, Aletta Eikelboom, Kees Van Dongen, & Guido Te Brake. (2005). Experimental evaluation of a critical thinking tool to support decision making in crisis situations. In B. C. B. Van de Walle (Ed.), Proceedings of ISCRAM 2005 – 2nd International Conference on Information Systems for Crisis Response and Management (pp. 181–189). Brussels: Royal Flemish Academy of Belgium.
Abstract: Building up proper situation awareness is one of the most difficult tasks in the beginning stages of large-scale accidents. As ambiguous information about the events becomes available, decision makers are often tempted to quickly choose a particular story to explain the events. Subsequent information that contradicts the initial story may easily be discarded and cognitive tunnel vision takes over. Our approach, as part of the COMBINED Systems project, is to prevent tunnel vision by providing critical thinking support. In a laboratory experiment with 60 participants, we tested this hypothesis by comparing the Critical Thinking tool with a 'no support' control condition and a 'minimal support' condition. Participants acted as crisis managers determining the likely cause of an accident based on different pieces of information. The results show a positive impact of the tool on both the decision making process and decision making effectiveness.
|
|
|
Nanja J. J. M. Smets, Guido Te Brake, Jasper Lindenberg, & Mark A. Neerincx. (2007). Influence of mobile map size and user capacities on situation awareness tested in a virtual environment. In K. Nieuwenhuis P. B. B. Van de Walle (Ed.), Intelligent Human Computer Systems for Crisis Response and Management, ISCRAM 2007 Academic Proceedings Papers (pp. 557–564). Delft: Information Systems for Crisis Response and Management, ISCRAM.
Abstract: In the near future, first responders may become equipped with mobile devices providing navigation, decision and communication support. Because of the complex and chaotic circumstances in which these devices will be used, the devices should support the creation and maintenance of adequate situation awareness. Extensive testing of such devices for crisis management in real-life is expensive, complex, risky and only possible for specific settings. Therefore, we developed a synthetic task environment that is suited for developing and evaluating new concepts. In this paper, we present the results of the first experiment in this environment. Participants had to rescue victims in a synthetic world, and were supported by a map of the area showing the location and orientation of the participant and the victims. The experiment focused on the effects of map size and user's spatial ability on the quality of the situational awareness that was developed by the first responders. Besides the results of the experiment, experiences with the use of a synthetic environment for evaluation and development purposes are presented.
|
|