Ahmed Abdeltawab Abdelgawad, & Tina Comes. (2019). Evaluation Framework for the iTRACK Integrated System. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: Evaluation and testing are major steps in the development of any information system, particularly if it is to be used in high-risk contexts such as conflicts. While thus far there are various approaches for testing against technology requirements; usability or usefulness, there is a lack of a comprehensive evaluation framework that combines the three elements. The lack of such a framework and commonly agreed standards constitutes a barrier for innovation, and at the same time imposes risks to responders if the technology is introduced without proper testing. This paper aims to close this gap. Based on a reviewing of evaluation methods and measurement metrics, we design a comprehensive evaluation framework including common code quality testing metrics, usability testing methods, subjective usefulness questionnaires, and performance indicators. We demonstrate our approach by using the example of an integrated system for the safety and security of humanitarian missions, and we highlight how our approach allows measuring the system?s quality and usefulness.
|
Bruna Diirr, & Marcos R. S. Borges. (2013). Applying software engineering testing techniques to evaluate emergency plans. In J. Geldermann and T. Müller S. Fortier F. F. T. Comes (Ed.), ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management (pp. 758–763). KIT; Baden-Baden: Karlsruher Institut fur Technologie.
Abstract: An emergency plan is an important artifact used throughout emergencies. Therefore, it is crucial that this plan covers the different scenarios faced by emergency team and that the prescribed procedures generate the desired results. To achieve these goals, emergency plans need to be exhaustively tested prior to its adoption. Emergency teams usually use simulation to test plans, but it is an expensive approach, demanding the building of complex environments and tools to get realistic situations. To be ready to go through a simulation exercise, a plan should not have many or critical mistakes, otherwise the simulation exercise would be wasted. We present a paired approach to evaluate emergency plans before simulation exercises, by applying software engineering testing techniques: Formal technical review (FTR) and mutation testing. The expected result is a plan more ready to go through simulation, as many of the mistakes are eliminated prior to simulation.
|
Gordon Gow, Peter Anderson, & Nuwan Waidyanatha. (2007). Hazard warnings in Sri Lanka: Challenges of internetworking with Common Alerting Protocol. In K. Nieuwenhuis P. B. B. Van de Walle (Ed.), Intelligent Human Computer Systems for Crisis Response and Management, ISCRAM 2007 Academic Proceedings Papers (pp. 281–293). Delft: Information Systems for Crisis Response and Management, ISCRAM.
Abstract: There is a growing call for the use of open source content standards for all-hazards, all-media alert and notification systems. This paper presents findings on the implementation of Common Alerting Protocol (CAP) as a content standard for a community-based hazard information network in Sri Lanka. CAP is being deployed as part of the HazInfo project, which has established last-mile networking capability for 32 tsunami-affected villages in Sri Lanka in order to study the suitability of various Information Communication Technologies (ICTs) for a standards-based community hazard information system. Results to date suggest that the basic internetworking arrangement at lower technical layers has proven to be reasonably robust and reliable but that a key challenge remains in the upper layers of application software and content provision. This is evident in the apparent difficulties faced when implementing CAP messaging over multiple last-mile systems that include commercial satellite and terrestrial network technologies (C/L/X-Band, GSM, and CDMA in modes of voice and text). Lessons learned from silent tests and live exercises point to several key bottlenecks in the system where the integrity of CAP messages is compromised due to problems associated with software interoperability or direct human intervention. The wider implication of this finding is that content standards by themselves are not sufficient to support appropriate and timely emergency response activities. Those working with content standards for hazard information systems must consider closely the interoperability issues at various layers of interconnectivity.
|
