Long, Z., McCreadiem, R., & Imran, M. (2023). CrisisViT: A Robust Vision Transformer for Crisis Image Classification. In Jaziar Radianti, Ioannis Dokas, Nicolas Lalone, & Deepak Khazanchi (Eds.), Proceedings of the 20th International ISCRAM Conference (pp. 309–319). Omaha, USA: University of Nebraska at Omaha.
Abstract: In times of emergency, crisis response agencies need to quickly and accurately assess the situation on the ground in order to deploy relevant services and resources. However, authorities often have to make decisions based on limited information, as data on affected regions can be scarce until local response services can provide first-hand reports. Fortunately, the widespread availability of smartphones with high-quality cameras has made citizen journalism through social media a valuable source of information for crisis responders. However, analyzing the large volume of images posted by citizens requires more time and effort than is typically available. To address this issue, this paper proposes the use of state-of-the-art deep neural models for automatic image classification/tagging, specifically by adapting transformer-based architectures for crisis image classification (CrisisViT). We leverage the new Incidents1M crisis image dataset to develop a range of new transformer-based image classification models. Through experimentation over the standard Crisis image benchmark dataset, we demonstrate that the CrisisViT models significantly outperform previous approaches in emergency type, image relevance, humanitarian category, and damage severity classification. Additionally, we show that the new Incidents1M dataset can further augment the CrisisViT models resulting in an additional 1.25% absolute accuracy gain.
|
Muhammad Imran, Prasenjit Mitra, & Jaideep Srivastava. (2016). Cross-Language Domain Adaptation for Classifying Crisis-Related Short Messages. In A. Tapia, P. Antunes, V.A. Bañuls, K. Moore, & J. Porto (Eds.), ISCRAM 2016 Conference Proceedings ? 13th International Conference on Information Systems for Crisis Response and Management. Rio de Janeiro, Brasil: Federal University of Rio de Janeiro.
Abstract: Rapid crisis response requires real-time analysis of messages. After a disaster happens, volunteers attempt to classify tweets to determine needs, e.g., supplies, infrastructure damage, etc. Given labeled data, supervised machine learning can help classify these messages. Scarcity of labeled data causes poor performance in machine training. Can we reuse old tweets to train classifiers? How can we choose labeled tweets for training? Specifically, we study the usefulness of labeled data of past events. Do labeled tweets in different language help? We observe the performance of our classifiers trained using different combinations of training sets obtained from past disasters. We perform extensive experimentation on real crisis datasets and show that the past labels are useful when both source and target events are of the same type (e.g. both earthquakes). For similar languages (e.g., Italian and Spanish), cross-language domain adaptation was useful, however, when for different languages (e.g., Italian and English), the performance decreased.
|
Nilani Algiriyage, Rangana Sampath, Raj Prasanna, Kristin Stock, Emma Hudson-Doyle, & David Johnston. (2021). Identifying Disaster-related Tweets: A Large-Scale Detection Model Comparison. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 731–743). Blacksburg, VA (USA): Virginia Tech.
Abstract: Social media applications such as Twitter and Facebook are fast becoming a key instrument in gaining situational awareness (understanding the bigger picture of the situation) during disasters. This has provided multiple opportunities to gather relevant information in a timely manner to improve disaster response. In recent years, identifying crisis-related social media posts is analysed as an automatic task using machine learning (ML) or deep learning (DL) techniques. However, such supervised learning algorithms require labelled training data in the early hours of a crisis. Recently, multiple manually labelled disaster-related open-source twitter datasets have been released. In this work, we create a large dataset with 186,718 tweets by combining a number of such datasets and evaluate the performance of multiple ML and DL algorithms in classifying disaster-related tweets in three settings, namely ``in-disaster'', ``out-disaster'' and ``cross-disaster''. Our results show that the Bidirectional LSTM model with Word2Vec embeddings performs well for the tweet classification task in all three settings. We also make available the preprocessing steps and trained weights for future research.
|
Reza Mazloom, HongMin Li, Doina Caragea, Muhammad Imran, & Cornelia Caragea. (2018). Classification of Twitter Disaster Data Using a Hybrid Feature-Instance Adaptation Approach. In Kees Boersma, & Brian Tomaszeski (Eds.), ISCRAM 2018 Conference Proceedings – 15th International Conference on Information Systems for Crisis Response and Management (pp. 727–735). Rochester, NY (USA): Rochester Institute of Technology.
Abstract: Huge amounts of data that are generated on social media during emergency situations are regarded as troves of critical information. The use of supervised machine learning techniques in the early stages of a disaster is challenged by the lack of labeled data for that particular disaster. Furthermore, supervised models trained on labeled data from a prior disaster may not produce accurate results, given the inherent variation between the current and the prior disasters. To address the challenges posed by the lack of labeled data for a target disaster, we propose to use a hybrid feature-instance adaptation approach based on matrix factorization and the k nearest neighbors algorithm, respectively. The proposed hybrid adaptation approach is used to select a subset of the source disaster data that is representative for the target disaster. The selected subset is subsequently used to learn accurate Naive Bayes classifiers for the target disaster.
|
Anne-Francoise Rutkowski, Willem Van Groenendaal, Bartel A. Van De Walle, & Jan Pol. (2004). Decision support technology to support risk analysis and disaster recovery plan formulation: Towards IT and business continuity. In B. C. B. Van de Walle (Ed.), Proceedings of ISCRAM 2004 – 1st International Workshop on Information Systems for Crisis Response and Management (pp. 127–132). Brussels: Royal Flemish Academy of Belgium.
Abstract: The paper presents a four-phase action research project that was (and still is) conducted at the department of Information Management Customer Support and Operations (IM\CS&O) of a large multi-national company. The department is in charge of ICT-service continuity and has to produce ICT recovery plans that are integrated with the organization's overall Business Continuity plan. Interviews, Group Support System (GSS) technologies as well as a risk survey have been used to gather information and identify risks and threats. A systematic quantitative classification, measuring the impact of loss of ICT services on the company's business processes in terms of cost and risk will allow in the near future to utilize an economic decision model to prioritize the core activities of training and implementation of a recovery disaster plan. The research has made clear to the involved protagonists the necessity to share information, to develop awareness, and to formulate a shared recovery disaster plan to ensure ICT/business continuity and/or recovery when ICT disruptions occurs. © Proceedings ISCRAM 2004.
|
Seungwon Yang, Haeyong Chung, Xiao Lin, Sunshin Lee, Liangzhe Chen, Andrew Wood, et al. (2013). PhaseVis1: What, when, where, and who in visualizing the four phases of emergency management through the lens of social media. In J. Geldermann and T. Müller S. Fortier F. F. T. Comes (Ed.), ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management (pp. 912–917). KIT; Baden-Baden: Karlsruher Institut fur Technologie.
Abstract: The Four Phase Model of Emergency Management has been widely used in developing emergency/disaster response plans. However, the model has received criticism contrasting the clear phase distinctions in the model with the complex and overlapping nature of phases indicated by empirical evidence. To investigate how phases actually occur, we designed PhaseVis based on visualization principles, and applied it to Hurricane Isaac tweet data. We trained three classification algorithms using the four phases as categories. The 10-fold cross-validation showed that Multi-class SVM performed the best in Precision (0.8) and Naïve Bayes Multinomial performed the best in F-1 score (0.782). The tweet volume in each category was visualized as a ThemeRiver[TM], which shows the 'What' aspect. Other aspects – 'When', 'Where', and 'Who' – Are also integrated. The classification evaluation and a sample use case indicate that PhaseVis has potential utility in disasters, aiding those investigating a large disaster tweet dataset.
|
Therese Habig, Richard Lüke, Simon Gehlhar, Torben Sauerland, & Daniel Tappe. (2021). A Consolidated Understanding of Disaster Community Technologies. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 778–791). Blacksburg, VA (USA): Virginia Tech.
Abstract: Since the beginning of this millennium, there has been an increasing use of social media and crowdsourcing (SMCS) technologies in disaster situations (Reuter & Kaufhold, 2018). Disaster management organizations and corresponding research are increasingly working on ways of integrating SMCS into the processes of crisis management. In a changing technological landscape to address disasters, and with increasing diversity of stakeholders in disasters, the purpose of this research is to provide an overview of technologies for SMCS within disasters to improve community resilience. The identified and analyzed technologies are summarized under the term “Disaster Community Technologies” (DCT). The paper presents a classification schema (the “DCT-schema”) for those technologies. The goal is to generate an overview of DCT in a rapidly evolving environment and to provide the practical benefit for different stakeholders to identify the right one from the overview.
|
Tiberiu Sosea, Iustin Sirbu, Cornelia Caragea, Doina Caragea, & Traian Rebedea. (2021). Using the Image-Text Relationship to Improve Multimodal Disaster Tweet Classification. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 691–704). Blacksburg, VA (USA): Virginia Tech.
Abstract: In this paper, we show that the text-image relationship of disaster tweets can be used to improve the classification of tweets from emergency situations. To this end, we introduce DisRel, a dataset which contains 4,600 multimodal tweets, collected during the disasters that hit the USA in 2017, and manually annotated with coherence image-text relationships, such as Similar and Complementary. We explore multiple models to detect these relationships and perform a comprehensive analysis into the robustness of these methods. Based on these models, we build a simple feature augmentation approach that can leverage the text-image relationship. We test our methods on 2 tasks in CrisisMMD: Humanitarian Categories and Damage Assessment, and observe an increase in the performance of the relationship-aware methods.
|
Xukun Li, & Doina Caragea. (2020). Improving Disaster-related Tweet Classification with a Multimodal Approach. In Amanda Hughes, Fiona McNeill, & Christopher W. Zobel (Eds.), ISCRAM 2020 Conference Proceedings – 17th International Conference on Information Systems for Crisis Response and Management (pp. 893–902). Blacksburg, VA (USA): Virginia Tech.
Abstract: Social media data analysis is important for disaster management. Lots of prior studies have focused on classifying a tweet based on its text or based on its images, independently, even if the tweet contains both text and images. Under the assumptions that text and images may contain complementary information, it is of interest to construct classifiers that make use of both modalities of the tweet. Towards this goal, we propose a multimodal classification model which aggregates text and image information. Our study aims to provide insights into the benefits obtained by combining text and images, and to understand what type of modality is more informative with respect to disaster tweet classification. Experimental results show that both text and image classification can be improved by the multimodal approach.
|
Xukun Li, Doina Caragea, Cornelia Caragea, Muhammad Imran, & Ferda Ofli. (2019). Identifying Disaster Damage Images Using a Domain Adaptation Approach. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: Approaches for effectively filtering useful situational awareness information posted by eyewitnesses of disasters,
in real time, are greatly needed. While many studies have focused on filtering textual information, the research
on filtering disaster images is more limited. In particular, there are no studies on the applicability of domain
adaptation to filter images from an emergent target disaster, when no labeled data is available for the target disaster.
To fill in this gap, we propose to apply a domain adaptation approach, called domain adversarial neural networks
(DANN), to the task of identifying images that show damage. The DANN approach has VGG-19 as its backbone,
and uses the adversarial training to find a transformation that makes the source and target data indistinguishable.
Experimental results on several pairs of disasters suggest that the DANN model generally gives similar or better
results as compared to the VGG-19 model fine-tuned on the source labeled data.
|
Zijun Long, & Richard Mccreadie. (2021). Automated Crisis Content Categorization for COVID-19 Tweet Streams. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 667–678). Blacksburg, VA (USA): Virginia Tech.
Abstract: Social media platforms, like Twitter, are increasingly used by billions of people internationally to share information. As such, these platforms contain vast volumes of real-time multimedia content about the world, which could be invaluable for a range of tasks such as incident tracking, damage estimation during disasters, insurance risk estimation, and more. By mining this real-time data, there are substantial economic benefits, as well as opportunities to save lives. Currently, the COVID-19 pandemic is attacking societies at an unprecedented speed and scale, forming an important use-case for social media analysis. However, the amount of information during such crisis events is vast and information normally exists in unstructured and multiple formats, making manual analysis very time consuming. Hence, in this paper, we examine how to extract valuable information from tweets related to COVID-19 automatically. For 12 geographical locations, we experiment with supervised approaches for labelling tweets into 7 crisis categories, as well as investigated automatic priority estimation, using both classical and deep learned approaches. Through evaluation using the TREC-IS 2020 COVID-19 datasets, we demonstrated that effective automatic labelling for this task is possible with an average of 61% F1 performance across crisis categories, while also analysing key factors that affect model performance and model generalizability across locations.
|
Zijun Long, & Richard McCreadie. (2022). Is Multi-Modal Data Key for Crisis Content Categorization on Social Media? In Rob Grace, & Hossein Baharmand (Eds.), ISCRAM 2022 Conference Proceedings – 19th International Conference on Information Systems for Crisis Response and Management (pp. 1068–1080). Tarbes, France.
Abstract: The user-base of social media platforms, like Twitter, has grown dramatically around the world over the last decade. As people post everything they experience on social media, large volumes of valuable multimedia content are being recorded online, which can be analysed to help for a range of tasks. Here we specifically focus on crisis response. The majority of prior works in this space focus on using machine learning to categorize single-modality content (e.g. text of the posts, or images shared), with few works jointly utilizing multiple modalities. Hence, in this paper, we examine to what extent integrating multiple modalities is important for crisis content categorization. In particular, we design a pipeline for multi-modal learning that fuses textual and visual inputs, leverages both, and then classifies that content based on the specified task. Through evaluation using the CrisisMMD dataset, we demonstrate that effective automatic labelling for this task is possible, with an average of 88.31% F1 performance across two significant tasks (relevance and humanitarian category classification). while also analysing cases that unimodal models and multi-modal models success and fail.
|
Zou, H. P., Caragea, C., Zhou, Y., & Caragea, D. (2023). Semi-Supervised Few-Shot Learning for Fine-Grained Disaster Tweet Classification. In Jaziar Radianti, Ioannis Dokas, Nicolas Lalone, & Deepak Khazanchi (Eds.), Proceedings of the 20th International ISCRAM Conference (pp. 385–395). Omaha, USA: University of Nebraska at Omaha.
Abstract: The shared real-time information about natural disasters on social media platforms like Twitter and Facebook plays a critical role in informing volunteers, emergency managers, and response organizations. However, supervised learning models for monitoring disaster events require large amounts of annotated data, making them unrealistic for real-time use in disaster events. To address this challenge, we present a fine-grained disaster tweet classification model under the semi-supervised, few-shot learning setting where only a small number of annotated data is required. Our model, CrisisMatch, effectively classifies tweets into fine-grained classes of interest using few labeled data and large amounts of unlabeled data, mimicking the early stage of a disaster. Through integrating effective semi-supervised learning ideas and incorporating TextMixUp, CrisisMatch achieves performance improvement on two disaster datasets of 11.2% on average. Further analyses are also provided for the influence of the number of labeled data and out-of-domain results.
|