|
Dat T. Nguyen, Firoj Alam, Ferda Ofli, & Muhammad Imran. (2017). Automatic Image Filtering on Social Networks Using Deep Learning and Perceptual Hashing During Crises. In eds Aurélie Montarnal Matthieu Lauras Chihab Hanachi F. B. Tina Comes (Ed.), Proceedings of the 14th International Conference on Information Systems for Crisis Response And Management (pp. 499–511). Albi, France: Iscram.
Abstract: The extensive use of social media platforms, especially during disasters, creates unique opportunities for humanitarian organizations to gain situational awareness and launch relief operations accordingly. In addition to the textual content, people post overwhelming amounts of imagery data on social networks within minutes of a disaster hit. Studies point to the importance of this online imagery content for emergency response. Despite recent advances in the computer vision field, automatic processing of the crisis-related social media imagery data remains a challenging task. It is because a majority of which consists of redundant and irrelevant content. In this paper, we present an image processing pipeline that comprises de-duplication and relevancy filtering mechanisms to collect and filter social media image content in real-time during a crisis event. Results obtained from extensive experiments on real-world crisis datasets demonstrate the significance of the proposed pipeline for optimal utilization of both human and machine computing resources.
|
|
|
Ferda Ofli, Firoj Alam, & Muhammad Imran. (2020). Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response. In Amanda Hughes, Fiona McNeill, & Christopher W. Zobel (Eds.), ISCRAM 2020 Conference Proceedings – 17th International Conference on Information Systems for Crisis Response and Management (pp. 802–811). Blacksburg, VA (USA): Virginia Tech.
Abstract: Multimedia content in social media platforms provides significant information during disaster events. The types of information shared include reports of injured or deceased people, infrastructure damage, and missing or found people, among others. Although many studies have shown the usefulness of both text and image content for disaster response purposes, the research has been mostly focused on analyzing only the text modality in the past. In this paper, we propose to use both text and image modalities of social media data to learn a joint representation using state-of-the-art deep learning techniques. Specifically, we utilize convolutional neural networks to define a multimodal deep learning architecture with a modality-agnostic shared representation. Extensive experiments on real-world disaster datasets show that the proposed multimodal architecture yields better performance than models trained using a single modality (e.g., either text or image).
|
|
|
Firoj Alam, Ferda Ofli, & Muhammad Imran. (2019). CrisisDPS: Crisis Data Processing Services. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: Over the last few years, extensive research has been conducted to develop technologies to support humanitarian aid
tasks. However, many technologies are still limited as they require both manual and automatic approaches, and
more importantly, are not ready to be integrated into the disaster response workflows. To tackle this limitation, we
develop automatic data processing services that are freely and publicly available, and made to be simple, efficient,
and accessible to non-experts. Our services take textual messages (e.g., tweets, Facebook posts, SMS) as input to
determine (i) which disaster type the message belongs to, (ii) whether it is informative or not, and (iii) what type of
humanitarian information it conveys. We built our services upon machine learning classifiers that are obtained from
large-scale comparative experiments utilizing both classical and deep learning algorithms. Our services outperform
state-of-the-art publicly available tools in terms of classification accuracy.
|
|
|
Firoj Alam, Ferda Ofli, Muhammad Imran, & Michael Aupetit. (2018). A Twitter Tale of Three Hurricanes: Harvey, Irma, and Maria. In Kees Boersma, & Brian Tomaszeski (Eds.), ISCRAM 2018 Conference Proceedings – 15th International Conference on Information Systems for Crisis Response and Management (pp. 553–572). Rochester, NY (USA): Rochester Institute of Technology.
Abstract: People increasingly use microblogging platforms such as Twitter during natural disasters and emergencies. Research studies have revealed the usefulness of the data available on Twitter for several disaster response tasks. However, making sense of social media data is a challenging task due to several reasons such as limitations of available tools to analyze high-volume and high-velocity data streams. This work presents an extensive multidimensional analysis of textual and multimedia content from millions of tweets shared on Twitter during the three disaster events. Specifically, we employ various Artificial Intelligence techniques from Natural Language Processing and Computer Vision fields, which exploit different machine learning algorithms to process the data generated during the disaster events. Our study reveals the distributions of various types of useful information that can inform crisis managers and responders as well as facilitate the development of future automated systems for disaster management.
|
|
|
Humaira Waqas, & Muhammad Imran. (2019). #CampFireMissing: An Analysis of Tweets About Missing and Found People From California Wildfires. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: Several research studies have shown the importance of social media data for humanitarian aid. Among others,
the issue of missing and lost people during disasters and emergencies is crucial for disaster managers. This work
analyzes Twitter data from a recent wildfire event to determine its usefulness for the mitigation of the missing and
found people issue. Data analysis performed using various filtering techniques, and trend analysis revealed that
Twitter contains important information potentially useful for emergency managers and volunteers to tackle this
issue. Many tweets were found containing full names, partial names, location information, and other vital clues
which could be useful for finding missing people.
|
|
|
Kiran Zahra, Muhammad Imran, & Frank O Ostermann. (2018). Understanding eyewitness reports on Twitter during disasters. In Kees Boersma, & Brian Tomaszeski (Eds.), ISCRAM 2018 Conference Proceedings – 15th International Conference on Information Systems for Crisis Response and Management (pp. 687–695). Rochester, NY (USA): Rochester Institute of Technology.
Abstract: Social media platforms such as Twitter provide convenient ways to share and consume important information during disasters and emergencies. Information from bystanders and eyewitnesses can be useful for law enforcement agencies and humanitarian organizations to get firsthand and credible information about an ongoing situation to gain situational awareness among other uses. However, identification of eyewitness reports on Twitter is challenging for many reasons. This work investigates the sources of tweets and classifies them into three types (i) direct eyewitnesses, (ii) indirect eyewitness, and (iii) vulnerable accounts. Moreover, we investigate various characteristics associated with each kind of eyewitness account. We observe that words related to perceptual senses (feeling, seeing, hearing) tend to be present in direct eyewitness messages, whereas emotions, thoughts, and prayers are more common in indirect witnesses. We believe these characteristics can help make more efficient computational methods and systems in the future for automatic identification of eyewitness accounts.
|
|
|
Michael Aupetit, & Muhammad Imran. (2017). Interactive Monitoring of Critical Situational Information on Social Media. In eds Aurélie Montarnal Matthieu Lauras Chihab Hanachi F. B. Tina Comes (Ed.), Proceedings of the 14th International Conference on Information Systems for Crisis Response And Management (pp. 673–683). Albi, France: Iscram.
Abstract: According to many existing studies, the data available on social media platforms such as Twitter at the onset of a crisis situation could be useful for disaster response and management. However, making sense of this huge data coming at high-rate is still a challenging task for crisis managers. In this work, we present an interactive social media monitoring tool that uses a supervised classification engine and natural language processing techniques to provide a detailed view of an on-going situation. The tool allows users to apply various filtering options using interactive timelines, critical entities, and other logical operators to get quick access to situational information. The evaluation of the tool conducted with crisis managers shows its significance for situational awareness and other crisis management related tasks.
|
|
|
Muhammad Imran, Carlos Castillo, Jesse Lucas, Patrick Meier, & Jakob Rogstadius. (2014). Coordinating human and machine intelligence to classify microblog communications in crises. In and P.C. Shih. L. Plotnick M. S. P. S.R. Hiltz (Ed.), ISCRAM 2014 Conference Proceedings – 11th International Conference on Information Systems for Crisis Response and Management (pp. 712–721). University Park, PA: The Pennsylvania State University.
Abstract: An emerging paradigm for the processing of data streams involves human and machine computation working together, allowing human intelligence to process large-scale data. We apply this approach to the classification of crisis-related messages in microblog streams. We begin by describing the platform AIDR (Artificial Intelligence for Disaster Response), which collects human annotations over time to create and maintain automatic supervised classifiers for social media messages. Next, we study two significant challenges in its design: (1) identifying which elements must be labeled by humans, and (2) determining when to ask for such annotations to be done. The first challenge is selecting the items to be labeled by crowd sourcing workers to maximize the productivity of their work. The second challenge is to schedule the work in order to reliably maintain high classification accuracy over time. We provide and validate answers to these challenges by extensive experimentation on real world datasets.
|
|
|
Muhammad Imran, Firoj Alam, Umair Qazi, Steve Peterson, & Ferda Ofli. (2020). Rapid Damage Assessment Using Social Media Images by Combining Human and Machine Intelligence. In Amanda Hughes, Fiona McNeill, & Christopher W. Zobel (Eds.), ISCRAM 2020 Conference Proceedings – 17th International Conference on Information Systems for Crisis Response and Management (pp. 761–773). Blacksburg, VA (USA): Virginia Tech.
Abstract: Rapid damage assessment is one of the core tasks that response organizations perform at the onset of a disaster to understand the scale of damage to infrastructures such as roads, bridges, and buildings. This work analyzes the usefulness of social media imagery content to perform rapid damage assessment during a real-world disaster. An automatic image processing system, which was activated in collaboration with a volunteer response organization, processed ~280K images to understand the extent of damage caused by the disaster. The system achieved an accuracy of 76% computed based on the feedback received from the domain experts who analyzed ~29K system-processed images during the disaster. An extensive error analysis reveals several insights and challenges faced by the system, which are vital for the research community to advance this line of research.
|
|
|
Muhammad Imran, Prasenjit Mitra, & Jaideep Srivastava. (2016). Cross-Language Domain Adaptation for Classifying Crisis-Related Short Messages. In A. Tapia, P. Antunes, V.A. Bañuls, K. Moore, & J. Porto (Eds.), ISCRAM 2016 Conference Proceedings ? 13th International Conference on Information Systems for Crisis Response and Management. Rio de Janeiro, Brasil: Federal University of Rio de Janeiro.
Abstract: Rapid crisis response requires real-time analysis of messages. After a disaster happens, volunteers attempt to classify tweets to determine needs, e.g., supplies, infrastructure damage, etc. Given labeled data, supervised machine learning can help classify these messages. Scarcity of labeled data causes poor performance in machine training. Can we reuse old tweets to train classifiers? How can we choose labeled tweets for training? Specifically, we study the usefulness of labeled data of past events. Do labeled tweets in different language help? We observe the performance of our classifiers trained using different combinations of training sets obtained from past disasters. We perform extensive experimentation on real crisis datasets and show that the past labels are useful when both source and target events are of the same type (e.g. both earthquakes). For similar languages (e.g., Italian and Spanish), cross-language domain adaptation was useful, however, when for different languages (e.g., Italian and English), the performance decreased.
|
|
|
Muhammad Imran, Shady Elbassuoni, Carlos Castillo, Fernando Díaz, & Patrick Meier. (2013). Extracting information nuggets from disaster- Related messages in social media. In J. Geldermann and T. Müller S. Fortier F. F. T. Comes (Ed.), ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management (pp. 791–801). KIT; Baden-Baden: Karlsruher Institut fur Technologie.
Abstract: Microblogging sites such as Twitter can play a vital role in spreading information during “natural” or man-made disasters. But the volume and velocity of tweets posted during crises today tend to be extremely high, making it hard for disaster-affected communities and professional emergency responders to process the information in a timely manner. Furthermore, posts tend to vary highly in terms of their subjects and usefulness; from messages that are entirely off-topic or personal in nature, to messages containing critical information that augments situational awareness. Finding actionable information can accelerate disaster response and alleviate both property and human losses. In this paper, we describe automatic methods for extracting information from microblog posts. Specifically, we focus on extracting valuable “information nuggets”, brief, self-contained information items relevant to disaster response. Our methods leverage machine learning methods for classifying posts and information extraction. Our results, validated over one large disaster-related dataset, reveal that a careful design can yield an effective system, paving the way for more sophisticated data analysis and visualization systems.
|
|
|
Reza Mazloom, HongMin Li, Doina Caragea, Muhammad Imran, & Cornelia Caragea. (2018). Classification of Twitter Disaster Data Using a Hybrid Feature-Instance Adaptation Approach. In Kees Boersma, & Brian Tomaszeski (Eds.), ISCRAM 2018 Conference Proceedings – 15th International Conference on Information Systems for Crisis Response and Management (pp. 727–735). Rochester, NY (USA): Rochester Institute of Technology.
Abstract: Huge amounts of data that are generated on social media during emergency situations are regarded as troves of critical information. The use of supervised machine learning techniques in the early stages of a disaster is challenged by the lack of labeled data for that particular disaster. Furthermore, supervised models trained on labeled data from a prior disaster may not produce accurate results, given the inherent variation between the current and the prior disasters. To address the challenges posed by the lack of labeled data for a target disaster, we propose to use a hybrid feature-instance adaptation approach based on matrix factorization and the k nearest neighbors algorithm, respectively. The proposed hybrid adaptation approach is used to select a subset of the source disaster data that is representative for the target disaster. The selected subset is subsequently used to learn accurate Naive Bayes classifiers for the target disaster.
|
|
|
Soudip Roy Chowdhury, Muhammad Imran, Muhammad Rizwan Asghar, Amer-Yahia, S., & Carlos Castillo. (2013). Tweet4act: Using incident-specific profiles for classifying crisis-related messages. In J. Geldermann and T. Müller S. Fortier F. F. T. Comes (Ed.), ISCRAM 2013 Conference Proceedings – 10th International Conference on Information Systems for Crisis Response and Management (pp. 834–839). KIT; Baden-Baden: Karlsruher Institut fur Technologie.
Abstract: We present Tweet4act, a system to detect and classify crisis-related messages communicated over a microblogging platform. Our system relies on extracting content features from each message. These features and the use of an incident-specific dictionary allow us to determine the period type of an incident that each message belongs to. The period types are: Pre-incident (messages talking about prevention, mitigation, and preparedness), during-incident (messages sent while the incident is taking place), and post-incident (messages related to the response, recovery, and reconstruction). We show that our detection method can effectively identify incident-related messages with high precision and recall, and that our incident-period classification method outperforms standard machine learning classification methods.
|
|
|
Starr Roxanne Hiltz, Amanda Hughes, Muhammad Imran, Linda Plotnick, Robert Power, & Murray Turoff. (2019). Requirements for Software to Support the use of Social Media in Emergency Management: A Delphi Study. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: Social Media contain a wealth of information that could improve the situational awareness of Emergency Managers during a crisis, but many barriers stand in the way. These include information overload, making it impossible to deal with the flood of raw posts, and lack of trust in unverified crowdsourced data. The purpose of this project is to build a communications bridge between emergency responders and technologists who can provide the advances needed to realize social media?s full potential. We are employing a Delphi study survey design, which is a technique for exploring and developing consensus among a group of experts around a particular topic. Participants include emergency managers and technologists with experience in software to support the use of social media in crisis response, from many countries. The topics of the study are described and preliminary, partial results presented for Round 1 of the study, based on 33 responses.
|
|
|
Xukun Li, Doina Caragea, Cornelia Caragea, Muhammad Imran, & Ferda Ofli. (2019). Identifying Disaster Damage Images Using a Domain Adaptation Approach. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: Approaches for effectively filtering useful situational awareness information posted by eyewitnesses of disasters,
in real time, are greatly needed. While many studies have focused on filtering textual information, the research
on filtering disaster images is more limited. In particular, there are no studies on the applicability of domain
adaptation to filter images from an emergent target disaster, when no labeled data is available for the target disaster.
To fill in this gap, we propose to apply a domain adaptation approach, called domain adversarial neural networks
(DANN), to the task of identifying images that show damage. The DANN approach has VGG-19 as its backbone,
and uses the adversarial training to find a transformation that makes the source and target data indistinguishable.
Experimental results on several pairs of disasters suggest that the DANN model generally gives similar or better
results as compared to the VGG-19 model fine-tuned on the source labeled data.
|
|
|
Zainab Akhtar, Ferda Ofli, & Muhammad Imran. (2021). Towards Using Remote Sensing and Social Media Data for Flood Mapping. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 536–551). Blacksburg, VA (USA): Virginia Tech.
Abstract: Ghana's capital, the Greater Accra Metropolitan Area (GAMA) is most vulnerable to flooding due to its high population density. This paper proposes the fusion of satellite imagery, social media, and geospatial data to derive near real-time (NRT) flood maps to understand human activity during a disaster and the extent of infrastructure damage. To that end, the paper presents an automatic thresholding technique for NRT flood mapping using Sentinel-1 images where four different speckle filters are compared using the VV, VH and VV/VH polarization to determine the best polarization(s) for delineating flood extents. The VV and VH bands together on Perona-Malik filtered images achieved the highest accuracy with an F1-score of 81.6%. Moreover, all tweet text and images were found to be located in flooded regions or in very close proximity to a flooded region, thus allowing crisis responders to better understand vulnerable communities and what humanitarian action is required.
|
|
|
Zoha Sheikh, Hira Masood, Sharifullah Khan, & Muhammad Imran. (2017). User-Assisted Information Extraction from Twitter During Emergencies. In eds Aurélie Montarnal Matthieu Lauras Chihab Hanachi F. B. Tina Comes (Ed.), Proceedings of the 14th International Conference on Information Systems for Crisis Response And Management (pp. 684–691). Albi, France: Iscram.
Abstract: Disasters and emergencies bring uncertain situations. People involved in such situations look for quick answers to their rapid queries. Moreover, humanitarian organizations look for situational awareness information to launch relief operations. Existing studies show the usefulness of social media content during crisis situations. However, despite advances in information retrieval and text processing techniques, access to relevant information on Twitter is still a challenging task. In this paper, we propose a novel approach to provide timely access to the relevant information on Twitter. Specifically, we employee Word2vec embeddings to expand initial users queries and based on a relevance feedback mechanism we retrieve relevant messages on Twitter in real-time. Initial experiments and user studies performed using a real world disaster dataset show the significance of the proposed approach.
|
|