Abstract: Assessing the effectiveness of crisis response is key to improving preparedness and adapting policies. One method for response evaluation is reviewing actual response activities and interactions. Response reports are often available in the form of natural language text data. Analyzing a large number of such reports requires automated or semi automated solutions. To improve the trustworthiness of methods for this purpose, we empirically validate the reliability of three relation extraction methods that we used to construct interorganizational collaboration networks by comparing them against human-annotated ground truth (crisis-specific situational reports and tweets). For entity extraction, we find that using a combination of two off-the-shelf methods (FlairNLP and SpaCy) is optimal for situational reports data and one method (SpaCy) for tweets data. For relation extraction, we find that a heuristics-based model that we built by leveraging word co-occurrence and deep and shallow syntax as features and training it on domain-specific text data outperforms two state-of-the-art relation extraction models (Stanford OpenIE and OneIE) that were pre-trained on general domain data. We also find that situational reports, on average, contain less entities and relations than tweets, but the extracted networks are more closely related to collaboration activities mentioned in the ground truth. As it is widely known that general domain tools might need adjustment to perform accurately in specific domains, we did not expect the tested off-the-shelf tools to perform highly accurately. Our point is to rather identify what accuracy one could reasonably expect when leveraging available resources as-is for domain specific work (in this case, crisis informatics), what errors (in terms of false positives and false negatives) to expect, and how to account for that.

Abstract: The Emergency Situation Awareness (ESA) tool began as a research study into automated web text mining to support emergency management use cases. It started in late 2009 by investigating how people respond on Twitter to specific emergency events and we quickly realized that every emergency situation is different and preemptively defining keywords to search for content on Twitter beforehand would likely miss important information. So, in late September 2011 we established location-based searches with the aim of collecting all the tweets published in Australia and New Zealand. This was the beginning of over a decade of collecting and processing tweets to help emergency response agencies and crisis coordination centres use social media content as a new channel of information to support their work practices and to engage with the community impacted by emergency events. This journey has seen numerous challenges overcome to continuously maintain a tweet stream for an operational system. This experience allows us to derive insights into the changing use of Twitter over this time. In this paper we present some of the lessons we’ve learned from maintaining a Twitter monitoring system for emergency management use cases and we provide some insights into the changing nature of Twitter usage by users over this period.

Abstract: Social media platforms have risen in popularity since their inception. These platforms have since then come to be at the forefront of controversies, from being accused of election interference to, more recently, disseminating fake news and campaigns to sway political behavior. One such episode took place on January 6 when a group of individuals stormed the United States Capitol, and the social media platform Parler came under scrutiny. The platform was accused of being a place for right-wing extremists and Trump supporters who claimed the 2020 election was fraudulent. Initial reports suggested these individuals used Parler to organize and call others to action. This paper explores the feasibility of using social media to detect alt-right radicalization and examines its possible relation to the Capitol Insurrection and Parler. Moreover, we examine if those events could have been detected and averted through the investigation of the platform.

Abstract: In times of emergency, crisis response agencies need to quickly and accurately assess the situation on the ground in order to deploy relevant services and resources. However, authorities often have to make decisions based on limited information, as data on affected regions can be scarce until local response services can provide first-hand reports. Fortunately, the widespread availability of smartphones with high-quality cameras has made citizen journalism through social media a valuable source of information for crisis responders. However, analyzing the large volume of images posted by citizens requires more time and effort than is typically available. To address this issue, this paper proposes the use of state-of-the-art deep neural models for automatic image classification/tagging, specifically by adapting transformer-based architectures for crisis image classification (CrisisViT). We leverage the new Incidents1M crisis image dataset to develop a range of new transformer-based image classification models. Through experimentation over the standard Crisis image benchmark dataset, we demonstrate that the CrisisViT models significantly outperform previous approaches in emergency type, image relevance, humanitarian category, and damage severity classification. Additionally, we show that the new Incidents1M dataset can further augment the CrisisViT models resulting in an additional 1.25% absolute accuracy gain.

Abstract: Emergency managers rely on formal and informal communication channels to identify needs in post-disaster environments. Message retransmission is a critical factor to ensure that help-seekers are identified by disaster responders. This paper uses a novel annotated dataset of Twitter posts from four major disasters that impacted the United States in 2021, to quantify the effect that expressed emotions and support typology have on retransmission. Poisson regression models are estimated, and the results show that messages seeking instrumental support are more likely to be retransmitted. Expressions of anger, fear, and sadness increase overall retweets. Moreover, expressions of anger, anticipation, or sadness increase the likelihood of retransmission for messages that seek instrumental help.

Abstract: Managing emergency events, such as natural disasters, requires management teams to have an up-to-date view of what is happening throughout the event. In this paper, we demonstrate how a method using a state-of-the-art open-sourced search engine and a large language model can generate accurate and comprehensive summaries by retrieving information from social media and online news sources. We evaluated our method on the TREC CrisisFACTS challenge dataset using automatic summarization metrics (e.g., Rouge-2 and BERTScore) and the manual evaluation performed by the challenge organizers. Our approach is the best in comprehensiveness despite presenting a high redundancy ratio in the generated summaries. In addition, since all pipeline components are few-shot, there is no need to collect training data, allowing us to deploy the system rapidly. Code is available at https://github.com/neuralmind-ai/visconde-crisis-summarization.

Abstract: Social media serves as an important communication channel for people affected by crises, creating a data source for emergency responders wanting to improve situational awareness. In particular, social listening on Twitter has been widely used for real-time analysis of crisis-related messages. This approach, however, is often hindered by the small fraction of (hyper-)localized content and by the inability to explicitly ask affected populations about aspects with the most operational value. Here, we explore a new form of social media data collected through targeted poll ads on Facebook. Using geo-targeted ads during flood events in six countries, we show that it is possible to collect thousands of poll responses within hours of launching the ad campaign, and at a cost of a few (US dollar) cents per response. We believe that this flexible, fast, and affordable data collection can serve as a valuable complement to existing approaches.

Abstract: Crisis Named Resources (or CNRs) are social media pages and accounts named after a crisis event. Using the COVID-19 Pandemic as a case study, we identified and examined the role of CNRs that shared humor on Twitter. Our analyses showed that humor-based CNRs shared virus-related rumors, stigma, safety measures, opinions, sarcasm, and news updates. These resources also shared the overall anger and frustration over the year 2020. We conclude by discussing the critical role of humor based CNRs in crisis response.

Abstract: Validation of findings is a challenge in practice-based research. While analysis is being conducted and findings are being constructed out of data collected in a defined period, practitioners continue with their activities. This issue is exacerbated in the field of crisis management, where high volatility and personnel turnover make the capacity to attend research demands scarce. Therefore, conducting classic member validation is logistically challenging for the researcher. The need for rigor and validity calls for alternative mechanisms to fulfill requirements for academic research. This article presents an approach for validation of results of a qualitative study with public organizations that use social media as a source of information in the context of crisis management. The unavailability of original interview-objects to validate our findings resulted in an alternative validation method that leveraged experts in crisis informatics. By presenting our approach, we contribute to encouraging rigor in qualitative research while maintaining the relationship between practice and academia.

Abstract: People increasingly use social media at the time of crisis, which produces a social media data deluge, where the public may find it difficult to locate trustworthy and credible information. Therefore, they often turn to authoritative sources: official individuals and organizations who are trusted to provide reliable information. It is then imperative that their credible messages reach and engage the widest possible audience, especially among those affected. In this study, we explore the role of metadata and linguistic factors in facilitating three types of engagement — retweets, replies, and favorites— with posts by authoritative sources. We find that many factors are similarly important across models (popularity, sociability, activity). However, some features are salient for only a specific type of engagement. We conclude by providing guidance to authoritative sources on how they may optimize specific types of engagement: retweets for information propagation, replies for in-depth sense-making, and favorites for cross-purpose visibility.

Abstract: Microblogging platforms have been increasingly used by the public and crisis managers in crisis. The increasing volume of data has made such platforms more difficult for officials to find on-the-ground information and understand the public’s perception of the evolving risks. The crisis informatics literature has proposed various technological solutions to find relevant information from social media. However, the cognitive processes of the affected population and their subsequent responses, such as perceptions, emotional and behavioral responses, are still under-examined at scale. Yet, such information is important for gauging public perception of risks, an important task for PIOs and emergency managers. In this work, we leverage the noise-cutting power of collective attention and take cues from the Protective Action Decision Model, to propose a method that estimates shifts in collective attention with a special focus on the cognitive processes of those affected and their subsequent responses.

Abstract: This paper summarizes the final year of the four-year Text REtrieval Conference Incident Streams track (TREC-IS), which has produced a large dataset comprising 136,263 annotated tweets, spanning 98 crisis events. Goals of this final year were twofold: 1) to add new categories for assessing messages, with a focus on characterizing the audience, author, and images associated with these messages, and 2) to enlarge the TREC-IS dataset with new events, with an emphasis of deeper pools for sampling. Beyond these two goals, TREC-IS has nearly doubled the number of annotated messages per event for the 26 crises introduced in 2021 and has released a new parallel dataset of 312,546 images associated with crisis content – with 7,297 tweets having annotations about their embedded images. Our analyses of this new crisis data yields new insights about the context of a tweet; e.g., messages intended for a local audience and those that contain images of weather forecasts and infographics have higher than average assessments of priority but are relatively rare. Tweets containing images, however, have higher perceived priorities than tweets without images. Moving to deeper pools, while tending to lower classification performance, also does not generally impact performance rankings or alter distributions of information-types. We end this paper with a discussion of these datasets, analyses, their implications, and how they contribute both new data and insights to the broader crisis informatics community.

Abstract: The frequency of wildfires is growing day by day due to vastly climate changes. Forest fires can have a severe impact on human lives and the environment, which can be minimised if the population has early and accurate warning mechanisms. To date, social media are able to contribute to early warning with the additional, crowd-sourced information they can provide to the emergency response workers during a crisis event. Nevertheless, the detection of real-world fire incidents using social media data, while filtering out the unavoidable noise, remains a challenging task. In this paper, we present an alert framework for the real-time detection of fire events and we propose a novel multimodal event detection model, which fuses both probabilistic and graph methodologies and is evaluated on the largest fires in Spain during 2019.