Robert Power, Bella Robinson, & Mark Cameron. (2023). Insights from a Decade of Twitter Monitoring for Emergency Management. In V. L. Thomas J. Huggins (Ed.), Proceedings of the ISCRAM Asia Pacific Conference 2022 (pp. 247–257). Palmerston North, New Zealand: Massey Unversity.
Abstract: The Emergency Situation Awareness (ESA) tool began as a research study into automated web text mining to support emergency management use cases. It started in late 2009 by investigating how people respond on Twitter to specific emergency events and we quickly realized that every emergency situation is different and preemptively defining keywords to search for content on Twitter beforehand would likely miss important information. So, in late September 2011 we established location-based searches with the aim of collecting all the tweets published in Australia and New Zealand. This was the beginning of over a decade of collecting and processing tweets to help emergency response agencies and crisis coordination centres use social media content as a new channel of information to support their work practices and to engage with the community impacted by emergency events. This journey has seen numerous challenges overcome to continuously maintain a tweet stream for an operational system. This experience allows us to derive insights into the changing use of Twitter over this time. In this paper we present some of the lessons we’ve learned from maintaining a Twitter monitoring system for emergency management use cases and we provide some insights into the changing nature of Twitter usage by users over this period.
|
McCreadie, R., & Buntain, C. (2023). CrisisFACTS: Buidling and Evaluating Crisis Timelines. In Jaziar Radianti, Ioannis Dokas, Nicolas Lalone, & Deepak Khazanchi (Eds.), Proceedings of the 20th International ISCRAM Conference (pp. 320–339). Omaha, USA: University of Nebraska at Omaha.
Abstract: Between 2018 and 2021, the Incident Streams track (TREC-IS) developed standard approaches for classifying information types and criticality of tweets during crises. While successful in producing substantial collections of labeled data, TREC-IS as a data challenge had several limitations: It only evaluated information at type-level rather than what was reported; it only used Twitter data; and it lacked measures of redundancy in system output. This paper introduces Crisis Facts and Cross-Stream Temporal Summarization (CrisisFACTS), a new data challenge piloted in 2022 and developed to address these limitations. The CrisisFACTS framework recasts TREC-IS into an event-summarization task using multiple disaster-relevant data streams and a new fact-based evaluation scheme, allowing the community to assess state-of-the-art methods for summarizing disaster events Results from CrisisFACTS in 2022 include a new test-collection comprising human-generated disaster summaries along with multi-platform datasets of social media, crisis reports and news coverage for major crisis events.
|
Chauhan, A. (2023). Humor-Based COVID-19 Twitter Accounts. In Jaziar Radianti, Ioannis Dokas, Nicolas Lalone, & Deepak Khazanchi (Eds.), Proceedings of the 20th International ISCRAM Conference (pp. 417–427). Omaha, USA: University of Nebraska at Omaha.
Abstract: Crisis Named Resources (or CNRs) are social media pages and accounts named after a crisis event. Using the COVID-19 Pandemic as a case study, we identified and examined the role of CNRs that shared humor on Twitter. Our analyses showed that humor-based CNRs shared virus-related rumors, stigma, safety measures, opinions, sarcasm, and news updates. These resources also shared the overall anger and frustration over the year 2020. We conclude by discussing the critical role of humor based CNRs in crisis response.
|
Anjum, U., Zadorozhny, V., & Krishnamurthy, P. (2023). Localization of Events Using Neural Networks in Twitter Data. In Jaziar Radianti, Ioannis Dokas, Nicolas Lalone, & Deepak Khazanchi (Eds.), Proceedings of the 20th International ISCRAM Conference (pp. 909–919). Omaha, USA: University of Nebraska at Omaha.
Abstract: In this paper, we develop a model with neural networks to localize events using microblogging data. Localization is the task of finding the location of an event and can be done by discovering event signatures in microblogging data. We use the deep learning methodology of Bi-directional Long Short-Term Memory (Bi-LSTM) to learn event signatures. We propose a methodology for labeling the Twitter date for use in Bi-LSTM However, there might not be enough data available to train the Bi-LSTM and learn the event signatures. Hence, the data is augmented using generative adversarial networks (GAN). Finally, we combine event signatures at different temporal and spatial granularity to improve the accuracy of event localization. We use microblogging data collected from Twitter to evaluate our model and compare it with other baseline methods.
|
Dario Salza, Edoardo Arnaudo, Giacomo Blanco, & Claudio Rossi. (2022). A 'Glocal' Approach for Real-time Emergency Event Detection in Twitter. In Rob Grace, & Hossein Baharmand (Eds.), ISCRAM 2022 Conference Proceedings – 19th International Conference on Information Systems for Crisis Response and Management (pp. 570–583). Tarbes, France.
Abstract: Social media like Twitter offer not only an unprecedented amount of user-generated content covering developing emergencies but also act as a collector of news produced by heterogeneous sources, including big and small media companies as well as public authorities. However, this volume, velocity, and variety of data constitute the main value and, at the same time, the key challenge to implement and automatic detection and tracking of independent emergency events from the real-time stream of tweets. Leveraging online clustering and considering both textual and geographical features, we propose, implement, and evaluate an algorithm to automatically detect emergency events applying a ‘glocal’ approach, i.e., offering a global coverage while detecting events at local (municipality level) scale.
|
Cody Buntain, Richard Mccreadie, & Ian Soboroff. (2022). Incident Streams 2021 Off the Deep End: Deeper Annotations and Evaluations in Twitter. In Rob Grace, & Hossein Baharmand (Eds.), ISCRAM 2022 Conference Proceedings – 19th International Conference on Information Systems for Crisis Response and Management (pp. 584–604). Tarbes, France.
Abstract: This paper summarizes the final year of the four-year Text REtrieval Conference Incident Streams track (TREC-IS), which has produced a large dataset comprising 136,263 annotated tweets, spanning 98 crisis events. Goals of this final year were twofold: 1) to add new categories for assessing messages, with a focus on characterizing the audience, author, and images associated with these messages, and 2) to enlarge the TREC-IS dataset with new events, with an emphasis of deeper pools for sampling. Beyond these two goals, TREC-IS has nearly doubled the number of annotated messages per event for the 26 crises introduced in 2021 and has released a new parallel dataset of 312,546 images associated with crisis content – with 7,297 tweets having annotations about their embedded images. Our analyses of this new crisis data yields new insights about the context of a tweet; e.g., messages intended for a local audience and those that contain images of weather forecasts and infographics have higher than average assessments of priority but are relatively rare. Tweets containing images, however, have higher perceived priorities than tweets without images. Moving to deeper pools, while tending to lower classification performance, also does not generally impact performance rankings or alter distributions of information-types. We end this paper with a discussion of these datasets, analyses, their implications, and how they contribute both new data and insights to the broader crisis informatics community.
|
Gaëtan Caillaut, Cécile Gracianne, Nathalie Abadie, Guillaume Touya, & Samuel Auclair. (2022). Automated Construction of a French Entity Linking Dataset to Geolocate Social Network Posts in the Context of Natural Disasters. In Rob Grace, & Hossein Baharmand (Eds.), ISCRAM 2022 Conference Proceedings – 19th International Conference on Information Systems for Crisis Response and Management (pp. 654–663). Tarbes, France.
Abstract: During natural disasters, automatic information extraction from Twitter posts is a valuable way to get a better overview of the field situation. This information has to be geolocated to support effective actions, but for the vast majority of tweets, spatial information has to be extracted from texts content. Despite the remarkable advances of the Natural Language Processing field, this task is still challenging for current state-of-the-art models because they are not necessarily trained on Twitter data and because high quality annotated data are still lacking for low resources languages. This research in progress address this gap describing an analytic pipeline able to automatically extract geolocatable entities from texts and to annotate them by aligning them with the entities present in Wikipedia/Wikidata resources. We present a new dataset for Entity Linking on French texts as preliminary results, and discuss research perspectives for enhancements over current state-of-the-art modeling for this task.
|
Jens Kersten, Jan Bongard, & Friederike Klan. (2022). Gaussian Processes for One-class and Binary Classification of Crisis-related Tweets. In Rob Grace, & Hossein Baharmand (Eds.), ISCRAM 2022 Conference Proceedings – 19th International Conference on Information Systems for Crisis Response and Management (pp. 664–673). Tarbes, France.
Abstract: Overload reduction is essential to exploit Twitter text data for crisis management. Often used pre-trained machine learning models require training data for both, crisis-related and off-topic content. However, this task can also be formulated as a one-class classification problem in which labeled off-topic samples are not required. Gaussian processes (GPs) have great potential in both, binary and one-class settings and are therefore investigated in this work. Deep kernel learning combines the representative power of text embeddings with the Bayesian formalism of GPs. Motivated by this, we investigate the potential of deep kernel models for the task of classifying crisis-related tweet texts with special emphasis on cross-event applications. Compared to standard binary neural networks, first experiments with one-class GP models reveal a great potential for realistic scenarios, offering a fast and flexible approach for interactive model training without requiring off-topic training samples and comprehensive expert knowledge (only two model parameters involved).
|
Ahmed Alnuhayt, Suvodeep Mazumdar, Vitaveska Lanfranchi, & Frank Hopfgartner. (2022). Understanding Reactions to Misinformation – A Covid-19 Perspective. In Rob Grace, & Hossein Baharmand (Eds.), ISCRAM 2022 Conference Proceedings – 19th International Conference on Information Systems for Crisis Response and Management (pp. 687–700). Tarbes, France.
Abstract: The increasing use of social media as an information source brings further challenges – social media platforms can be an excellent medium for disseminating public awareness and critical information, that can be shared across large populations. However, misinformation in social media can have immense implications on public health, risking the effectiveness of health interventions as well as lives. This has been particularly true in the case of COVID-19 pandemic, with a range of misinformation, conspiracy theories and propaganda being spread across social channels. In our study, through a questionnaire survey, we set out to understand how members of the public interact with different sources when looking for information on COVID-19. We explored how participants react when they encounter information they believe to be misinformation. Through a set of three behaviour tasks, synthetic misinformation posts were provided to the participants who chose how they would react to them. In this work in progress study, we present initial findings and insights into our analysis of the data collected. We highlight what are the most common reactions to misinformation and also how these reactions are different based on the type of misinformation.
|
Usman Anjum, Vladimir Zadorozhny, & Prashant Krishnamurthy. (2021). TBAM: Towards An Agent-Based Model to Enrich Twitter Data. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 146–158). Blacksburg, VA (USA): Virginia Tech.
Abstract: Twitter is widely being used by researchers to understand human behavior, e.g. how people behave when an event occurs and how it changes their microblogging pattern. The changing microblogging behavior can have an important application in the form of detecting events. However, the Twitter data that is available has limitations in it has incomplete and noisy information and has irregular samples. In this paper we create a model, calledTwitter Behavior Agent-Based Model (TBAM)to simulate Twitter pattern and behavior using Agent-Based Modeling(ABM). The generated data can be used in place or to complement the real-world data and improve the accuracy of event detection. We confirm the validity of our model by comparing it with real data collected from Twitter
|
Cody Buntain, Richard Mccreadie, & Ian Soboroff. (2021). Incident Streams 2020: TRECIS in the Time of COVID-19. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 621–639). Blacksburg, VA (USA): Virginia Tech.
Abstract: Between 2018 and 2019, the Incident Streams track (TREC-IS) has developed standard approaches for classifying the types and criticality of information shared in online social spaces during crises, but the introduction of SARS-CoV-2 has shifted the landscape of online crises substantially. While prior editions of TREC-IS have lacked data on large-scale public-health emergencies as these events are exceedingly rare, COVID-19 has introduced an over-abundance of potential data, and significant open questions remain about how existing approaches to crisis informatics and datasets built on other emergencies adapt to this new context. This paper describes how the 2020 edition of TREC-IS has addressed these dual issues by introducing a new COVID-19-specific task for evaluating generalization of existing COVID-19 annotation and system performance to this new context, applied to 11 regions across the globe. TREC-IS has also continued expanding its set of target crises, adding 29 new events and expanding the collection of event types to include explosions, fires, and general storms, making for a total of 9 event types in addition to the new COVID-19 events. Across these events, TREC-IS has made available 478,110 COVID-related messages and 282,444 crisis-related messages for participant systems to analyze, of which 14,835 COVID-related and 19,784 crisis-related messages have been manually annotated. Analyses of these new datasets and participant systems demonstrate first that both the distributions of information type and priority of information vary between general crises and COVID-19-related discussion. Secondly, despite these differences, results suggest leveraging general crisis data in the COVID-19 context improves performance over baselines. Using these results, we provide guidance on which information types appear most consistent between general crises and COVID-19.
|
Apoorva Chauhan, & Amanda Hughes. (2021). COVID-19 Named Resources on Facebook, Twitter, and Reddit. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 679–690). Blacksburg, VA (USA): Virginia Tech.
Abstract: Crisis Named Resources (CNRs) are social media accounts and pages named after a crisis event. They are created soon after an event occurs. CNRs share a lot of information around an event and are followed by many. In this study, we identify CNRs created around COVID-19 on Facebook, Twitter, and Reddit. We analyze when these resources were created, why they were created, how they were received by members of the public, and who created them. We conclude by comparing CNRs created around COVID-19 with past crisis events and discuss how CNR owners attempt to manage content and combat misinformation.
|
Jens Kersten, Jan Bongard, & Friederike Klan. (2021). Combining Supervised and Unsupervised Learning to Detect and Semantically Aggregate Crisis-Related Twitter Content. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 744–754). Blacksburg, VA (USA): Virginia Tech.
Abstract: Twitter is an immediate and almost ubiquitous platform and therefore can be a valuable source of information during disasters. Current methods for identifying and classifying crisis-related content are often based on single tweets, i.e., already known information from the past is neglected. In this paper, the combination of tweet-wise pre-trained neural networks and unsupervised semantic clustering is proposed and investigated. The intention is to (1) enhance the generalization capability of pre-trained models, (2) to be able to handle massive amounts of stream data, (3) to reduce information overload by identifying potentially crisis-related content, and (4) to obtain a semantically aggregated data representation that allows for further automated, manual and visual analyses. Latent representations of each tweet based on pre-trained sentence embedding models are used for both, clustering and tweet classification. For a fast, robust and time-continuous processing, subsequent time periods are clustered individually according to a Chinese restaurant process. Clusters without any tweet classified as crisis-related are pruned. Data aggregation over time is ensured by merging semantically similar clusters. A comparison of our hybrid method to a similar clustering approach, as well as first quantitative and qualitative results from experiments with two different labeled data sets demonstrate the great potential for crisis-related Twitter stream analyses.
|
Antone Evans Jr., Yingyuan Yang, & Sunshin Lee. (2021). Towards Predicting COVID-19 Trends: Feature Engineering on Social Media Responses. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 792–807). Blacksburg, VA (USA): Virginia Tech.
Abstract: During the course of this pandemic, the use of social media and virtual networks has been at an all-time high. Individuals have used social media to express their thoughts on matters related to this pandemic. It is difficult to predict current trends based on historic case data because trends are more connected to social activities which can lead to the spread of coronavirus. So, it's important for us to derive meaningful information from social media as it is widely used. Therefore, we grouped tweets by common keywords, found correlations between keywords and daily COVID-19 statistics and built predictive modeling. The features correlation analysis was very effective, so trends were predicted very well. A RMSE score of 0.0425504, MAE of 0.03295105 and RSQ of 0.5237014 in relation to daily deaths. In addition, we found a RMSE score of 0.07346836, MAE of 0.0491152 and RSQ 0.374529 in relation to daily cases.
|
Shivam Sharma, & Cody Buntain. (2021). An Evaluation of Twitter Datasets from Non-Pandemic Crises Applied to Regional COVID-19 Contexts. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 808–815). Blacksburg, VA (USA): Virginia Tech.
Abstract: In 2020, we have witnessed an unprecedented crisis event, the COVID-19 pandemic. Various questions arise regarding the nature of this crisis data and the impacts it would have on the existing tools. In this paper, we aim to study whether we can include pandemic-type crisis events with general non-pandemic events and hypothesize that including labeled crisis data from a variety of non-pandemic events will improve classification performance over models trained solely on pandemic events. To test our hypothesis we study the model performance for different models by performing a cross validation test on pandemic only held-out sets for two different types of training sets, one containing only pandemic data and the other a combination of pandemic and non-pandemic crisis data, and comparing the results of the two. Our results approve our hypothesis and give evidence of some crucial information propagation upon inclusion of non-pandemic crisis data to pandemic data.
|
Richard McCreadie, Cody Buntain, & Ian Soboroff. (2020). Incident Streams 2019: Actionable Insights and How to Find Them. In Amanda Hughes, Fiona McNeill, & Christopher W. Zobel (Eds.), ISCRAM 2020 Conference Proceedings – 17th International Conference on Information Systems for Crisis Response and Management (pp. 744–760). Blacksburg, VA (USA): Virginia Tech.
Abstract: The ubiquity of mobile internet-enabled devices combined with wide-spread social media use during emergencies is posing new challenges for response personnel. In particular, service operators are now expected to monitor these online channels to extract actionable insights and answer questions from the public. A lack of adequate tools makes this monitoring impractical at the scale of many emergencies. The TREC Incident Streams (TREC-IS) track drives research into solving this technology gap by bringing together academia and industry to develop techniques for extracting actionable insights from social media streams during emergencies. This paper covers the second year of TREC-IS, hosted in 2019 with two editions, 2019-A and 2019-B, contributing 12 new events and approximately 20,000 new tweets across 25 information categories, with 15 research groups participating across the world. This paper provides an overview of these new editions, actionable insights from data labelling, and the automated techniques employed by participant systems that appear most effective.
|
Jeremy Diaz, Lise St. Denis, Maxwell B. Joseph, Kylen Solvik, & Jennifer K. Balch. (2020). Classifying Twitter Users for Disaster Response: A Highly Multimodal or Simple Approach? In Amanda Hughes, Fiona McNeill, & Christopher W. Zobel (Eds.), ISCRAM 2020 Conference Proceedings – 17th International Conference on Information Systems for Crisis Response and Management (pp. 774–789). Blacksburg, VA (USA): Virginia Tech.
Abstract: We report on the development of a classifier to identify Twitter users contributing first-hand information during a disaster. Identifying such users helps social media monitoring teams identify critical information that might otherwise slip through the cracks. A parallel study (St. Denis et al., 2020) demonstrates that Twitter user filtering creates an information-rich stream of content, but the best way to approach this task is unexplored. A user's profile contains many different “modalities” of data, including numbers, text, and images. To integrate these different data types, we constructed a multimodal neural network that combines the loss function of all modalities, and we compared the results to many individual unimodal models and a decision-level fusion approach. Analysis of the results suggests that unimodal models acting on Twitter users' recent tweets are sufficient for accurate classification. We demonstrate promising classification of Twitter users for crisis response with methods that are (1) easy to implement and (2) quick to both optimize and infer.
|
Liuqing Li, & Edward A. Fox. (2020). Disaster Response Patterns across Different User Groups on Twitter: A Case Study during Hurricane Dorian. In Amanda Hughes, Fiona McNeill, & Christopher W. Zobel (Eds.), ISCRAM 2020 Conference Proceedings – 17th International Conference on Information Systems for Crisis Response and Management (pp. 838–848). Blacksburg, VA (USA): Virginia Tech.
Abstract: We conducted a case study analysis of disaster response patterns across different user groups during Hurricane Dorian in 2019. We built a tweet collection about the hurricane, covering a two week period. We divided Twitter users into two groups: brand/organization or individual. We found a significant difference in response patterns between the groups. Brand users increasingly participated as the disaster unfolded, and they posted more tweets than individual users on average. Regarding emotions, brand users posted more tweets with joy and surprise, while individual users posted more tweets with sadness. Fear was a common emotion between the two groups. Further, both groups used different types of hashtags and words in their tweets. Some distinct patterns were also discovered in their concerns on specific topics. These results suggest the value of further exploration with more tweet collections, considering the behavior of different user groups during disasters.
|
Rob Grace. (2020). Hyperlocal Toponym Usage in Storm-Related Social Media. In Amanda Hughes, Fiona McNeill, & Christopher W. Zobel (Eds.), ISCRAM 2020 Conference Proceedings – 17th International Conference on Information Systems for Crisis Response and Management (pp. 849–859). Blacksburg, VA (USA): Virginia Tech.
Abstract: Crisis responders need to locate events reported in social media messages that typically lack geographic metadata such as geotags. Toponyms, places names referenced in messages, provide another source of geographic information, however, the availability and granularity of toponyms in crisis social media remain poorly understood. This study examines toponym usage and granularity across six categories of crisis-related information posted on Twitter during a severe storm. Findings show users often include geographic information in messages describing local and remote storm events but do so rarely when discussing other topics, more often use toponyms than geotags when describing local events, and tend to include fine-grained toponyms in reports of infrastructure damage and service disruption and course-grained toponyms in other kinds of storm-related messages. These findings present requirements for hyperlocal geoparsing techniques and suggest that social media monitoring presents more immediate affordances for course-grained damage assessment than fine-grained situational awareness during a crisis.
|
Thomas Spielhofer, Anna Sophie Hahne, Christian Reuter, Marc-André Kaufhold, & Stefka Schmid. (2019). Social Media Use in Emergencies of Citizens in the United Kingdom. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: People use social media in various ways including looking for or sharing information during crises or emergencies
(e.g. floods, storms, terrorist attacks). Few studies have focused on European citizens? perceptions, and just one
has deployed a representative sample to examine this. This article presents the results of one of the first
representative studies on this topic conducted in the United Kingdom. The study shows that around a third (34%)
have used social media during an emergency and that such use is more widespread among younger people. In
contrast, the main reasons for not using social media in an emergency include technological concerns and that the
trustworthiness of social media content is doubtful. However, there is a growing trend towards increased use. The
article deduces and explores implications of these findings, including problems potentially arising with more
citizens sharing information on social media during emergencies and expecting a response.
|
Liuqing Li, & Edward A. Fox. (2019). Understanding patterns and mood changes through tweets about disasters. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: We analyzed a sample of large tweet collections gathered since 2011, to expand understanding about tweeting
patterns and emotional responses of different types of tweeters regarding disasters. We selected three examples for
each of four disaster types: school shooting, bombing, earthquake, and hurricane. For each collection, we deployed
our novel model TwiRole for user classification, and an existing deep learning model for mood detection. We
found differences in the daily tweet count patterns, between the different types of events. Likewise, there were
different average scores and patterns of moods (fear, sadness, surprise), both between types of events, and between
events of the same type. Further, regarding surprise and fear, there were differences among roles of tweeters. These
results suggest the value of further exploration as well as hypothesis testing with our hundreds of event and trend
related tweet collections, considering indications in those that reflect emotional responses to disasters.
|
Richard McCreadie, Cody Buntain, & Ian Soboroff. (2019). TREC Incident Streams: Finding Actionable Information on Social Media. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: The Text Retrieval Conference (TREC) Incident Streams track is a new initiative that aims to mature social
media-based emergency response technology. This initiative advances the state of the art in this area through an
evaluation challenge, which attracts researchers and developers from across the globe. The 2018 edition of the track
provides a standardized evaluation methodology, an ontology of emergency-relevant social media information types,
proposes a scale for information criticality, and releases a dataset containing fifteen test events and approximately
20,000 labeled tweets. Analysis of this dataset reveals a significant amount of actionable information on social
media during emergencies (> 10%). While this data is valuable for emergency response efforts, analysis of the
39 state-of-the-art systems demonstrate a performance gap in identifying this data. We therefore find the current
state-of-the-art is insufficient for emergency responders? requirements, particularly for rare actionable information
for which there is little prior training data available.
|
Shane Errol Halse, Rob Grace, Jess Kropczynski, & Andrea Tapia. (2019). Simulating real-time Twitter data from historical datasets. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: In this paper, we will discuss a system design for simulating social media data based on historical datasets. While many datasets containing data collected from social media during crisis have become publicly available, there is a lack of tools or systems can present this data on the same timeline as it was originally posted. Through the design and use of the tool discussed in this paper, we show how historical datasets can be used for algorithm testing, such as those used in machine learning, to improve the quality of the data. In addition, the use of simulated data also has its benefits in training scenarios, which would allow participants to see real, non-fabricated social media messages in the same temporal manner as found on a social media platform. Lastly, we will discuss the positive reception and future improvements suggested by 911 Public Service Answering Point (PSAP) professionals.
|
Jens Kersten, Anna Kruspe, Matti Wiegmann, & Friederike Klan. (2019). Robust filtering of crisis-related tweets. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: Social media enables fast information exchange and status reporting during crises. Filtering is usually required to
identify the small fraction of social media stream data related to events. Since deep learning has recently shown to
be a reliable approach for filtering and analyzing Twitter messages, a Convolutional Neural Network is examined for
filtering crisis-related tweets in this work. The goal is to understand how to obtain accurate and robust filtering
models and how model accuracies tend to behave in case of new events. In contrast to other works, the application
to real data streams is also investigated. Motivated by the observation that machine learning model accuracies
highly depend on the used data, a new comprehensive and balanced compilation of existing data sets is proposed.
Experimental results with this data set provide valuable insights. Preliminary results from filtering a data stream
recorded during hurricane Florence in September 2018 confirm our results.
|
Anna Kruspe, Jens Kersten, & Friederike Klan. (2019). Detecting event-related tweets by example using few-shot models. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: Social media sources can be helpful in crisis situations, but discovering relevant messages is not trivial. Methods
have so far focused on universal detection models for all kinds of crises or for certain crisis types (e.g. floods).
Event-specific models could implement a more focused search area, but collecting data and training new models for
a crisis that is already in progress is costly and may take too much time for a prompt response. As a compromise,
manually collecting a small amount of example messages is feasible. Few-shot models can generalize to unseen
classes with such a small handful of examples, and do not need be trained anew for each event. We show how
these models can be used to detect crisis-relevant tweets during new events with just 10 to 100 examples and
counterexamples. We also propose a new type of few-shot model that does not require counterexamples.
|
