|   | 
Details
   web
Records
Author (up) Cody Buntain; Richard Mccreadie; Ian Soboroff
Title Incident Streams 2021 Off the Deep End: Deeper Annotations and Evaluations in Twitter Type Conference Article
Year 2022 Publication ISCRAM 2022 Conference Proceedings 19th International Conference on Information Systems for Crisis Response and Management Abbreviated Journal Iscram 2022
Volume Issue Pages 584-604
Keywords Emergency Management; Crisis Informatics; Twitter; Categorization; Priorization; Multi-Modal; Public Safety; PSCR; TREC
Abstract This paper summarizes the final year of the four-year Text REtrieval Conference Incident Streams track (TREC-IS), which has produced a large dataset comprising 136,263 annotated tweets, spanning 98 crisis events. Goals of this final year were twofold: 1) to add new categories for assessing messages, with a focus on characterizing the audience, author, and images associated with these messages, and 2) to enlarge the TREC-IS dataset with new events, with an emphasis of deeper pools for sampling. Beyond these two goals, TREC-IS has nearly doubled the number of annotated messages per event for the 26 crises introduced in 2021 and has released a new parallel dataset of 312,546 images associated with crisis content with 7,297 tweets having annotations about their embedded images. Our analyses of this new crisis data yields new insights about the context of a tweet; e.g., messages intended for a local audience and those that contain images of weather forecasts and infographics have higher than average assessments of priority but are relatively rare. Tweets containing images, however, have higher perceived priorities than tweets without images. Moving to deeper pools, while tending to lower classification performance, also does not generally impact performance rankings or alter distributions of information-types. We end this paper with a discussion of these datasets, analyses, their implications, and how they contribute both new data and insights to the broader crisis informatics community.
Address University of Maryland, College Park (UMD); University of Glasgow; National Institute of Standards and Technology (NIST)
Corporate Author Thesis
Publisher Place of Publication Tarbes, France Editor Rob Grace; Hossein Baharmand
Language English Summary Language Original Title
Series Editor Series Title Abbreviated Series Title
Series Volume Series Issue Edition
ISSN 2411-3387 ISBN 978-82-8427-099-9 Medium
Track Social Media for Crisis Management Expedition Conference
Notes Approved no
Call Number ISCRAM @ idladmin @ Serial 2441
Share this record to Facebook
 

 
Author (up) Shivam Sharma; Cody Buntain
Title Bang for your Buck: Performance Impact Across Choices in Learning Architectures for Crisis Informatics Type Conference Article
Year 2022 Publication ISCRAM 2022 Conference Proceedings 19th International Conference on Information Systems for Crisis Response and Management Abbreviated Journal Iscram 2022
Volume Issue Pages 719-736
Keywords Incident Streams; TREC; TRECIS; crisis informatics
Abstract Over the years, with the increase in social media engagement, there has been an in increase in various pipelines to analyze, classify and prioritize crisis-related data on various social media platforms. These pipelines utilize various data augmentation methods to counter imbalanced crisis data, sophisticated and off-the-shelf models for training. However, there is a lack of comprehensive study which compares these methods for the various sections of a pipeline. In this study, we split a general crisis-related pipeline into 3 major sections, namely, data augmentation, model selection, and training methodology. We compare various methods for each of these sections and then present a comprehensive evaluation of which section to prioritize based on the results from various pipelines. We compare our results against two separate tasks, information classification and priority scoring for crisis-related tweets. Our results suggest that data augmentation, in general,improves the performance. However, sophisticated, state-of-the-art language models like DeBERTa only show performance gain in information classification tasks, and models like RoBERTa tend to show a consistent performance increase over our presented baseline consisting of BERT. We also show that, though training two separate task-specific BERT models does show better performance than one BERT model with multi-task learning methodology over an imbalanced dataset, multi-task learning does improve performance for more sophisticated model like DeBERTa with a much more balanced dataset after augmentation.
Address New Jersey Institute of Technology; New Jersey Institute of Technology
Corporate Author Thesis
Publisher Place of Publication Tarbes, France Editor Rob Grace; Hossein Baharmand
Language English Summary Language Original Title
Series Editor Series Title Abbreviated Series Title
Series Volume Series Issue Edition
ISSN 2411-3387 ISBN 978-82-8427-099-9 Medium
Track Social Media for Crisis Management Expedition Conference
Notes Approved no
Call Number ISCRAM @ idladmin @ Serial 2451
Share this record to Facebook
 

 
Author (up) Shivam Sharma; Cody Buntain
Title An Evaluation of Twitter Datasets from Non-Pandemic Crises Applied to Regional COVID-19 Contexts Type Conference Article
Year 2021 Publication ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management Abbreviated Journal Iscram 2021
Volume Issue Pages 808-815
Keywords covid19, twitter, trecis, cross-validation, machine learning, transfer learning
Abstract In 2020, we have witnessed an unprecedented crisis event, the COVID-19 pandemic. Various questions arise regarding the nature of this crisis data and the impacts it would have on the existing tools. In this paper, we aim to study whether we can include pandemic-type crisis events with general non-pandemic events and hypothesize that including labeled crisis data from a variety of non-pandemic events will improve classification performance over models trained solely on pandemic events. To test our hypothesis we study the model performance for different models by performing a cross validation test on pandemic only held-out sets for two different types of training sets, one containing only pandemic data and the other a combination of pandemic and non-pandemic crisis data, and comparing the results of the two. Our results approve our hypothesis and give evidence of some crucial information propagation upon inclusion of non-pandemic crisis data to pandemic data.
Address New Jersey Institute of Technology; New Jersey Institute of Technology
Corporate Author Thesis
Publisher Virginia Tech Place of Publication Blacksburg, VA (USA) Editor Anouck Adrot; Rob Grace; Kathleen Moore; Christopher W. Zobel
Language English Summary Language English Original Title
Series Editor Series Title Abbreviated Series Title
Series Volume Series Issue Edition
ISSN 978-1-949373-61-5 ISBN Medium
Track Social Media for Disaster Response and Resilience Expedition Conference 18th International Conference on Information Systems for Crisis Response and Management
Notes cbuntain@njit.edu Approved no
Call Number ISCRAM @ idladmin @ Serial 2375
Share this record to Facebook