1.1
1
xml
info:srw/schema/1/dc-v1.1
Bang for your Buck: Performance Impact Across Choices in Learning Architectures for Crisis Informatics
Shivam Sharma
Cody Buntain
Rob Grace
Hossein Baharmand
Over the years, with the increase in social media engagement, there has been an in increase in various pipelines to analyze, classify and prioritize crisis-related data on various social media platforms. These pipelines utilize various data augmentation methods to counter imbalanced crisis data, sophisticated and off-the-shelf models for training. However, there is a lack of comprehensive study which compares these methods for the various sections of a pipeline. In this study, we split a general crisis-related pipeline into 3 major sections, namely, data augmentation, model selection, and training methodology. We compare various methods for each of these sections and then present a comprehensive evaluation of which section to prioritize based on the results from various pipelines. We compare our results against two separate tasks, information classification and priority scoring for crisis-related tweets. Our results suggest that data augmentation, in general,improves the performance. However, sophisticated, state-of-the-art language models like DeBERTa only show performance gain in information classification tasks, and models like RoBERTa tend to show a consistent performance increase over our presented baseline consisting of BERT. We also show that, though training two separate task-specific BERT models does show better performance than one BERT model with multi-task learning methodology over an imbalanced dataset, multi-task learning does improve performance for more sophisticated model like DeBERTa with a much more balanced dataset after augmentation.
urn:ISBN:978-82-8427-099-9
openurl:?ctx_ver=Z39.88-2004&rfr_id=info%3Asid%2Fidl.iscram.org%2F&genre=proceeding&title=Bang%20for%20your%20Buck%3A%20Performance%20Impact%20Across%20Choices%20in%20Learning%20Architectures%20for%20Crisis%20Informatics&stitle=Iscram%202022&issn=2411-3387&isbn=978-82-8427-099-9&date=2022&spage=719&epage=736&aulast=Shivam%20Sharma&au=Cody%20Buntain&place=Tarbes%2C%20France&sid=refbase%3AISCRAM
url:http://idl.iscram.org/show.php?record=2451
citekey:ShivamSharma+CodyBuntain2022
citation:Shivam Sharma, & Cody Buntain. (2022). Bang for your Buck: Performance Impact Across Choices in Learning Architectures for Crisis Informatics. In Rob Grace, & Hossein Baharmand (Eds.), ISCRAM 2022 Conference Proceedings – 19th International Conference on Information Systems for Crisis Response and Management (pp. 719-736). Tarbes, France.
2022
ConferencePaper
text
Incident Streams
TREC
TRECIS
crisis informatics
file:http://idl.iscram.org/files/shivamsharma/2022/2451_ShivamSharma+CodyBuntain2022.pdf
English
2411-3387
ISCRAM 2022 Conference Proceedings – 19th International Conference on Information Systems for Crisis Response and Management
2022
719
736
1