Ly Dinh, Sumeet Kulkarni, Pingjing Yang, & Jana Diesner. (2023). Reliability of Methods for Extracting Collaboration Networks from Crisis-related Situational Reports and Tweets. In V. L. Thomas J. Huggins (Ed.), Proceedings of the ISCRAM Asia Pacific Conference 2022 (pp. 181–195). Palmerston North, New Zealand: Massey Unversity.
Abstract: Assessing the effectiveness of crisis response is key to improving preparedness and adapting policies. One method for response evaluation is reviewing actual response activities and interactions. Response reports are often available in the form of natural language text data. Analyzing a large number of such reports requires automated or semi automated solutions. To improve the trustworthiness of methods for this purpose, we empirically validate the reliability of three relation extraction methods that we used to construct interorganizational collaboration networks by comparing them against human-annotated ground truth (crisis-specific situational reports and tweets). For entity extraction, we find that using a combination of two off-the-shelf methods (FlairNLP and SpaCy) is optimal for situational reports data and one method (SpaCy) for tweets data. For relation extraction, we find that a heuristics-based model that we built by leveraging word co-occurrence and deep and shallow syntax as features and training it on domain-specific text data outperforms two state-of-the-art relation extraction models (Stanford OpenIE and OneIE) that were pre-trained on general domain data. We also find that situational reports, on average, contain less entities and relations than tweets, but the extracted networks are more closely related to collaboration activities mentioned in the ground truth. As it is widely known that general domain tools might need adjustment to perform accurately in specific domains, we did not expect the tested off-the-shelf tools to perform highly accurately. Our point is to rather identify what accuracy one could reasonably expect when leveraging available resources as-is for domain specific work (in this case, crisis informatics), what errors (in terms of false positives and false negatives) to expect, and how to account for that.
|
|
Congcong Wang, Paul Nulty, & David Lillis. (2021). Transformer-based Multi-task Learning for Disaster Tweet Categorisation. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 705–718). Blacksburg, VA (USA): Virginia Tech.
Abstract: Social media has enabled people to circulate information in a timely fashion, thus motivating people to post messages seeking help during crisis situations. These messages can contribute to the situational awareness of emergency responders, who have a need for them to be categorised according to information types (i.e. the type of aid services the messages are requesting). We introduce a transformer-based multi-task learning (MTL) technique for classifying information types and estimating the priority of these messages. We evaluate the effectiveness of our approach with a variety of metrics by submitting runs to the TREC Incident Streams (IS) track: a research initiative specifically designed for disaster tweet classification and prioritisation. The results demonstrate that our approach achieves competitive performance in most metrics as compared to other participating runs. Subsequently, we find that an ensemble approach combining disparate transformer encoders within our approach helps to improve the overall effectiveness to a significant extent, achieving state-of-the-art performance in almost every metric. We make the code publicly available so that our work can be reproduced and used as a baseline for the community for future work in this domain.
|
|
Yitong Li, Duoduo Liao, Jundong Li, & Wenying Ji. (2021). Automated Generation of Disaster Response Networks through Information Extraction. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 431–438). Blacksburg, VA (USA): Virginia Tech.
Abstract: Following a disaster, maintaining and restoring community lifelines require collective efforts from various stakeholders. Aiming at reducing the efforts associated with generating stakeholder collaboration networks (SCNs), this research proposes a systematic approach for reliably extracting stakeholder collaboration information from texts and automatically generating SCNs. In the proposed approach, stakeholders and their interactions are automatically extracted from texts through a natural language processing technique--Named Entity Recognition. Once extracted, the collaboration information is stored into structured datasets to automate the generation of SCNs. A case study on stakeholder collaboration in response to Hurricane Harvey is used to demonstrate the feasibility and applicability of the proposed approach. Overall, the proposed approach achieves the reliable and automated generation of SCNs from texts, which largely reduces practitioners' interpretation loads and eases the data collection process.
|
|
Xiaojing Guo, Xinzhi Wang, Luyao Kou, & Hui Zhang. (2021). A Question Answering System Applied to Disasters. In Anouck Adrot, Rob Grace, Kathleen Moore, & Christopher W. Zobel (Eds.), ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management (pp. 2–16). Blacksburg, VA (USA): Virginia Tech.
Abstract: In emergency management, identifying disaster information accurately and promptly out of numerous documents like news articles, announcements, and reports is important for decision makers to accomplish their mission efficiently. This paper studies the application of the question answering system which can automatically locate answers in the documents by natural language processing to improve the efficiency and accuracy of disaster knowledge extraction. Firstly, an improved question answering model was constructed based on the advantages of the existing neural network models. Secondly, the English question answering dataset pertinent to disasters and the Chinese question answering dataset were constructed. Finally, the improved neural network model was trained on the datasets and tested by calculating the F1 and EM scores which indicated that a higher question answering accuracy was achieved. The improved system has a deeper understanding of the semantic information and can be used to construct the disaster knowledge graph.
|
|
Hussein Mouzannar, Yara Rizk, & Mariette Awad. (2018). Damage Identification in Social Media Posts using Multimodal Deep Learning. In Kees Boersma, & Brian Tomaszeski (Eds.), ISCRAM 2018 Conference Proceedings – 15th International Conference on Information Systems for Crisis Response and Management (pp. 529–543). Rochester, NY (USA): Rochester Institute of Technology.
Abstract: Social media has recently become a digital lifeline used to relay information and locate survivors in disaster situations. Currently, officials and volunteers scour social media for any valuable information; however, this approach is implausible as millions of posts are shared by the minute. Our goal is to automate actionable information extraction from social media posts to efficiently direct relief resources. Identifying damage and human casualties allows first responders to efficiently allocate resources and save as many lives as possible. Since social media posts contain text, images and videos, we propose a multimodal deep learning framework to identify damage related information. This framework combines multiple pretrained unimodal convolutional neural networks that extract features from raw text and images independently, before a final classifier labels the posts based on both modalities. Experiments on a home-grown database of labeled social media posts showed promising results and validated the merits of the proposed approach.
|
|