Kiran Zahra, Rahul Deb Das, Frank O. Ostermann, & Ross S. Purves. (2022). Towards an Automated Information Extraction Model from Twitter Threads during Disasters. In Rob Grace, & Hossein Baharmand (Eds.), ISCRAM 2022 Conference Proceedings – 19th International Conference on Information Systems for Crisis Response and Management (pp. 637–653). Tarbes, France.
Abstract: Social media plays a vital role as a communication source during large-scale disasters. The unstructured and informal nature of such short individual posts makes it difficult to extract useful information, often due to a lack of additional context. The potential of social media threads– sequences of posts– has not been explored as a source of adding context and more information to the initiating post. In this research, we explored Twitter threads as an information source and developed an information extraction model capable of extracting relevant information from threads posted during disasters. We used a crowdsourcing platform to determine whether a thread adds more information to the initial tweet and defined disaster-related information present in these threads into six themes– event reporting, location, time, intensity, casualty and damage reports, and help calls. For these themes, we created the respective thematic lexicons from WordNet. Moreover, we developed and compared four information extraction models trained on GloVe, word2vec, bag-of-words, and thematic bag-of-words to extract and summarize the most critical information from the threads. Our results reveal that 70 percent of all threads add information to the initiating post for various disaster-related themes. Furthermore, the thematic bag-of-words information extraction model outperforms the other algorithms and models for preserving the highest number of disaster-related themes.
|
McCreadie, R., & Buntain, C. (2023). CrisisFACTS: Buidling and Evaluating Crisis Timelines. In Jaziar Radianti, Ioannis Dokas, Nicolas Lalone, & Deepak Khazanchi (Eds.), Proceedings of the 20th International ISCRAM Conference (pp. 320–339). Omaha, USA: University of Nebraska at Omaha.
Abstract: Between 2018 and 2021, the Incident Streams track (TREC-IS) developed standard approaches for classifying information types and criticality of tweets during crises. While successful in producing substantial collections of labeled data, TREC-IS as a data challenge had several limitations: It only evaluated information at type-level rather than what was reported; it only used Twitter data; and it lacked measures of redundancy in system output. This paper introduces Crisis Facts and Cross-Stream Temporal Summarization (CrisisFACTS), a new data challenge piloted in 2022 and developed to address these limitations. The CrisisFACTS framework recasts TREC-IS into an event-summarization task using multiple disaster-relevant data streams and a new fact-based evaluation scheme, allowing the community to assess state-of-the-art methods for summarizing disaster events Results from CrisisFACTS in 2022 include a new test-collection comprising human-generated disaster summaries along with multi-platform datasets of social media, crisis reports and news coverage for major crisis events.
|
Nasik Muhammad Nafi, Avishek Bose, Sarthak Khanal, Doina Caragea, & William H. Hsu. (2020). Abstractive Text Summarization of Disaster-Related Documents. In Amanda Hughes, Fiona McNeill, & Christopher W. Zobel (Eds.), ISCRAM 2020 Conference Proceedings – 17th International Conference on Information Systems for Crisis Response and Management (pp. 881–892). Blacksburg, VA (USA): Virginia Tech.
Abstract: Abstractive summarization is intended to capture key information from the full text of documents. In the application domain of disaster and crisis event reporting, key information includes disaster effects, cause, and severity. While some researches regarding information extraction in the disaster domain have focused on keyphrase extraction from short disaster-related texts like tweets, there is hardly any work that attempts abstractive summarization of long disaster-related documents. Following the recent success of Reinforcement Learning (RL) in other domains, we leverage an RL-based state-of-the-art approach in abstractive summarization to summarize disaster-related documents. RL enables an agent to find an optimal policy by maximizing some reward. We design a novel hybrid reward metric for the disaster domain by combining \underline{Vec}tor Similarity and \underline{Lex}icon Matching (\textit{VecLex}) to maximize the relevance of the abstract to the source document while focusing on disaster-related keywords. We evaluate the model on a disaster-related subset of a CNN/Daily Mail dataset consisting of 104,913 documents. The results show that our approach produces more informative summaries and achieves higher \textit{VecLex} scores compared to the baseline.
|
Pereira, J., Fidalgo, R., Lotufo, R., & Nogueira, R. (2023). Crisis Event Social Media Summarization with GPT-3 and Neural Reranking. In Jaziar Radianti, Ioannis Dokas, Nicolas Lalone, & Deepak Khazanchi (Eds.), Proceedings of the 20th International ISCRAM Conference (pp. 371–384). Omaha, USA: University of Nebraska at Omaha.
Abstract: Managing emergency events, such as natural disasters, requires management teams to have an up-to-date view of what is happening throughout the event. In this paper, we demonstrate how a method using a state-of-the-art open-sourced search engine and a large language model can generate accurate and comprehensive summaries by retrieving information from social media and online news sources. We evaluated our method on the TREC CrisisFACTS challenge dataset using automatic summarization metrics (e.g., Rouge-2 and BERTScore) and the manual evaluation performed by the challenge organizers. Our approach is the best in comprehensiveness despite presenting a high redundancy ratio in the generated summaries. In addition, since all pipeline components are few-shot, there is no need to collect training data, allowing us to deploy the system rapidly. Code is available at https://github.com/neuralmind-ai/visconde-crisis-summarization.
|