| Record |
| Author |
Tiberiu Sosea; Iustin Sirbu; Cornelia Caragea; Doina Caragea; Traian Rebedea |
| Title |
Using the Image-Text Relationship to Improve Multimodal Disaster Tweet Classification |
Type |
Conference Article |
| Year |
2021 |
Publication |
ISCRAM 2021 Conference Proceedings – 18th International Conference on Information Systems for Crisis Response and Management |
Abbreviated Journal |
Iscram 2021 |
| Volume |
|
Issue |
|
Pages |
691-704 |
| Keywords |
Multi-modal disaster tweet classification, Image-text coherence relationship prediction, ViLBERT |
| Abstract |
In this paper, we show that the text-image relationship of disaster tweets can be used to improve the classification of tweets from emergency situations. To this end, we introduce DisRel, a dataset which contains 4,600 multimodal tweets, collected during the disasters that hit the USA in 2017, and manually annotated with coherence image-text relationships, such as Similar and Complementary. We explore multiple models to detect these relationships and perform a comprehensive analysis into the robustness of these methods. Based on these models, we build a simple feature augmentation approach that can leverage the text-image relationship. We test our methods on 2 tasks in CrisisMMD: Humanitarian Categories and Damage Assessment, and observe an increase in the performance of the relationship-aware methods. |
| Address |
University of Illinois at Chicago; University Politehnica of Bucharest; University of Illinois at Chicago; Kansas State University; University Politehnica of Bucharest |
| Corporate Author |
|
Thesis |
|
| Publisher |
Virginia Tech |
Place of Publication |
Blacksburg, VA (USA) |
Editor |
Anouck Adrot; Rob Grace; Kathleen Moore; Christopher W. Zobel |
| Language |
English |
Summary Language |
English |
Original Title |
|
| Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
| Series Volume |
|
Series Issue |
|
Edition |
|
| ISSN |
978-1-949373-61-5 |
ISBN |
|
Medium |
|
| Track |
Social Media for Disaster Response and Resilience |
Expedition |
|
Conference |
18th International Conference on Information Systems for Crisis Response and Management |
| Notes |
tsosea2@uic.edu |
Approved |
no |
| Call Number |
ISCRAM @ idladmin @ |
Serial |
2365 |
| Share this record to Facebook |
