Asmelash Teka Hadgu, Sallam Abualhaija, & Claudia Niederée. (2019). Real-time Adaptive Crawler for Tracking Unfolding Events on Twitter. In Z. Franco, J. J. González, & J. H. Canós (Eds.), Proceedings of the 16th International Conference on Information Systems for Crisis Response And Management. Valencia, Spain: Iscram.
Abstract: When a major event such as a crisis situation occurs, people post messages on social media sites such as Twitter, in
order to exchange information or to share emotions. These posts can provide useful information to raise situation
awareness and support decision making, e.g., by aid organizations. In this paper, we propose a novel method for
social media crawling, which exploits a Bayesian inference framework to keep track of keyword changes over time
and uses a counter-stream to gauge the inclusion of noise and irrelevant information. In addition, we present a
framework to evaluate real-time adaptive social search algorithms in a reproducible manner, which relies on a
semi-automated approach for ground-truth construction. We show that our method outperforms previous methods
for very large scale events.