Abstract: We take a human-centered design approach to develop a fully functional prototype, SMART (“Social Media Analyst Responding Tool”), informed by emergency practitioners. The prototype incorporates machine learning techniques to identify relevant information during emergencies. In this paper, we report the result of a user study to gather qualitative feedback on SMART. The evaluation results offer recommendations into the design of Social Media analysis tools for emergencies. The evaluation findings show the interest of emergency practitioners into designing such solutions; it reflects their need to not only identify relevant information but also to further perceive the outcome of their actions in social media. We found out there is a notable emphasis on the sentiment from these practitioners and social media analysis tools need to do a better job of handling negative sentiment within the emergency concept.