Designing a Multimodal Analytics System to Improve Emergency Response Training

Abstract

The role of high-fidelity simulation-based training is critical for preparing first responders to perform effectively in emergency response and firefighting environments. However, training simulation instructors currently rely on direct observation and radio-based audio communication methods to observe behaviors and evaluate interactions of trainees to provide feedback to trainees during debriefing sessions. Such human-driven evaluative methods, while valuable, lose fidelity of information for learning due to working memory limitations and the mind’s inability to capture all relevant behavioral information in realtime and to provide relevant detailed information that is easily interpretable to instructors. Thus, there is a need and the potential to explore alternative data collection and processing methods leveraging advanced wireless technologies to attempt to enhance the effectiveness of simulation training process for emergency response, to ultimately help save lives and better prepare our communities building emergency and disaster resilience. In this paper, we present a multimodal streaming analytics system to support learning by leveraging a user-centered design approach in consultation with a regional fire and rescue training academy. This conceptualized system provides real-time collection, visualization, and analysis of heterogeneous data streams including location sensing using Internet of Things (IoT) devices, audio communication, video observations, as well as social media and 911 call log streams. We describe the associated design challenges and lessons learned from the initial prototyping activities to strive toward enhancing the situational awareness and learning of emergency response personnel and leadership instructors with iterative design cycles at the regional fire and rescue training academy.