Regresar

YarnSense: Automated Data Storytelling for Multimodal Learning Analytics

Abstract:

Professional development and training often require students to reflect on their performance, especially recalling the mistakes they have made in safe training environments, but these can occur in rapidly evolving and busy environments where key actions are often missed. Promisingly, rapid improvements in wearable sensing technologies are opening up new opportunities to capture large amounts of multimodal behaviour data that can serve as evidence to support student reflection about their performance. However, while some preliminary research has highlighted the potential of analysing such data to identify interesting patterns, less work has focused on the problem of automatically communicating meaningful and contextualised data and insights to end-users. Based on the notion of data storytelling as a means of extracting actionable insights from data, this paper presents YarnSense, an architecture to automatically generate data stories with the intention of supporting student reflection and learning. YarnSense maps low-level sensor data to the pedagogical intentions of teachers, bringing human instructors into the data analysis loop. We illustrate this approach with a reference implementation of the system and an in-the-wild study in the context of immersive simulation in healthcare.