Tobias Drey, member of the research group human-computer interaction, has successfully defended his Ph.D. project of the title Towards Mixed Reality in Education: Overcoming Challenges of Teaching, Learning, and Authoring. The jury consisted of Prof. Dr Enrico Rukzio (Institute of Media Informatics, Ulm University), Prof. Dr Florian Alt (Universität der Bundeswehr Munich), Prof. Dr Tina Seufert, Prof. Dr Timo Ropinski (both Ulm University), and Prof. Dr Matthias Tichy (head of the commission and minutes).
Abstract:
Digitization is ubiquitous in our everyday life and has already revolutionized education. Especially the rise of Mixed Reality (MR) during the last decade has shown multiple new opportunities for innovative digitized education to increase learning outcomes in the cognitive and psychomotor domains. However, previous study results are ambiguous on how MR environments should be designed and how existing education theory, including multimedia design principles, can be transferred to MR. The obstacles such research has to overcome are further interdisciplinary, as interaction theory driven by Human-Computer Interaction (HCI) strongly influences the MR learning environment’s design and, consequently, learning outcomes.
To ensure such interdisciplinary research, I created the interdisciplinary TriDactIX framework based on Norman’s interaction cycle as well as the model-view-controller pattern (HCI theory) and the didactic triangle (education theory). It emphasizes the role of the digital system in digitized education and the interaction with the respective users, namely learners and teachers. This framework guided this thesis and the five core publications to identify and overcome current teaching, learning, and authoring challenges in MR.
The first core publication investigated the design requirements for collaborative Virtual Reality (VR) environments in the context of pair-learning by comparing a symmetric setup with an asymmetric one. The study’s findings reveal that users (N=46) favored the symmetric system as it facilitated more straightforward communication. This led to the formulation of six guidelines for designing collaborative MR learning applications.
The second core publication, SpARklingPaper, is an Augmented Reality (AR) system that integrates the genuine haptic feedback of traditional pen and paper with digital education’s immediate and personalized feedback to improve children’s handwriting training. In a user study with children and their parents (N=26), it was compared to a state-of-the-art digital tablet, and they acknowledged its superior haptic feedback in supporting handwriting training. The results show that SpARklingPaper can serve as a reference system design.
As constant teaching and learning progress is always aspired and should not be impeded, core publication three investigated the undesirable "being stuck" state, specifically for adaptive learning environments, by conducting a Systematic Literature Review (SLR). It defines this state and a related continuum, including triggers, indicators, and countermeasures based on the individual characteristics of the users. This research will support developing MR applications that consider individual characteristics to personalize teaching, learning, and authoring, thereby enhancing learning outcomes.
The fourth core publication, VRSketchIn, investigated a solution to the function vs. complexity trade-off for 3D content creation by introducing a VR in-situ authoring system. It utilizes an interaction concept based on unconstrained 3D mid-air and constrained 2D surface-supported drawing. VRSketchIn was assessed with six participants (N=6) and it demonstrated that it allows non-expert users, such as teachers and learners, to generate content that would typically necessitate expert tools as well as skills.
Core publication five investigated how individual characteristics, namely spatial abilities, influence teaching, learning, and authoring in MR by investigating their effects on object manipulation task performance in VR. The user studies’ findings (N=66) show that considering these spatial abilities enhances task performance. The developed seven guidelines will assist in creating customized teaching, learning, and authoring experiences in MR.
The core publications and their findings show that the TriDactIX framework can be used to guide HCI and education theory-based interdisciplinary research in a structured way by being able to identify novel research gaps and research questions, including the definition of independent and dependent variables. The joint discussion of the core publications based on the TriDactIX framework in this thesis shows how it can advance both HCI and education theory. Therefore, this discussion provides interlinked interaction, system, and learning unit design principles for researchers and practitioners. This thesis also proposes how HCI factors such as usability could possibly be used to extend education theory (see Cognitive Affective Model of Immersive Learning (CAMIL)). Therefore, it concludes that researchers should use the TriDactIX framework as guidance in their future works.
Concluding, this dissertation provides contributions for teaching, learning, and authoring in MR regarding (1) collaborative learning, (2) handwriting skill training, (3) the individual "being stuck" state, (4) in-situ content authoring, (5) the influence of individual spatial abilities on object manipulation performance, and (6) the TriDactIX framework.
