Tobias Drey, member of the research group human-computer interaction, discusses his PhD thesis project.
Abstract:
The speed of technological progress is constantly increasing, bringing huge changes in daily life.
In education, however, this revolution is still lagging far behind, and teaching is still mostly done in the traditional analog way. Among the many new technologies, Mixed Reality (MR) systems stand out and promise multiple benefits for education, such as immersive visualization and interaction as well as high motivation, offering entirely new ways of education. In educational research, the didactic triangle is a common model to represent the relationship between students, teachers, and the content being taught, resulting in a teaching relationship bet ween students and teachers, a learning relationship between students and the content, and an authoring relationship between the teachers and the content. I have used the didactic triangle in my research and extended it based on Norman’s interaction model to combine the didactic view with human-computer interaction. This combined model represents the basis of my work on how to overcome the challenges of MR in education.
First, I discuss the technical challenges of MR systems for teaching and learning. This is done by my work on how collaborative systems should be designed for virtual reality (VR) pair-learning. I compared a symmetric collaboration (all users use similar devices, e.g., head- mounted displays (HMDs)) with an asymmetric collaboration (users use different devices, e.g., tablets or HMDs), resulting in design guidelines. I further show how analog devices such as pen and paper, which are still used for graphomotor handwriting training due to their beneficial genuine haptic feedback, can be digitized with augmented reality (AR). Therefore, I developed a tablet-based occlusion- free AR handwriting training prototype. Second, I discuss how MR systems can be designed to foster learning outcome. I used the classic multimedia design principle signaling and investigated how this can be used to guide attention in VR pair-learning. I also investigated how instant feedback for AR handwriting training is perceived. To ensure continuous learning progress, I have developed a stuck theory, including a continuum proposing when and how to adapt virtual environments according to the learners' needs automatically. Third, I show that it is important for a broad distribution of MR in education that educational content can be authored by everyone as easily as currently poss ible for slides using, e.g., PowerPoint. Therefore, I show the possibilities of a VR sketching application combining unconstrained 3D mid-air with constrained 2D surface- based sketching using a 6 degree of freedom (DoF) -tracked pen and a 6DoF -tracked tablet as input devices for in-situ content creation and why it is usable by everyone. I further show how considering personal capabilities for the user interface, such as individual spatial abilities, can be used to increase the task performance in MR, including the presentation of guidelines. The task performance can be further improved by using expert strategy tutorials.
I conclude by setting these works into the context of my extended didactic triangle model, including implications for future works
We invite all interested persons to listen and discuss his research.
