The Usability of Interaction Methods in Augmented Reality

Augmented reality (AR) is a type of virtual reality in which computer generated holograms can be seen overlaid on the real world. Pokemon Go is a popular game that uses AR. In my research I compare the usability of three distinct head gaze-based selection methods in an AR game for children (aged 10-13): voice recognition, gesture, and physical button (clicker). Currently there are not established standards for how children can effectively interact with holograms (computer-generated objects in AR). Prior work on AR applications in STEM (science, technology, engineering and math) education has focused on how it compares with non-AR methods such as having students observing microorganisms under a microscope or learning about physics through experimenting with how physical objects interact with air currents. Less is known about how children respond to different interaction modalities in AR. I developed a hidden object AR game on the Microsoft HoloLens designed to test the efficacy of the three interaction methods. I found significant differences in input errors between the voice and gesture conditions. I also found significant differences in elapsed tutorial time between the voice and clicker conditions. I also explore the usability of three different interaction methods on a head-mounted Augmented Reality (AR) device with children aged 9-11 years. The three interaction methods I look at are voice recognition, gesture recognition, and controller. I conducted a within-subjects study using a 3D selection task performed by children with a Microsoft HoloLens. I measured elapsed time during the completion of the selection tasks. Also, I collected usability and fatigue measures using the System Usability Scale and the OMNI RPE (Ratings of Perceived Exertion) scale. I found significant differences between voice and controller for time, fatigue and usability. I also found significant differences between gesture and controller for time, fatigue and usability. I conducted a third study looking at usability from a different point of view. In this study, I developed a novel observation protocol to explore children's use of augmented reality. My observation protocol (Observation Protocol for use of Augmented Reality by Children, or OPARC) covers positive and negative emotional reactions and behaviors as well as signs of persistence and resilience as they use novel technology such as AR. I found significant differences in scores from my protocol when used during the observation of a participant using the Microsoft HoloLens using different interfaces. In particular, I found that the negative emotions OPARC subgroup score differed significantly between the gesture and controller conditions and between the voice and controller conditions. I also found that the resilience OPARC subgroup score differed significantly between the gesture and controller interfaces and between the voice and gesture interfaces. I hope to apply the results of these studies to improve the usability of augmented reality educational tools for children in the future.