Hand gestures are widely regarded as an intuitive means of interaction, and are increasingly being researched as a naturally intuitive form of interaction for a wide range of applications (Vuletic et al., 2019; Gentile et al., 2019; Modanwal und Sarawadekar, 2018; Tscharn et al., 2016). Numerous studies have focused on possible interactions using free hand gestures (Buchmann et al., 2004; Rehman et al., 2020; Le und Kim, 2017; Bai et al., 2014; Shi et al., 2023; Boudjelthia et al., 2018; Yu et al., 2022). However, intuitive proposals by users might not be their favourite (Morris et al., 2014). Hoffmann et al. (2019) conducted an elicitation study in which participants were asked to propose orders by voice, mid-air gestures and a touch display to control smart homes. They had participants rate three modalities, and found mid-air gestures received mainly negative comments. Reasons cited by participants included (mid-air gestures) being counterintuitive, complex to use, need for long term memory and long periods of operation. The problem, that users' gesture suggestions are often influenced by their experience with previous interfaces and technologies, was known as legacy bias. (Morris et al., 2014). But legacy bias is not inherently negative (Köpsel & Bubalo, 2015), and appropriate feedback could help with improving user experience (Schönauer et al., 2015). In a study by Lindeman et al. (1999), four different interaction techniques in immersive virtual environments were compared, two conditions with passive haptic feedback and two without. The passive-haptic feedback conditions required the subject to hold a physical paddle in their non-dominant hand, while the other conditions only allowed the subject to hold the handle of a paddle. The subjects performed docking and selecting tasks with their dominant hand. The study results indicate that holding a physical paddle in virtual reality helps subjects to perform faster and more precise.
As technology advances, gestures, a natural way of interacting, are more supported by technology (Tscharn et al., 2016). Gestures also used in Mixed Reality to freely interact with the virtual elements (Buchmann et al., 2004; Rehman et al., 2020; Le und Kim, 2017; Bai et al., 2014; Shi et al., 2023; Yu et al., 2022). AR could be used to train new employees in industry (Eder et al., 2020; Barsom et al., 2016), and it is not far-fetched to imagine its integration into everyday life, such as with the Vision Pro. Regardless of which system is used for gesture interaction, the user's hands may not always be empty in actual use. Our research question is whether holding objects in hand could affect the gestures proposed by users.
The focus of this work is to look at hand gestures when holding objects and to see what gestures the user would suggest to interact with the system when holding a tool. In addition, this work would also investigate how holding a tool affects the user's behaviour.
Basic knowledge of Unity development