Towards More Effective User Gesture Elicitation: Considering System-Sided Detectability

Elicitation studies allow for a user-centered creation of interaction gestures. Typically, in those studies, users are asked what gesture they would perform to change a system from status A to status B, for example, what gesture they would perform to skip from the current song playing (status A) on their Apple watch to the next song (status B). Repeating this procedure over a pool of participants and different tasks reveals patterns and similarities. As designers and researchers, we can use these patterns and similarities to confidently develop gesture sets for a specific interaction paradigm that is most likely suitable to a large part of potential users.

Elicitation studies have to account for a large variety of variables, such as the use context, the kind of computer and interaction scenario (VR, AR, MR, Smartphone, Mid-Are, etc..), the user, design space limitations, and more. However, one aspect primarily overlooked in previous work using elicitation studies is gesture detectability. Since elicitation studies establish gestures based on user input, the system-sided detectability (the system here refers to the computer that needs to register and interpret the user gesture) is largely not accounted for. Thus, elicitation studies may produce gestures that are not detectable by the system, for example, if the motion is obfuscated by the user's own hand.

To summarize, while elicitation studies are a great tool to establish gesture sets for novel and existing interaction paradigms, they do not consider the system-sided detectability of the developed gesture set. Currently, it is unclear how we can design better elicitation studies and better gestures.

The goal of this work is to evaluate how we can account for and quantify system-sided detectability when designing elicited user gestures. There are several different ways this problem can be approached, which we will discuss in future meetings. However, the general structure of this master thesis is as follows:

1. Conduct a literature review about elicitation studies and gesture detection approaches.
2. Define the study context (VR, AR, Smartphone, type of gesture, etc.).
3. Conduct study.
4. Either design a post-hoc detectability system or integrate the level of detectability directly into 3. (Needs to be discussed).
5. Analyze and conclude.

We will discuss all aspects of this work in detail in an initial meeting. Particularly, points 2. and 3. are highly adaptable to the student's preference (what medium do you want to work with, what are your prior experiences?). However, given the scope of this work, it is only suitable for Master's students.

see above

None, but some prior experience may be helpful to adapt the work to the student's interest.

Will be provided.