Latency is defined as the time between a user’s action and the response of the system to it (Friston et al., 2016). It is known to decrease the performance (Friston et al., 2016; MacKenzie & Ware, 1993; Tochioka et al., 2019) and user experience (Caserman et al., 2019; Ng & Dietz, 2014) when interacting with computers. The decrease in performance has been noticed in simple Fitts’ Law tasks (MacKenzie & Ware, 1993), steering law tasks (Friston et al., 2016; Tochioka et al., 2019) and gaming scenarios (Claypool, 2005; Liu et al., 2021). For this reason, many researchers in Human-Computer-Interaction (HCI) tried to minimize and compensate latency. One way to achieve this is by reducing the latency in the system’s hardware (Le et al., 2017). However, faster components, which cause less latency, are more expensive (Ushirobira et al., 2016) and latency will always be present to some degree with every hardware (Tochioka et al., 2019).
Instead of reducing the latency of a systems’ hard and software components, previous work also tried to reduce latency using software that predicts the user’s actions. Researchers, as well as practitioners, experimented with several methods including time derivates, heuristic approaches, linear and polynomial extrapolation, and curve fitting to predict the user’s actions (Nancel et al., 2018). Promising results by Henze et al. (2016) suggest that using neural networks for this prediction can be an effective solution (Henze et al., 2016). Neural network predictions of user behavior have been effectively used in the context of touchscreens (Henze et al., 2016; Le et al., 2017), shooter games (Halbhuber et al., 2021), and virtual reality (Schwind et al., 2020). They were able to improve users’ performance and/or experience.
Predicting users’ mouse movement to reduce perceived latency could be expanded by predicting the position of the mouse cursor beyond the system’s latency, therefore decreasing the latency to a negative value. As some research found effects even of very low latencies on performance and user experience (Deber et al., 2015; Ng et al., 2012), and intuitively, a system that performs user actions preemptively appears more performant, it can be suspected that such a system would lead to a higher performance. However, some experts think there is a lower limit for the effects of latency (Friston et al., 2016). Moreover, users might be irritated by a prediction beyond their actions. Consequently, it remains unclear whether such a prediction can improve performance and user experience.
The goal of this thesis is to investigate how prediction beyond the system’s latency influences user performance and experience in simple tasks. For this purpose, a state-of-the-art neural network will be trained to predict the mouse cursor position at a time point that is beyond the latency of the system.
To train the neural network, mouse movement data from participants completing a simple browser-based task must be collected. Thus, a standard task known from HCI research will be implemented in JavaScript and made available through the Internet. With this data, the network architecture will be optimized, and a prediction algorithm will be trained. After training the network its predictions will be integrated into the web-based implementation of the HCI task. Thereupon, an empirical user study will be conducted to assess the influence of different prediction timeframes on user performance and experience.