While carrying out our main study we started to write our paper. To provide a summary about the content, the abstract as well as the table of content with notes is shown in this blog entry.

In order to create an adequate table of contents which let the paper tell a continuous, conclusive story, we had several team meetings and discussion with our supervisor. This is the result:

• Introduction
• Definining Latency
• Latency and performance
• Study Design
• Results
• Discussion
• Limitations
• Conclusion
• Future Work

We decided not to implement an own chapter „Related work“ yet split it thematically: the different components of latency and their definitions and in another section studies of other researchers (MacKenzie, Claypool & Claypool). Their studies are relevant for our research since we decided to conduct our studies with modified games as well.

Study Design was the widest section of the paper. We first describe approaches for measuring end-to-end latency, as Casiez et al. provide for example. This part could be considered moving to the related work chapters, yet we decided to put it to study design in order to tell a continuous, conclusive story with our paper. Our pre study and concept for the main study get presented, as well as the variables described and the hypotheses listed. Those were the following for our research:

        * H1: Higher average latency effects lower performance while playing.
        * H1.1: Higher average latency effects lower game scores.
        * H1.2: Higher average latency effects a lower rate of successful target clicks.
        * H2: Higher relative latency variance of a base latency effects lower performance while playing.
        * H2.1: Higher relative latency variance of a base latency effects lower game scores.
        * H2.2: Higher relative latency variance of a base latency effects a lower rate of successful clicks.
        * H3: Higher average latency effects lower user experience while playing.
        * H3.1: Higher average latency effects a lower SAM-pleasure rating.
        * H3.2: Higher average latency effects a higher SAM-arousal rating.
        * H3.3: Higher average latency effects a lower SAM-dominance rating.
        * H4: Higher relative latency variance of a base latency effects lower user experience while playing.
        * H4.1: Higher latency variance effects a lower SAM-pleasure rating.
        * H4.2: Higher latency variance effects a higher SAM-arousal rating.
        * H4.3: Higher latency variance effects a lower SAM-dominance rating.


 

As suggested in the abstract, our study was accompanied by several limitations. First of all, measuring the user experience. As discussed in previous blog entries, we had problems chosing adequate ways measuring UX. We tried two different methods, video annotations and the SAM questionnaire but none of them showed results or just for very high base latency differences. The main limitation was the high base latency (139.167 ms +- 5.590ms).

Conclusion„: Our study takes a step towards better understanding and quantify- ing the effects of latency and latency variance on user experience and performance. For this purpose, we conducted a preliminary study and a more extensive main study to evaluate the influence of latency and latency variance on performance and user experience in the context of gaming. A simple game prototype was developed, which can be configured with custom base latencies and latency variances. We could provide evidence that increasing base latency decreases performance in games containing pointing on moving tar- gets significantly, leading to more off target clicks and lower game scores. Also, we were able to show that a high constant latency (210ms artificially added to the native system latency of 139.167ms) reduces the pleasure while playing. We could not prove that latency variance affects user experience in games, which contain pointing and clicking moving targets, but we would like to allude to the limitations of our prototype, which only provided a narrow range of absolute latency variances, which may not have been distinct enough.“

       * Replicating the study providing a lower end-to-end latency than ours
       * Further exploring of users’ “calibration” to the current system latency
       * Exploring the learning effect on player performance and user experience

After each of us submitted two reviews for the papers of our fellow students, we received 5 reviews for our paper plus a meta review by our supervisor Andreas Schmid. We read them carefully and summarized them in a shared google document in different categories (e.g. formal issues, wrong naming, missing target group,…). We discussed which of those issues are really helpful and appropriate and based on this we wrote our rebuttal letter.

At the moment, we transfer the accepted critics to our paper. We are looking forward to present our research and work of this semester in front of the class at March, 29th.