Replicating Studies

Replication of research should be seen as one of the cornerstones of good and honest science (Wilson et al., 2013). Replications confirm the results, strengthen research, and ensure that results are based on solid foundations. However, studies indicate that an alarming number of published study results can not or will not be reproduced. In the field of human-computer interaction (HCI) and computer science, replication is performed even less frequently than in others. We discuss two main questions: (i) What are the reasons for this undesirable effect? (ii) How can this be remedied? We focus on the impact of replication on science and show why it is important to describe the challenges and give some guidelines that should be followed to enable valuable scientific replication. We replicated relevant parts of a study from Harrison et al. (Harrison et al., 2010) on the subjective perception of the duration of various progress bars and applied our elaborated guidelines to the conduct of the study. We compared five different behaviors of progress bars, with different frequencies alternating colors from light blue to blue. In study 1, the slow increasing behaviour in relation to the fast increasing behaviour of the progress bars has a statistically significant effect. The three behaviours slow increasing, fast increasing and constant are generally perceived faster than the slow decreasing and fast decreasing behaviour. Study 2 indicates that lower-height progress bars are perceived faster than larger ones. When comparing progress bars and progress circles, participants more often perceived the progress bar faster. The challenges and implementation problems are discussed and the results of both studies are analyzed in detail. For replication, it is important that the authors provide as much and accurate data as possible from the original study.

Members: Marlene Bauer, Florian Habler

Keywords: Replication, Human-Computer Interaction, User Studies, Progress Bars