Replicating Studies

Replication of research should be seen as one of the cornerstones of good and honest science (Wilson et al., 2013). Replications confirm the results, strengthen research, and ensure that results are based on solid foundations. However, studies indicate that an alarming number of published study results can not or will not be reproduced. In the field of human-computer interaction (HCI) and computer science, replication is performed even less frequently than in others. We discuss two main questions: (i) What are the reasons for this undesirable effect? (ii) How can this be remedied? We focus on the impact of replication on science and show why it is important to describe the challenges and give some guidelines that should be followed to enable valuable scientific replication. We replicated relevant parts of a study from Harrison et al. (Harrison et al., 2010) on the subjective perception of the duration of various progress bars and applied our elaborated guidelines to the conduct of the study. We compared five different behaviors of progress bars, with different frequencies alternating colors from light blue to blue. In study 1, the slow increasing behaviour in relation to the fast increasing behaviour of the progress bars has a statistically significant effect. The three behaviours slow increasing, fast increasing and constant are generally perceived faster than the slow decreasing and fast decreasing behaviour. Study 2 indicates that lower-height progress bars are perceived faster than larger ones. When comparing progress bars and progress circles, participants more often perceived the progress bar faster. The challenges and implementation problems are discussed and the results of both studies are analyzed in detail. For replication, it is important that the authors provide as much and accurate data as possible from the original study.

Members: Marlene Bauer, Florian Habler

Keywords: Replication, Human-Computer Interaction, User Studies, Progress Bars

A replication study attempts to validate the findings of a prior piece of research. By doing so, that prior research is confirmed as being both accurate and broadly applicable, since the replication study typically changes one or more variables of the original study, such as sample population, industry sector, etc. From the perspective of contributing to scientific research, replication studies are important for the continued progress of science. Without validation, how do future researchers know whether to build on the findings of that original work? Source: https://www.enago.com/academy/importance-of-replication-studies/

Goals

  • Specify reasons for replicating studies
  • Gather information about the replication process
  • Analyze when it is useful to replicate
  • Define quality criteria for studies
  • Replication of a relevant study

Updates

Carrying out the study (2019-02-07)

Last week, we had 24 participants carry out our study. (more...)


Final Study Design (2019-01-31)

The study design is finally implemented and can be used for the tests. (more...)


Further preparation and study goals (2019-01-10)

We have agreed to set up the prototype based on an HTML document and perform the functions of the loading bars with Javascript. We also worked out the final study design and divided it into 3 studies after consultation with the supervisor. (more...)


Preparation of study prototyp (2019-01-03)

We are currently implementing the progress bar prototyp for the study. (more...)


Studies breakdown and topic preparation (2018-12-28)

Finalization of the study search and topic preparation (more...)


Searching for a study to replicate (2018-12-18)

Further research to find a replicable study. The main focus lies on the practicability, the user groups, the study design and the needed equipment. (more...)


Snapchat Demographic Comparison (2018-12-06)

Further examination of a replicable Snapchat user study (more...)


Status update (2018-11-29)

Examination of a possibly replicable study about Snapchat users (more...)


Goals and status (2018-11-22)

Definition of goals and status (more...)


Further Resources