On February 24th we handed in the first version of our paper for peer review. We received helpful and detailed evaluations for our paper while on the other hand giving feedback to the other teams. We took the feedback to heart and are currently revising our paper in order to further enhance the quality of our work. Furthermore, we handed in a rebuttal on March 10th. Finally, we handed in the final version of our paper on March 24th. We summarize our results down below, but if you want to read into our work in full detail, you can view our paper here: paper-hartl-kulik-wittmann.pdf

With the corona virus being omnipresent many educational institutions were forced to move their presence classes to the internet using different media such as videos. Previous focused on the effect of videos and live lectures on student learning or examined additional factors like the playback speed while there is only few research considering the presentation form itself. To find out if there are any differences in content understanding we conducted two studies: an online survey among university teachers asking about their current style of video presentations and a user study with university students to compare the different video presentations with each other. Participants watched the three presentation forms: slides with voice-over, talking head and picture-in-picture created by a university lecturer and answered an exam containing fill-in-the-blank, multiple choice and free text questions about the content of the video. In the following we summarize the results of our two studies and explain how the survey influenced the design of our main study.

The results of our preliminary survey gave us some hints for carrying out the main study: considering recordings and online live lecture slides, as well as the lecturer seemed to be the most frequent shown elements while blackboards and whiteboards were presented less often in both versions. For these reasons, we decided to use the two most common elements lecturer (talking head) and slides for our main study. We also chose to investigate a hybrid style, the picture-in-picture format. In addition, we considered the results of the camera angle as well recording the lecturer from a frontal perspective.

The results of our study are showing no significant difference in learning considering the presentation style of videos. We thus support the statement made by Homer et al. (2008), who also found no significant difference in their research. In contrast to Guo et al. (2014) who found that it is better to see the lecturer our participants achieved better results when watching the slides or the picture-in-picture format while not both of them are containing the lecturer but a visualization of the spoken text. Moreover, we counted how often each presentation form proved itself over the others focusing on the percentage of correctly answered questions. This also gives some hints that the talking head style alone does not work that well. We therefore conclude that it may be helpful to see what the lecturer is talking about let it be any visual information such as texts, bullet points or pictures.

Our results also show that there is a significant difference in the percentage of correctly answered questions between the video topics functionality of a camera and focal length. An explanation for this could be that the concentration dropped during the study, as a continuous session could take up to 90 Minutes and 51 out of 72 participants did not take a break in between the videos. Another explanation for the difference in scores could be that the video topic focal length was just a bit harder to understand than the video topic functionality of a camera.

Overall, we can say that none of the tested video versions can be declared as clearly better. As Homer et al. (2008) found, the presence of the lecturer increases the cognitive effort of the students while otherwise Ilioudi et al. (2013) reported in their study that the talking head variant was more suitable for explaining more complex topics than only learning by book. So, the presence of the lecturer has its benefits, too. Keeping that in mind, we would recommend that instructors create their video content visually as well to make the learning process easier. We therefore suggest a combination of slides and the lecturer view like in the picture-in-picture format we used.

Bruce D. Homer, Jan L. Plass, and Linda Blake. 2008. The effects of video on cognitive load and social presence in multimedia-learning. 24, 3 (2008), 786–797. https://doi.org/10.1016/j.chb.2007.02.009

Philip J. Guo, Juho Kim, and Rob Rubin. 2014. How Video Production Affects Student Engagement: An Empirical Study of MOOC Videos. In In Proceedings of ACM Conference on Learning at Scale (L@S. 4–5.

Christina Ilioudi, Michail N. Giannakos, and Konstantinos Chorianopoulos. 2013. Investigating Differences among the Commonly Used Video Lecture Styles. (2013). https://doi.org/10.13140/2.1.3524.9284 Publisher: Unpublished.