06 - Strategy for the search and the qualitative analysis of the collected chat-logs (2020-12-07)

Tagged as: blog, interaction, chat, chat-log, qualitative analysis, analysis
Group: J_20/21 An overview over how we want to analyze the qualitative data that was gathered from the chat-logs of online livestream lectures.

Documenting the search strategy for online livestream chat-logs

  • We have been searching for online lectures (no exercises, no seminars) with a live chat on YouTube and Twitch.
  • We searched for lectures in German since the lecture we are going to hold for our research is also being held in German.
  • Searching methods include searching for the terms „online vorlesung“, „vorlesung“, „live vorlesung“.
  • We picked lectures from different Universities to gain a broader, more accurate and more representative sample.
  • We also picked only one lecture from ongoing courses, preferably a lecture that appeared around the same time as the others in our database and that have roughly the same length (around 1:30 hours if possible).
  • Picking only that one lecture prevents one course from being over represented for all others just because there would be more videos for it.
  • Topics of the lectures with a chat (8 in total) were: „Informatik(4)“, „Industrielle Informationstechnik(1)“, „Mathematik(1)“, „Medizin(1)“, „Computergraphik und VR(1)“.

Searching on Twitch

  • On Twitch we also searched in the „Science and Technology“ category for other fitting lectures.
  • After thoroughly searching at the time on Twitch, these were the only ones we could find, some from the past were possibly deleted since Twitch deletes VODs after a certain amount of time without an appropriate account.
  • The chat was then gathered with the twitch-chatlog tool from https://github.com/freaktechnik/twitch-chatlog.

Searching on YouTube

  • On YouTube, there is no real way to search for previously streamed videos and a lot of videos had their chat turned off.
  • Out of the 13 unique online lectures (again we only counted lectures from different courses and different universities) that we found that were streamed on YouTube, 8 had the chat turned off (62%), which makes it hard to find them on YouTube as well
  • 4 out of the YouTube videos that had chat enabled did not have a single message posted in the chat, 1 other video had only one message in it.
  • Reasons for this are: two of the lectures (and possibly a third one as well, it was not specified in the lecture) were also streamed on Zoom and students asked questions there, one lecture used their own communication tool called tweedback and one lecture had students present in the room that asked questions.

Qualitative Coding

  • We use grounded theory for the chat-logs to understand the behavior of the interaction in livestream chats and to specify our research question, approach and next steps based on our findings
  • Gathered all chat-logs in an Excel file in different tables.
  • Looking at every message posted by a user during the lecture.
  • Exploring the data message by message and applying open coding to each message.
  • We noted when we felt another lable was more appropriate (mostly less broad and more specific since the specification happens with axial coding anyway).

Next Steps

  • After we analyze the first chat-log, we apply axial coding to the first sample to gain a first set of categories
  • We then go through the other seven chat-logs that were present and adjust the labels if necessary
  • At the end we apply selective coding to find our core categories