Expert Interviews (2019-01-09)

Tagged as: expert, interviews, interview, remote, usability, testing tool
Group: A The gathered information from the conducted interviews, concerning usability testing tools, with special focus on remote testing tools.

Parallel to the market analysis, expert interviews were conducted. The interview focused on two main aspects, the first part thrives to gather information about Human-Computer-Interaction Research in common, which tools are used, which functions are needed and their good or bad properties, while the second part focuses on moderated, synchronous, remote usability testing tools. It was asked, which tools were used and the reason behind it. Other questions focused on the positive and negative features of the programs used. The types of data and their quality were also addressed. Invitations for the interview were sent to nine usability experts, seven from an academic and the other two from an economic background. Recruiting participants for the interviews has proved to be difficult, while the two participants situated in the industry, we received four declines from the academics. The one arguments were that their research focus on other kinds of data, which aren’t gathered by remote usability tools, like the OBS-Tool, for example eye-tracking data. Another one was that testing prototypes, for new forms of interactions, are hard to analyse with standard devices and that for these evaluations own tools are developed. This tells us that moderated remote usability tools aren’t that present in academic research, because often other kinds of data is needed and researchers will develop their own tools to gather these. From the conducted interviews we extracted that synchronous remote usability tools, respectively the videos, are often used to quantify emotions, in other words to capture emotions and reactions of participants in order to have an example for a problem. For this kind of tests the broadcasted video often wasn’t the only data gathered, in some cases it was accompanied by questionnaires, task-completion-rate, task-completion-time and click-paths. It can be said, that it’s rate if only the video is gathered, this might be useful in order to test prototypes or early builds of software, in order to gather first impressions on the product in early in the development cycle. The program Morae was mentioned by the interviewees. It is a chargeable program from the company TechSmith, it was used to record the screens and the reactions of participants. One negative aspect mentioned, was the uncertainty in which parts of the tool were working. There were many different parts to be checked, like audio and video transmission, if problems occurred they couldn’t say where they came from and how to remove these. Another problem occurred when they tried to depart from the standard setup, consisting of a monitor for the researcher and one for the participant. To setup the program for a test was also considered as to complex for the kind of tests they were trying to carry out. Due to scheduling reasons, not all interviews were ready to be analysed in time, possible new insights will be added in another entry.