After analyzing the results from the preliminary study, we decided to conduct the main study to determine whether ranking was an alternative to likert scales using appropriate data as ground truth. For this purpose we decided on 4 mouse tracking speeds and determined the ground truth data of the participants with the help of a Fitts´ law test. The demographic questionnaire contains questions to gender (male, female, various), age, nationality, professional career, experience with ranking systems and the use of computers. After the 4 tests at the selected speeds, the participants assessment of their own performance is then asked. One half of the respondents will first answer the 4 questions for each individual mouse tracking speed in the Likert scale system and then perform the ranking. The other half will reverse the order, first ranking and then likert scales.
Since the preliminary study revealed that the ranking was very well accepted, we decided to implement this in the main study. The ranking was therefore carried out with 4 paper squares, which the participants can sort in the desired order.