After the expert interview and a consequential discussion within the group, we made it to a first concept of the pre-study. The variables we had in mind came to a concrete schedule (see the details below). The subject of the pre-study (comparison of ranking presentation: all vs pairwise) was enriched with a new psychologic dimension in the form of the “B5T”, a personality test of Dr. Lars Satow. Our advisor came up with this idea to have the possibility of discover potential correlations between personality and, for example, ranking choices. The pre-study is divided up in four main parts: A demographic questionnaire, the “B5T”, the pre-study tasks and a qualitative post-questionnaire.

The demographic questionnaire contains questions to gender (male, female, diverse), age, nationality, professional career and experience with ranking systems. The nationality is important because of the fact, that we have the opportunity of conduct our pre-study in Germany and Colombia.

The “B5T” is an official personality test of Dr. Lars Satow and captures five fundamental personality “factors” and three basic motives (requirements of: respect and performance, influence and power, security and peace). For further information (in German) see https://www.drsatow.de/tests/persoenlichkeitstest/.

In the pre-study setting, every participant has to iterate a single task in both ranking presentations. The tasks are divided up into three different categories: Colors, logos and websites, which represent the optical complexity of the tasks. Additional we differ the quantity of the items in one category in three steps: 3 – 6 – 9 (for the all-ranking-presentation, this means 3 – 15 – 36 combinations for the pairwise). These variables have been randomized (counter-balanced except the pairwise combinations and the order of items in one category) and, for easier handling, classified as profiles for each of the 18 participants. The number of participants revealed of the randomization. Those profiles have been grouped in three parts (à six members) again: One group will be tested in Colombia with a digital representation of the categories, one in Germany also with a digital representation and the last one also in Germany but with analog representation (printed paper cards). To get quantitative data the time of each task is measured.

Qualitative Questions are nearly the most important part of this pre-study. Because of the fact that the subjective questions of the content are hardly to compare between the participants, we have to find out the reasons why participants make choices and what emotions they have during the study. Therefore, we built a questionnaire to ensure that all of the group members ask the same questions in the qualitative feedback round at the end of the pre-study. The questionnaire contains questions of four categories: presentation, complexity, quantity and familiarity.

Finally, we are very excited to see the results of this pre-study and if they meet our expectations.