At first, it was difficult to adjust all thoughts and ideas for a ground understanding within the group - because of the topic's „sponginess“, we had to intensify the meeting discussions to eliminate superficial ideas that lead to misunderstandings. At this point, every concept was discussed and questioned in detail, e.g. the procedure of the pre-study and main-study, objects of investigation, sequences and randomization, the collection of data (qualitative, quantitative).

By asking for expert advice, we first introduced our broad understanding and conception of the scientific work we are working on. Soon, it became clear that we lost sight of the wood for the trees because of information overload. The expert interview helped us to formulate specific and valid research questions and evaluate previous thoughts, e.g. the subjects of the pre-study (comparison of ranking presentation: all vs pairwise, etc.) and the actual main-study's topics (real websites that are ranked via Schulze method and that are also evaluated via standardized questionnaire (SUS or SEQ) in comparison, etc.). Also, we gained useful tips for data collection (further research literature, collecting qualitative data via interview, within or between subject, using website-genre for both studies to capture data on the same level, etc.). The interview helped us to formulate our research destination and reinforce our corporate understanding of the topic.

Now - with these thoughts in mind, we are planning and designing the procedure of the current pre-study (as well as a rough schedule for the actual main-study). Still, there are many variables (e.g. number and types of participants, number and types of categories (colours, logos, website-main-pages, etc.), number of topics (usability, execution time, complexity, etc.), randomization within or between categories or participants, measuring quantitative data like task-execution-time for navigating websites, questionnaires, comparison and potential findings, biasing, experimental setup) that need to be considered. Therefore, our plan finally needs to be discussed and prepared in detail to launch our first pre-study trial.