Methods and User Study (2018-12-13)

Tagged as: blog, mixed methods, Contextual Inquiry, expert interview, KLM
Group: F Our third bloq entry is introducing the different methods we are going to use for our user study. Contextual Inquiry, Expert Interviews and the KLM methods will be explained. Also a short introduction about our Preliminary Study will be shown.

Contextual Inquiry:

A Contextual Inquiry is an analysis method of the contextual design, which focusses on the typical working life of the subject (Beyer & Holtzblatt, 1997, S.37f). The structure of a contextual inquiry is usually as followed (Beyer & Holtzblatt, 1997, S.64):

  • Short introduction about topic and reason for this study
  • Main Part: Everyday working life of the subject will be observed and
    discussed
  • Conclusion: Summary and results

The inquiry gives the task administrator an extensive image of the user context, his work and what he is doing and is especially collecting tacit knowledge (Collins, 2001, Kapitel 7 – tacit knowledge), which are steps the subjects does unconsciously. Therefor the subject is describing his everyday working life and uses the “thinking aloud” principle (Someren, Barnard & Sandberg, 1994, Kapitel 1), where he is doing his normal routine, but is explaining loudly what and why he is doing a certain move. Beyer & Holtzblatt defined 4 essential processes (Beyer & Holtzblatt, 1997):

1.Context

Most of the times at the office of the subject. He should feel comfy and should not change his daily routine only because he is observed. The test supervisor is observing the subject, but does not interrupt him unless the subject has questions or something is not clear. Also the subject should retelling specific events he thinks they might be relevant for the supervisor (Beyer & Holtz-blatt, 1997, S.47ff).

2.Partnership

Subject and test supervisor are communicating with each other and it will be switched between observation and discussion (Beyer & Holtzblatt, 1997, S.51ff).

3.Interpretation

The interpretation of certain moves can be different for every person. To avoid false information, the test supervisor should discuss with the subject and the subject should remind the supervisor, If the supervisor misunderstood something (Beyer & Holtz-blatt, 1997, S.56ff).

4.Focus

The focus of the CI is on relevant tasks for the working environment. The conversation needs to distinguish between what falls within the projects scope and stuff that are outside of it.

This method is a good way for us to get data about „WHY“ people would use file manager. In the CI, we can investigate things we usually would not be able to capture with a normal interview, which could give us leads to the reason of using an obscure file manager over a normal GUI based one.

KLM – Keystroke Level Model (Card, Moran & Newell, 1980):

The idea behind it: deconstruct tasks down to keystroke-level operators:

  • Keystroke
  • Pointing with mouse
  • Button press or release on mouse
  • Mental act
  • Hand switching between keyboard and mouse
  • Waiting for system response

KLM: Standard Values nach (Card, Moran, Newell (1980) and Kieras (2001))

Every step needs to be considered, basically writing down the Steps in Operators and sum them up, that will be the time needed to complete the task.

We use KLM as a method for getting the time in which the test subjects completed each task. This helps us comparing the results and measuring completion time.

Expert interview

Expert interviews are used in order to gather qualitative data.

The user base of obscure file managers is much smaller than the one for regular, standard file managers. Therefore trying to get quantitative data is not feasable for this user group.

As we only found one expert with obscure file managers so far, our preliminary study will include a smaller size of interview sample. It will be conducted as a semi structured interview and will be used to gather information about how experts use obscure file managers.

We want to find out, which tasks experts use, how they work with their file managers, if they use unique shortcuts and if yes, why and for what.

During the interview we would like to see the use of the file manager in action and get a feeling about which tasks it is used for. We also want to give a few small tasks from our task korpus the subject has to perform, to see how they differ compared to regular gui based file managers.

Preliminary Study

  • Trial studies to avoid conducting a whole study, which does not bring the results it should
  • Preliminary studies to develop different aspects of the trial procedures or to collect data to facilitate the planning and conduct of the
    trial [Retrieved from https://www.ncbi.nlm.nih.gov/books/NBK305518/]
  • Tests of the full trial procedures on a small sample of potential participants to make sure, in so far as is possible, that any problems with the conduct of the trial will be identified, so that procedures can be changed before the full trial starts [Retrieved from
    https://www.ncbi.nlm.nih.gov/books/NBK305518/]
  • Other preliminary studies are helpful to refine the design of specific methods for use in the process and/or impact the evaluation within the main trial, and to evaluate their acceptability, feasibility, and cost

General task korpus:

Common Tasks:

  1. Copy
  2. Move
  3. Edit
  4. Delete
  5. Search
  6. Open File
  7. Switch window
  8. Sort
  9. Create
  10. Navigate to folders
  11. Change folder view

Advanced Tasks: For the Advanced Tasks we would like to conduct our Contextual Inquiry first. First tasks were added to our korpus, but this list is far from completed. Since we are mostly normal user aswell, we need the opinion of an expert to be able to complement our task korpus.

  1. Create own shortcuts
  2. Common tasks only with keyboard (shortcuts)
  3. Open a text file in the manager and modify it

Structure of our preliminary Study:

Introduction

  • Demopgrahical data
  • Experience about media in general
  • Experience about file manager

User Tasks

  • Show the user the different file manager and ask him which one he likes the most (Esthethic)
  • Standard Explorer tasks (Windows, Linux, Mac)
  • Commandline Tasks (Windows, Linux, Max)
  • Column based file manager tasks
  • Advanced file manager tasks

Conclusion

  • User Experience (Survey)
  • Which file manager did he like the most? (After usage)

Sources

Beyer, H., & Holtzblatt, K. (1997). Contextual design: defining customer-centered systems. Elsevier.

Collins, H. M. (2001). What is tacit knowledge. In The practice turn in contemporary theory, 107-119.

Someren, M. V., Barnard, Y. F., & Sandberg, J. A. (1994). The think aloud method: a practical approach to modelling cognitive processes. Academic Press.

Card, Moran, Newell (1980): “The keystroke-level model for user performance with interactive systems”6, Communications of the ACM 23(7), ACM, New York