Evaluation (2019-03-21)

Tagged as: evaluation
Group: A Evaluation of the Regensburger Usability Platform


  • SUS Score: 68,6 → offers acceptable usability (according to Bangor, Kortum and Miller (2009)
  • Feedback from our participants:
  • Positive
    • Clear Structure
    • Reduced Functionalities
    • Easy to Learn
    • Feels reliable, unambiguous and conclusive
  • Major Usability Issues:
    • OBS-Setup has to be supported by clear instructions
    • Browser-Back should be supported
    • Chat: Missing notifications for incoming chat messages
    • Chat: Automated scrolling with the latest chat history
    • Annotations: Workflow of the annotations should be changed: add timestamp first and entry afterwards
      1. Annotations: Connect annotation with current task
      2. Creating a test: test should be saved before adding single tasks
      3. Creating a test: tests and tasks should be editable
      4. Creating a test: It should be possible to start tests right after creating them

      - Most importantly: Missing Feedback

    • Feedback is missing for almost all implemented functions (saving/deleting a test/task, adding an annotation, subject completes task…)


  • SUS-Score 64,1 → according to Bangor et al. (2009) Morae offers a usability between being okay and good
  • Feedback from our participants, major issues (will not be shown in detail as the main purpose of the paper is the evaluation of the RUP):
    • Outdated look and feel of the tool
    • Unstructured, low predictability
    • Many functions, not structured conclusively (p.ex. nested tabs, hierarchy and placement of elements, highlighting of elements)
    • At least two screens have to be used by the test supervisor to not get confused during the test
    • No notifications for incoming chat messages

Aaron Bangor, Phil Kortum, and James Miller. 2009. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. J. Usability Stud. 4 (04 2009), 114–123. John Brooke. 1995. SUS: A quick and dirty usability scale. Usability Eval. Ind. 189 (11 1995).