We conducted a short prestudy to find if there are any notable differences between paper and digital tools while proofreading documents in order to inform our final study design.

The prestudy was planned as Within-Subject design. The first condition was the annotation medium (pen and printed document or web-based digital tool). The second condition was the text to annotate. We alternated the order of conditions between participants. We prepared two texts of about the same lengths (520 words). Both with the same amount of errors of the same type: 7 misspellings, 7 capitalization errors, 7 punctuation errors, 7 formatting errors and 3 word doubles, 31 errors in total.

7 Subjects between 20-62, without Dyslexia and without uncorrected impaired sight.

We explained to the participants that they had to proofread a document as if for a friend or colleague. They should try to find and mark every error in such a way that the friend could correct these errors. They were told to take as much time as they needed, and were given time to familiarize themselves with the features of the digital tool. After the task was done, they were asked to fill out questionnaires about the text. This procedure was repeated for the respective alternate condition.

The dependent variable was the amount of errors found. We measured the total amount of errors found and the amount of different kinds of error found. With a shortened NASA TLX we measured the experienced workload of the participants for each task.

Results indicate no significant differences between the paper and digital tools. However, even though participants were given some time to get used to the digital tool, participants took 60% longer on average using the digital tool, which affirmed findings from literature. How much of this difference could be remedied through practice is unclear at this point. Participants also found 45% more formatting errors using the digital tool, cancelling out inverse differences in spelling errors and capitalization errors found. Differences between the medium for other types of errors ranged between 0 and 13%.