During our analysis and also through a meeting with our supervisor, we noticed that some phrases in the schemas are not well chosen and one category was arranged in the false supercategory (3D-structured light). Here are our final schemas for the classification of AR technologies after Display, Interface and Tracking.

Further we decided to classify the area of an AR application too and determined the following categories:

• Assembly and Inspection
  (Dey et al., 2016; Papagiannakis, Singh, & Magnenat-Thalmann, 2008; Van Krevelen and Poelman, 2010)
• Education
  (Billinghurst, Clark, & Lee, 2015; Carmigniani et al., 2011; Dey et al., 2016; Papagiannakis et al., 2008; Van Krevelen and Poelman, 2010)
• Culture, Arts and Architecture
  (Billinghurst, Clark, & Lee, 2015;Papagiannakis, Singh, & Magnenat-Thalmann, 2008)
• Collaboration
  (Dey et al., 2016; Papagiannakis, Singh, & Magnenat-Thalmann, 2008; Van Krevelen and Poelman, 2010)
• Entertainment and Gaming
  (Billinghurst, Clark, & Lee, 2015; Carmigniani et al., 2011; Dey et al., 2016; Papagiannakis et al., 2008; Van Krevelen and Poelman, 2010)
• Interaction
  (Papagiannakis, Singh, & Magnenat-Thalmann, 2008)
• Medical
  (Carmigniani et al., 2011; Dey et al., 2016; Van Krevelen and Poelman, 2010)
• Navigation and Driving
  (Dey et al., 2016; Papagiannakis, Singh, & Magnenat-Thalmann, 2008; Van Krevelen and Poelman, 2010)
• Advertising and commercial
  (Billinghurst, Clark, & Lee, 2015; Carmigniani et al., 2011; Van Krevelen and Poelman, 2010)
• Tourism and exploration
  (Dey et al. ,2016)
• Science
  (Billinghurst, Clark, & Lee, 2015)

Regarding the classification of evaluation methods we made a lot of changes and improvements.

• We don’t classify only the evaluation method anymore. We added items that are in our paper classification table already (we presented in our last blog). For example informations about the participants, the evaluation subject etc.
• The classification is not separated into three layers anymore, rather the layout and organisation is adapted from the classification schema for the AR technologie.
• Before the update our schema was mainly based on the survey Duenser et al. (2008) conducted. Now we include three additional references, including two AR surveys.

Also this schema will be adapted by us during our analysis, so we will add the exact method (last item by “evaluation method”) in the correct category. Examples: We add every particular questionnaire as subcategories under this item ( evaluation method → subjective measurements → questionnaire). At the moment we found following questionnaires that were conducted:

• Likert-Scales
• matching question (left) to the answer (right)
• open-ended questions
• Social Presence in Gaming Questionnaire (SPGQ)
• System Usability Scale (SUS)
• user experience questionnaire
• Attrakdiff questionnaire
• before-after comparison: z.B: lernverhalten
• UTAUT2 model
• Game Experience Questionnaire (GEQ)
• Task Load Index (TLX)
  

We add the subcategory “informal discussion” under the supercategory “informal evaluations”.

If we found further categories for the area of an AR application, for the technologies or the evaluation methods, during our paper analysis we will add the additional category in our schemas and collect them. We will present them in our paper as a lesson learned aspect and will discuss them in our analysis of the results of the state-of-the-art paper analysis. Maybe it is possible to draw conclusions when we compare our results to the already conducted AR related surveys.

Azuma, R. T. (1997). A Survey of Augmented Reality. Presence: Teleoper. Virtual Environ., 6(4), 355–385. https://doi.org/10.1162/pres.1997.6.4.355

Billinghurst, M., Clark, A., & Lee, G. (2015). A Survey of Augmented Reality. Foundations and Trends® in Human–Computer Interaction, 8(2–3), 73–272. https://doi.org/10.1561/1100000049

Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damiani, E., & Ivkovic, M. (2011). Augmented reality technologies, systems and applications. Multimedia Tools and Applications, 51(1), 341–377. https://doi.org/10.1007/s11042-010-0660-6

Milgram, P., Takemura, H., Utsumi, A., & Kishino, F. (1995). Augmented reality: a class of displays on the reality-virtuality continuum. In H. Das (Hrsg.) (S. 282–292). Gehalten auf der Photonics for Industrial Applications, Boston, MA. https://doi.org/10.1117/12.197321

Papagiannakis, G., Singh, G., & Magnenat-Thalmann, N. (2008). A survey of mobile and wireless technologies for augmented reality systems. Computer Animation and Virtual Worlds, 19(1), 3–22. https://doi.org/10.1002/cav.221

Schmalstieg, D., & Höllerer, T. (2016). Augmented Reality: Principles and Practice. Addison-Wesley Professional. Abgerufen von https://proquest.tech.safaribooksonline.de/9780133153217

Van Krevelen, R., & Poelman, R. (2010). A Survey of Augmented Reality Technologies, Applications and Limitations. International Journal of Virtual Reality (ISSN 1081-1451), 9, 1.

Billinghurst, M., Clark, A., & Lee, G. (2015). A Survey of Augmented Reality. Foundations and Trends® in Human–Computer Interaction, 8(2–3), 73–272. https://doi.org/10.1561/1100000049

Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damiani, E., & Ivkovic, M. (2011). Augmented reality technologies, systems and applications. Multimedia Tools and Applications, 51(1), 341–377. https://doi.org/10.1007/s11042-010-0660-6

Dey, A., Billinghurst, M., Lindeman, R. W., & Swan II, J. E. (2016). A Systematic Review of Usability Studies in AugmentedReality between 2005 and 2014. In 2016IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) (S. 49–50). Merida,Yucatan, Mexico: IEEE.https://doi.org/10.1109/ISMAR-Adjunct.2016.0036

Papagiannakis, G., Singh, G., & Magnenat-Thalmann, N. (2008). A survey of mobile and wireless technologies for augmented reality systems. Computer Animation and Virtual Worlds, 19(1), 3–22. https://doi.org/10.1002/cav.221

Van Krevelen, R., & Poelman, R. (2010). A Survey of Augmented Reality Technologies, Applications and Limitations. International Journal of Virtual Reality (ISSN 1081-1451), 9, 1.

Edward, J., Ii, S., & Gabbard, J. L. (2005).Survey of User-Based Experimentation in augmented Reality. In In 1st International Conference on VirtualReality, Las Vegas

Dünser,A., Grasset, R., & Billinghurst, M. (2008). A survey of evaluation techniques used in augmented reality studies. In ACM SIGGRAPH ASIA 2008 courses on - SIGGRAPH Asia’08 (S. 1–27). Singapore: ACMPress.https://doi.org/10.1145/1508044.1508049

Dünser,A., & Billinghurst, M. (2011). Evaluating Augmented Reality Systems. In B.Furht (Hrsg.), Handbook of Augmented Reality (S. 289–307).New York, NY: Springer New York.https://doi.org/10.1007/978-1-4614-0064-6_13

Dey, A., Billinghurst, M., Lindeman, R. W., & Swan II, J. E. (2016). A Systematic Review of Usability Studies in Augmented Reality between 2005 and 2014. In 2016IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) (S. 49–50). Merida,Yucatan, Mexico: IEEE.https://doi.org/10.1109/ISMAR-Adjunct.2016.0036