User-Centered Development of a Framework for Prototyping Interaction Techniques

While the field of Human-Computer Interaction originated from the study of interactions between computers and foundational devices like the keyboard and mouse [1], advancements have introduced a diverse range of newly developed sensors, empowering researchers to capture implicit user actions, thus significantly contributing to research and education [2]. Sensors play a pivotal role for prototyping within the domain of Human-Computer Interaction, facilitating the exploration of novel interactive computer artifacts [3]. As technology has progressed, sensors have facilitated the rapid proliferation of Internet of Things (IoT) devices — distinctly identifiable objects interconnected via the Internet and equipped with a spectrum of sensors [4]. This has led to a surge in the number of such devices, which is expected to persist [5,6]. The advent of increasingly powerful computational resources, enhanced connectivity, and smaller form factors has not only facilitated the development of cutting-edge technologies such as wearable devices for health tracking [7], but has also become an integral component of computer science education [8]. Consequently, ensuring an engaging experience with IoT devices is crucial for both students and researchers.

The process of working with sensor data involves the retrieval of data from microcontrollers such as Arduino, Raspberry Pi, or BBC micro boards. Subsequent tasks encompass the visualization of these data points, their integration into applications, or training machine learning algorithms. However, before utilising sensor data, students and researchers must overcome several challenges associated with IoT devices. These include the necessity to gain an in-depth understanding of the functionality of different sensors and associated APIs, as well as the initial configuration of an integrated development environment (IDE) for IoT application development and deployment software. Additionally, IoT devices lack comprehensive documentation regarding the handling of sensor data [9,10]. These challenges are time-consuming, error-prone, and impede students' interaction with sensor data and the development of new interaction techniques.

The objective of this study is to develop a prototyping framework for prototyping interaction techniques, grounded in user-centered design principles and specifically tailored to make use of common IoT devices. The framework aims to enhance usability for both students and researchers within the domain of Human-Computer Interaction (HCI).

Starting with a comprehensive literature review, we delve into the domain of IoT, end-user programming, and frameworks [e.g. 11,12] showcasing similar approaches. Subsequently, an extensive prestudy is conducted, seeking the requirements and suggestions of the target audience. This phase encompasses an exploration of issues and improvement suggestions through focus groups comprising students proficient in IoT devices. Additionally, expert interviews delve deeper into challenges with sensor data, project development and learning new APIs. Moreover, an analysis of the software and hardware employed in interaction technique conferences held in 2023 is undertaken to understand researchers' utilization patterns of IoT devices and their corresponding implementation methodologies.

The insights gained from the preceding stages are used to build a prototype which provides an abstraction for a prototyping framework tailored for sensor-related research and education. Following to the prototype's concept, a reference implementation is created using a convenient hardware and software environment, enabling the evaluation of our approach via a hands-on task employing heuristics for API usability [13]. The findings derived from this work are then combined and discussed, providing insights on enhancements and future extensions of the framework.

• Literature research
• Conduct prestudy:
  • Systematic interaction conference analysis
  • Focus Groups with media informatic students proficient in working with IoT devices
  • Expert interviews reviewing current findings and exploring their expertise in the field of interaction techniques, prototyping and computer science education
• Build an abstract prototype based on literature and prestudy
• Develop a reference implementation
• Heuristic evaluation with hands-on-tasks
• Integrate Feedback
• Write paper

• Experience working with IoT Devices and related technology

[1] Card, S. K. (2018). The psychology of human-computer interaction. Crc Press.

[2] Schmidt, A. (2000). Implicit human computer interaction through context. Personal technologies, _4_, 191-199.

[3] Houde, S., & Hill, C. (1997). What do prototypes prototype?. In Handbook of human-computer interaction (pp. 367-381). North-Holland.

[4] Ashton, K. (2009). That ‘internet of things’ thing. RFID journal, 22(7), 97-114.

[5] Macenski, S., Foote, T., Gerkey, B., Lalancette, C., & Woodall, W. (2022). Robot Operating System 2: Design, architecture, and uses in the wild. Science robotics, _7_(66), eabm6074.

[6] The Future of Jobs Report 2023. (2023). World Economic Forum. Retrieved 17 May 2024, from https://www.weforum.org/publications/the-future-of-jobs-report-2023/

[7] Guk, K., Han, G., Lim, J., Jeong, K., Kang, T., Lim, E. K., & Jung, J. (2019). Evolution of wearable devices with real-time disease monitoring for personalized healthcare. Nanomaterials, _9_(6), 813.

[8] Hodges, S., Sentance, S., Finney, J., & Ball, T. (2020). Physical computing: A key element of modern computer science education. Computer, 53(4), 20-30.

[9] Robillard, M. P. (2009). What makes APIs hard to learn? Answers from developers. IEEE software, 26(6), 27-34.

[10] Corno, F., De Russis, L., & Sáenz, J. P. (2019). On the challenges novice programmers experience in developing IoT systems: A survey. Journal of Systems and Software, 157, 110389.

[11] Kaltenbrunner, M., & Echtler, F. (2018). The TUIO 2.0 protocol: An abstraction framework for tangible interactive surfaces. Proceedings of the ACM on Human-Computer Interaction, _2_(EICS), 1-35.

[12] Web of Things (WoT) Architecture. (2020). Retrieved 17 May 2024, from https://www.w3.org/TR/2020/REC-wot-architecture-20200409/

[13] Myers, B. A., & Stylos, J. (2016). Improving API usability. Communications of the ACM, 59(6), 62-69.