The social side of robots and AI. Exploring video-based method for the analysis of interaction with robots and artificial intelligence – August 2024

Enrolment deadline: October 14

Enrolment requirements:

No paper will be required. However, we will need an abstract from every participant before the course starts to get an overview over the topics and be able to relate to their projects during the course. Participation will also require preparing and hold a presentation during the course.

Description:
This PhD course prepares students to explore video-based methods for the analysis and design of peoples’ interactions with robots or artificial intelligence (AI) in social settings.

Social and collaborative robots as well as AI systems are increasingly leaving factory shop floors and research laboratories and enter the everyday life and routines of various people in private, public, institutional, and organizational settings. Robots and AI are integrated in diverse contexts including family interactions, education, teamwork, public transport, and hospital settings. In difference to controlled laboratory environments, the technologies increasingly enter the complexity and “messiness” of real-world contexts. They meet a diversity of people engaging and interacting in various forms and for various purposes with the technology. Traditionally, laboratory research uses quantitative methods often focusing on dyadic task-based interaction, with one user and one robotic or AI system solving one specific task. However, to understand what happens when people engage with robots and AI in real-world settings, we need to open our perspective to the specific social context, the diversity of people that engage with the technologies and the various forms in which people engage/disengage, interact or collaborate with the technology as a single user, in groups or with other participation forms or roles. Therefore, we need to turn to qualitative methods that can deal with the complexity and dynamics of interaction with robots and AI in real world settings.  

The course introduces video-based methods for analyzing and designing robots or AI for real-world settings, combining video/ethnography and co-creation tools from participatory and interaction design. We will focus on methodological issues concerning 1) Describing practices using ethnomethodology, video ethnography and analysis, 2) Intervening in practices using enactments, bodystorming, and voice-overs, 3) Reflecting on how these practices inform our understanding of real-world practices and concrete guidelines and how this in turn can inform the design of robots and AI, thus facilitating interdisciplinary dialogue and collaboration.

Learning Objectives:

The students will gain knowledge and practical experience in:

a)      Planning and conducting video ethnographic studies, including data collection, ethics, transcription, and video analysis.

b)      Preparation and facilitation of collaborative data sessions and co-creation workshops, using techniques for rapid prototyping, including doing enactments, bodystorming and vocal sketching.

c)      Facilitating and navigating interdisciplinary collaborations by using techniques that can bridge both fields.

Teaching methods:

The course will be divided into two parts. The first part will be taught through lectures that introduce methods and workshops in which the students gain firsthand practical experience with these methods. In the second part of the course, the students will relate and apply the acquired knowledge to their own PhD-projects and present their results for discussion at the end of the course.

For practical and pedagogical reasons, the exercises in the first part of this course will particularly focus on video-based methods exploring the impact of robots on social practices in public settings and explore how a deeper understanding of these practices can inform design. Public spaces are easy to access for students during exercises in the workshops and the course lectures have the relevant data to supplement the workshops. In the second part of the course, we will encourage students to apply the methods to their own PhD projects, which may include robots and AI in any social settings.

Target group

This course is directed to an interdisciplinary audience looking for

a)      students of social science and humanities aiming to understand meaning-making processes in social settings in which robots or AI are or should be embedded,

b)      design students who are interested in exploring material and social aspects when designing robots or AI for interactions in real world settings, and

c)      students of robotics, HRI, HCI who are programming or building robots and AI and are interested in learning techniques for grounding their development in a deeper understanding of human practices and human-centred design processes.  

To summarize, the course is for every PhD student who is studying, designing or developing human-robot or human-AI interaction in any social setting. Both Danish and foreign students are welcome. The course will be taught in English.

 

Program outline:

Day 1: Describing

• Overview of the course & welcome (0.5hr)
• Introduction Lecture to the topic of observation for design (video observation, transcription, and video analysis (1,5 hrs)
• Workshops on observation, transcription, and video analysis (5 hrs)
• Q&A (0,5 hrs)

Day 2: Intervening

• Introduction Lecture video-based and participatory methods for rapid prototyping with focus on co-creation methods like enactment and bodystorming (1,5)
• Workshops testing different methods of enacting human robot interaction (5 hrs)
• Open Q&A session (0,5)

Day 3:  Describing and intervening in own PhD projects

• Broadening the perspective to other application scenarios & bridging analysis and design in a mini-project: Dive into particular themes of observing, analyzing enacting and informing/designing in regard to your own project (lecture and discussion 3 hrs)
• Afternoon: group or individual work relating the knowledge to the students’ own projects (5hrs)

Day 4: Reflecting

• Finalizing mini-project work and individual / group presentation preparation (4 hrs)
• Students’ presentations (2 hrs)

Final Discussions and evaluation (1hrs)

Organizer:
The Doctoral Programme in Communication and Psychology
  
Lecturers:

Dr. Antonia L. Krummheuer, Associate Professor for Qualitative Methods and Technology Studies, Institute for Communication and Psychology, member of the HRI Lab at Aalborg University and the steering committee of Aalborg Robotic Challenge, is organizing and teaching on the course.

Dr. Hannah R. M. Pelikan, postdoc in the AI in Motion group, Department of Culture and Society, Linköping University, developed several techniques for bridging video ethnography and interaction design. She is part of the program committee for the HRI 2024 conference and co-editor of the special issue on Sound in HRI at the ACM Transactions on Human-Robot Interaction.
 

Key literature (I am very sorry; I noted the wrong application deadline for the PhD course. We are still working on the list. We will also remember to split it up in mandatory and secondary literature.):

Mandatory

Interaction analysis (methods)

1. Heath, C., Hindmarsh, J., & Luff, P. (2010). Video in Qualitative Research: Analysing Social Interaction in Everyday Life. Sage.
2. Hannah Pelikan. 2023. Transcribing human–robot interaction: Methodological implications of participating machines. In P. Haddington, T. Eilittä, A. Kamunen, L. Kohonen-Aho, T. Oittinen, I. Rautiainen, & A. Vatanen (Eds.), Ethnomethodological Conversation Analysis in Motion: Emerging Methods and New Technologies (1st ed.). Routledge. https://doi.org/10.4324/9781003424888

Interaction with Robots

1. Alač, M., J. Movellan & F. Tanaka, 2011: When a Robot Is Social: Spatial Arrangements and Multimodal Semiotic Engagement in the Practice of Social Robotics. Social Studies of Science, 41(6): 893–926. https://doi.org/0.1177/0306312711420565.
2. Dautenhahn K. Some Brief Thoughts on the Past and Future of Human-Robot Interaction. ACMTrans. Hum.-Robot Interact. 2018;7(1). Available from: https://doi.org/10.1145/3209769.
3. Hornecker, E., Krummheuer, A. L., Bischof, A., & Rehm, M. (2022). Beyond dyadic HRI: building robots for society. interactions, 29(3), 48–53. https://dl.acm.org/doi/10.1145/3526119

Interaction Design

1. Brandt, E., Binder, T., & Sanders, E. B.-N. (2012). Tools and techniques: Ways to engage telling, making and enacting. I J. Simonsen & Toni. Robertson (Red.), Routledge International Handbook of Participatory Design (s. 145–181). Routledge.
2. Krummheuer, A. L. (2023). Practice-based robotics: How sociology can inform the development of social robots. I F. Muhle (red.), Soziale Robotik: Eine sozialwissenschaftliche Einführung (s. 117-136). De Gruyter. https://doi.org/10.1515/9783110714944-006
3. Pelikan, H, & Jung, M.F. (2023). Designing Robot Sound-In-Interaction: The Case of Autonomous Public Transport Shuttle Buses. In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’23). Association for Computing Machinery, New York, NY, USA, 172–182. https://doi.org/10.1145/3568162.3576979
4. Randall, D., & Rouncefield, M. (2018). Ethnographic Approach to Design. In K. L. Norman & J. Kirakowski (Eds.), The Wiley Handbook of Human Computer Interaction (pp. 125–141). https://doi.org/10.1002/9781118976005.ch7
5. Lucero, A., Dalsgaard, P., Halskov, K., & Buur, J. (2016). Designing with Cards. I P. Markopoulos, J.-B. Martens, J. Malins, K. Coninx, & A. Liapis (Red.), Collaboration in Creative Design: Methods and Tools (s. 75–95). Springer International Publishing. https://doi.org/10.1007/978-3-319-29155-0_5

Secondary

Related to the case of traffic:

1. Haddington, Pentti and Rauniomaa, M. (2013). Interaction between road users: offering space in traffic. Space and Culture Vol 17 (2).
2. Laurier, E., Muñoz, D., Miller, R., & Brown, B. (2020). A Bip, a Beeeep, and a Beep Beep: How Horns Are Sounded in Chennai Traffic. Research on Language and Social Interaction, 53(3), 341–356. https://doi.org/10.1080/08351813.2020.1785775
3. Liberman, K. (2013). The Local Orderliness of Crossing Kincaid. I K. Liberman (Red.), More Studies in Ethnomethodology (s. 11–43). SUNY Press.

More general

1. Crabtree, A., Rouncefield, M., & Tolmie, P. (2012). Doing Design Ethnography. Springer. https://doi.org/10.1007/978-1-4471-2726-0
2. Matarić, M., 2018: On Relevance: Balancing Theory and Practice in HRI. ACM Transactions on Human-Robot Interaction, 7(1):8:1–8:2. https://doi.org/10.1145/3209770.
3. Moore, R.J., M.H. Szymanski, R. Arar & G-J. Ren, (Eds.), 2018: Studies in Conversational UX Design. Wiesbaden: Springer. https://doi.org/10.1007/978-3-319-95579-7.
4. Šabanović, S., 2010: Robots in Society, Society in Robots. International Journal of Social Robotics, 2(4): 439–450. https://doi.org/10.1007/s12369-010-0066-7.
5. Šabanović, S. & W.-L. Chang, 2016: Socializing Robots: Constructing Robotic Sociality in the Design and Use of the Assistive Robot PARO. AI & SOCIETY, 31(4): 537–551. https://doi. org/10.1007/s00146-015-0636-1.
6. Suchman L. Human-Machine Reconfigurations: Plans and Situated Actions. Second edition. Cambridge University Press; 2007.
7. Weiss A, Spiel K. Robots beyond Science Fiction: mutual learning in human–robot interaction on the way to participatory approaches. AI & Society. 2022; 37:1-15. Available from: https://doi.org/10.1007/s00146-021-01209-w.
8. Ylirisku, S. & J. Buur, 2007: Designing with Video. Focusing the User-Centred Design Process. Wiesbaden: Springer.

ECTS: 3
Time: November 12-15
Place: Aalborg University, Rendsburggade 14, 9000 Aalborg. Room will follow
Number of seats: 12

Disclaimer:
DDSA has explicit permission from Arcanic and the owners of the https://phdcourses.dk/ website to display the courses on ddsa.dk.