Welcome to The social side of robots and AI. Exploring video-based method for the analysis of interaction with robots and artificial intelligence 

This PhD course prepares students to explore video-based methods for the analysis and design of people’s interactions with robots or artificial intelligence (AI) in social settings.

Social and collaborative robots as well as AI agents are increasingly leaving factory shop floors and research laboratories and enter the everyday life and routines of various people in private, public, institutional, and organizational settings. Robots and AI agents are integrated in diverse contexts including family interactions, education, teamwork, public transport, and hospital settings. In contrast to controlled laboratory environments, these technologies now enter the complexity and “messiness” of real-world contexts where they meet a diversity of people engaging and interacting with the technology in various forms and for various purposes. Traditionally, laboratory research uses quantitative methods often focusing on dyadic task-based interaction, with one user and one robotic or AI system solving one specific task. However, to understand what happens when people engage with robots and AI in real-world settings, we need to open our perspective to the specific social context, the diversity of people that engage with the technologies and the various forms in which people engage/disengage, interact or collaborate with the technology as a single user, in groups or with other participation forms or roles. Therefore, we need to turn to qualitative methods that can deal with the complexity and dynamics of interaction with robots and AI in real world settings.

The course introduces video-based methods for analyzing and designing robots and AI for real-world settings, combining video/ethnography and co-creation tools from participatory and interaction design. We will focus on methodological issues centered around: 1) Describing practices using ethnomethodology, video ethnography and analysis, 2) Intervening in practices using enactments, bodystorming, and voice-overs, 3) Reflecting on how these practices inform our understanding of real-world practices and concrete guidelines and how this in turn can inform the design of robots and AI, thus facilitating interdisciplinary dialogue and collaboration.

Learning Objectives:

The students will gain knowledge and practical experience in:

a)      Planning and conducting video ethnographic studies, including data collection, transcription, and video analysis.

b)      Preparation and facilitation of collaborative data sessions and co-creation workshops, using techniques for rapid prototyping, including enactments, role-play and bodystorming/vocal sketching.

c)      Facilitating and navigating interdisciplinary collaborations by using techniques that can bridge both fields.

No prerequisites are required.

Participation requires taking part in the course activities, including the preparation and presentation of a brief talk during the course.

Teaching methods:

The course is divided into two parts. The first part is taught through lectures that introduce methods and workshops in which the students gain firsthand practical experience. In the second part of the course, the students will relate and apply the acquired knowledge to their own PhD-project and present their results for discussion at the end of the course.

For practical and pedagogical reasons, the exercises in the first part of this course will particularly focus on video-based methods exploring the impact of robots on social practices and explore how a deeper understanding of these practices can inform design. In the second part of the course, we will encourage students to apply the methods to their own PhD projects, which may include robots and AI agents in any social setting.

Target group

This course is directed to an interdisciplinary audience looking for students of social science and humanities, design and robotics, HRI or HCI who are interested in:

a)      Using video/audio recordings for analysing meaning-making processes in social settings in which robots or AI agents are or should be embedded,

b)      exploring interactional and embodied methods for designing robots and AI agents for interactions in real world settings, and

c)      developing an embodied and interactional understanding of human practices and human-centred design processes.

Both Danish and foreign students are welcome. The course will be taught in English.

Description of poster:

No paper will be required.

However, we will need a poster of every participant introducing their PhD project. The poster should be handed in ca 2 weeks before the course starts (a deadline will be announced). The posters help us to get an overview over the topics and enable us to relate to their projects during the course. The posters are on display during the entire course to facilitate sparing between the PhD students.

Program outline:

Day 1: Describing (9.00-17.00)

• Overview of the course & welcome
• Introduction Lectures to the topic of observation for design (video observation, transcription, and video analysis
• Workshops on transcription and video analysis
• Q&A

Day 2: Intervening (9.00-17.00)

• Introduction Lectures video-based and participatory methods for rapid prototyping with focus on co-creation methods like enactment and bodystorming
• Workshops testing different methods of enacting human robot interaction
• Q&A

Day 3:  Describing and intervening in own PhD projects (9.00-17.00)

• Broadening the perspective to other application scenarios & bridging analysis and design in mini projects: Dive into self-selected themes of observing, analyzing enacting and informing/designing regarding your own project (lectures and discussion)
• Afternoon: individual work on relating the methods to the students’ own projects

Day 4: Reflecting (9.00-13.00)

• Students’ presentations

Final Discussions and evaluation

Organizer:
Antonia Krummheuer

Course teachers – we will be co-teaching

Dr. Antonia L. Krummheuer, Associate Professor for Qualitative Methods and Technology Studies, Department of Communication and Psychology, member of the HRI Lab at Aalborg University and the steering committee of Aalborg Robotic Challenge (organizer and lecturer)

Dr. Hannah R. M. Pelikan, postdoc in the AI in Motion group, Department of Culture and Society, Linköping University. Expert on video methods and HRI. She has developed several techniques for bridging video ethnography and interaction design and taught several courses on video analysis and design methods at CHI and HRI conferences and summer schools (lecture).

ECTS:
3

Time:
12,13,14,15 August 2025

Place:
TBA

Zip code:
9000

City:
Aalborg

Number of seats:
12

Deadline for enrolment:
29. July 2025

Deadline for uploading the poster:
TBA

Key literature:

Mandatory

Interaction analysis (methods)

1. Heath, C., Hindmarsh, J., & Luff, P. (2010). Video in Qualitative Research: Analysing Social Interaction in Everyday Life. Sage.
2. Hannah Pelikan. 2023. Transcribing human–robot interaction: Methodological implications of participating machines. In P. Haddington, T. Eilittä, A. Kamunen, L. Kohonen-Aho, T. Oittinen, I. Rautiainen, & A. Vatanen (Eds.), Ethnomethodological Conversation Analysis in Motion: Emerging Methods and New Technologies (1st ed.). Routledge. https://doi.org/10.4324/9781003424888

Interaction with Robots

1. Alač, M., J. Movellan & F. Tanaka, 2011: When a Robot Is Social: Spatial Arrangements and Multimodal Semiotic Engagement in the Practice of Social Robotics. Social Studies of Science, 41(6): 893–926. https://doi.org/0.1177/0306312711420565.
2. Dautenhahn K. Some Brief Thoughts on the Past and Future of Human-Robot Interaction. ACMTrans. Hum.-Robot Interact. 2018;7(1). Available from: https://doi.org/10.1145/3209769.
3. Hornecker, E., Krummheuer, A. L., Bischof, A., & Rehm, M. (2022). Beyond dyadic HRI: building robots for society. interactions, 29(3), 48–53. https://dl.acm.org/doi/10.1145/3526119

Interaction Design

1. Brandt, E., Binder, T., & Sanders, E. B.-N. (2012). Tools and techniques: Ways to engage telling, making and enacting. I J. Simonsen & Toni. Robertson (Red.), Routledge International Handbook of Participatory Design (s. 145–181). Routledge.
2. Krummheuer, A. L. (2023). Practice-based robotics: How sociology can inform the development of social robots. I F. Muhle (red.), Soziale Robotik: Eine sozialwissenschaftliche Einführung (s. 117-136). De Gruyter. https://doi.org/10.1515/9783110714944-006
3. Pelikan, H, & Jung, M.F. (2023). Designing Robot Sound-In-Interaction: The Case of Autonomous Public Transport Shuttle Buses. In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’23). Association for Computing Machinery, New York, NY, USA, 172–182. https://doi.org/10.1145/3568162.3576979
4. Randall, D., & Rouncefield, M. (2018). Ethnographic Approach to Design. In K. L. Norman & J. Kirakowski (Eds.), The Wiley Handbook of Human Computer Interaction (pp. 125–141). https://doi.org/10.1002/9781118976005.ch7
5. Lucero, A., Dalsgaard, P., Halskov, K., & Buur, J. (2016). Designing with Cards. I P. Markopoulos, J.-B. Martens, J. Malins, K. Coninx, & A. Liapis (Red.), Collaboration in Creative Design: Methods and Tools (s. 75–95). Springer International Publishing. https://doi.org/10.1007/978-3-319-29155-0_5

Secondary

1. Crabtree, A., Rouncefield, M., & Tolmie, P. (2012). Doing Design Ethnography. Springer. https://doi.org/10.1007/978-1-4471-2726-0
2. Matarić, M., 2018: On Relevance: Balancing Theory and Practice in HRI. ACM Transactions on Human-Robot Interaction, 7(1):8:1–8:2. https://doi.org/10.1145/3209770.
3. Moore, R.J., M.H. Szymanski, R. Arar & G-J. Ren, (Eds.), 2018: Studies in Conversational UX Design. Wiesbaden: Springer. https://doi.org/10.1007/978-3-319-95579-7.
4. Šabanović, S., 2010: Robots in Society, Society in Robots. International Journal of Social Robotics, 2(4): 439–450. https://doi.org/10.1007/s12369-010-0066-7.
5. Suchman L. Human-Machine Reconfigurations: Plans and Situated Actions. Second edition. Cambridge University Press; 2007.
6. Weiss A, Spiel K. Robots beyond Science Fiction: mutual learning in human–robot interaction on the way to participatory approaches. AI & Society. 2022; 37:1-15. Available from: https://doi.org/10.1007/s00146-021-01209-w.
7. Ylirisku, S. & J. Buur, 2007: Designing with Video. Focusing the User-Centred Design Process. Wiesbaden: Springer.

Important information concerning PhD courses: 

There is a no-show fee of DKK 3,000 for each course where the student does not show up. Cancellations are accepted no later than 2 weeks before the start of the course. Registered illness is of course an acceptable reason for not showing up on those days. Furthermore, all courses open for registration approximately four months before start of the course.

We cannot ensure any seats before the deadline for enrolment, all participants will be informed after the deadline, approximately 3 weeks before the start of the course.

For inquiries regarding registration, cancellation or waiting list, please contact the PhD administration at aauphd@adm.aau.dk When contacting us please state the course title and course period. Thank you.

Disclaimer:
DDSA has explicit permission from Arcanic and the owners of the https://phdcourses.dk/ website to display the courses on ddsa.dk.