SCOPE
Computer gaming has been acknowledged as one of the computing disciplines which proposes new interaction paradigms to be replicated by software engineers and developers in other fields. The abundance of high-performance, yet lightweight and mobile devices and wireless controllers has revolutionized gaming, especially when taking into account the individual affective expressivity of each player and the possibility to exploit social networking infrastructure. As a result, new gaming experiences are now possible, maximizing users’ skill level, while also maintaining their interest to the challenges in the same, resulting in a state which psychologists call flow: “a state of concentration or complete absorption with the activity at hand and the situation”. The result of this amalgamation of gaming, affective and social computing has brought increased interest in the field in terms of interdisciplinary research.
Natural interaction plays an important role in this process, since it gives game players the opportunity to leave behind traditional interaction paradigms, based on keyboards and mice, and control games using the same concepts they employ in everyday human-human interaction: hand gestures, facial expressions and head nods, body stance and speech. These means of interaction are now easy to capture, thanks to low-cost visual, audio and physiological signal sensors, while models from psychology, theory of mind and ergonomics can be put to use to map features from those modalities to higher-level concepts, such as desires, intentions and goals. In addition to that, non-verbal cues such as eye gaze and facial expressions can serve as valuable indicators of player satisfaction and help game designers provide the optimal experience for players: games which are not frustratingly hard, but still challenging and not boring.
Another aspect which makes computer gaming an important field for multimodal interaction is the new breed of multimodal data it can generate: besides videos of people playing games in front of computer screens or consoles, which include facial, body and speech expressivity, researchers in the field of affective computing and multimodal interaction may benefit from mapping events in those videos (e.g. facial signs of frustration) to specific events in the game (large number of enemies or obstacles close to the player) and infer additional user states such as engagement and immersion. Individual and prototypical user models can be built based on that information, helping produce affective and immersive experiences which maintain the concept of ‘flow’. This workshop will cover real-time and off-line computational techniques for the recognition and interpretation of multimodal verbal and non-verbal activity and behaviour, modelling and evolution of player and interaction contexts, and synthesis of believable behaviour and task objectives for non-player characters in games and human-robot interaction.
The workshop also welcomes studies that provide new insights into the use of gaming to capture multimodal, affective databases, low-cost sensors to capture user expressivity beyond the visual and speech modalities and concepts from collective intelligence and group modelling to support multi-party interaction.
WORKSHOP TOPICS
- Multimodal affect and behaviour recognition, including:
- Facial expressions
- Body language
- Speech
- Physiological
- Other modalities
- Affect and behaviour generation in non-player characters, including:
- Gaze/engagement
- Gestures/Body stance
- Facial expressions
- Speech
- Other modalities
- Higher-level concepts in gaming
- NPC Tasks, objectives and adaptation
- User engagement, attention and satisfaction
- Maximising user engagement
- Affective and behavioural states in gaming
- Social context awareness and adaptation
- Modality replacement for gaming across devices
- Mapping hand to touch gestures
- Extracting expressivity and affect on mobile devices
- Low-cost recognition of user actions
- Cognitive and affective ‘mentalising’
- Natural Human-Robot Interaction (HRI) / Human-Computer Interaction (HCI)
- Game-based corpora (naturally evoked or induced emotion)
- Applications to interactive games, robots and virtual agents
WORKSHOP ORGANISERS
Kostas Karpouzis (ICCS-NTUA, Greece) – kkarpou (at) cs.ntua.gr
Ginevra Castellano (Queen Mary University of London, United Kingdom) - ginevra (at) eecs.qmul.ac.uk
Christopher Peters (Coventry University, United Kingdom) - Christopher.Peters (at) coventry.ac.uk
Louis-Philippe Morency (University of Southern California, USA) - morency (at) ict.usc.edu
Laurel Riek (University of Notre Dame, USA) - Laurel.Riek (at) nd.edu
Georgios Yannakakis (IT University, Denmark) - yannakakis (at) itu.dk
