PIs: Nicoletta Adamo-Villani, Purdue University (Award Details)
Richard Mayer, University of California-Santa Barbara (Award Details)

While most research on embodied pedagogical agents (adaptive virtual agents that guide and mentor learners) explores cognitive features, this project investigates the role of agent affect/emotion. This project examines how the agent’s affective state (e.g., seeming interested or concerned) impacts different types of students (e.g., differing by knowledge level, gender, underrepresented group status, interest in STEM fields, and personality profile) when learning from online statistics lessons. The project integrates several areas of research: a) computer graphics research on life-like and believable representation of emotion in embodied agents, b) advanced methods and techniques from artificial intelligence and computer vision for real-time recognition of emotions, c) cognitive psychology research on learning from affective agents, and d) education research on the efficacy of affective agents for improving student learning of STEM concepts. Through experimental research the project will advance the state of the art in agent design and implementation by integrating findings on effective emotion regulation with algorithms that support life-like expression of emotions in embodied agents.

To investigate the multimodal design features of affective pedagogical agents, the project has two main objectives: (1) research and develop novel algorithms for emotion recognition and for life-like emotion representation in embodied animated agents, and (2) develop an empirically grounded research base to guide the design of affective pedagogical agents for different types of learners. In one series of experiments the project will determine evidence-based design principles to guide the development of agents that demonstrate emotion/affect, including which kinds of affective states are most effective for which kinds of learners. In a second series of experiments, the project will implement a web-camera system to detect the emotional state of the learner (e.g., confused, interested, content, or bored), adapting the emotional state displayed by the agent in response. Of interest is whether students learn the statistics lesson better when the pedagogical agent is sensitive to the learner’s emotional state than when it is not. In addition to its scientific merit, the project will develop and make available a toolkit of affective animated pedagogical agents that adapt to learner characteristics to be used by learners of all ages, for education and training in a variety of subject matters and settings.