PIs: Martina Rau, Xiaojin Zhu, Robert Nowak
University of Wisconsin-Madison
Award Details

Instructors often use visuals to help students learn (e.g., pie charts of fractions, or ball-and-stick models of chemical molecules) and assume that students can quickly discern relevant information (e.g., whether or not two visuals show the same chemical) once that visual representation has been introduced. But comprehension is not the same as fluency — students still expend significant mental effort and time interpreting even visuals that they understand conceptually, and the resulting cognitive load can cause them to miss other important information that instructors are imparting. To help improve student fluency with visuals, a series of experiments with undergraduate students and chemistry professors will investigate which visual features they pay attention to and use sophisticated statistical methods to devise example sequences that will most efficiently help students learn to pay attention to relevant visual features. Based on this research, the project team will develop a visual fluency training that will be incorporated into an existing, successful online learning technology for chemistry. The potential educational impact will not be limited to chemistry instruction: given the pervasiveness of visual representations in STEM fields and the number of students who struggle with rapid processing of those visuals, the products of this research could be integrated into other educational technologies.

The PIs will develop a methodology for cognitive modeling of perceptual learning processes that can create adaptive support for perceptual learning tasks. The research will combine machine learning with educational psychology experiments using an Intelligent Tutoring System (ITS) for undergraduate chemistry. In Phase 1, metric learning will assess which visual features of representations novice students and chemistry experts focus on. Applying metric learning to a novice-expert experiment will establish a skill model of student perceptions and perceptual learning goals for the ITS. In Phase 2, the team will use machine learning to develop a cognitive model of perceptual learning. The team will conduct a chemistry learning experiment and apply machine learning to test cognitive models. In Phase 3, the team will use the cognitive model to reverse-engineer optimal sequences of perceptual learning tasks. An experiment will evaluate the effectiveness of these sequences, and the team will build on this analysis to create an adaptive version of perceptual learning tasks. A final experiment will evaluate whether incorporating adaptive perceptual learning tasks with conceptually focused instruction enhances learning. Because educational technologies have traditionally focused on explicit learning processes that lead to conceptual competencies, they cannot currently assess the implicit learning processes that lead to perceptual fluency. Combining educational psychology, cognitive science, and machine learning will yield new cognitive models that could transform the adaptive capabilities of educational technologies to support such perceptual fluency as well as other implicit forms of learning. The project will also yield next-generation computational algorithms to model human similarity judgments and to use adaptive surveying to collect data on perceptual judgments more efficiently.