PI: David Johnson, Kirsten Butcher
University of Utah
Award Details

This project team, led by a learning scientist and a computer graphics and augmented reality expert are working together to design a new kind of science laboratory environment that uses augmented reality (AR) to make invisible scientific phenomena visible and continuously reinforce the conceptual principles at work during hands-on scientific experimentation. The envisioned technology visually captures the what students see as they are physically carrying out experiments, interactively processes the imagery to make key aspects available for analysis and exploration, and, in real time, automatically augments the imagery with visualizations that overlay visualizations of scientific phenomena on the real-world phenomena students are experiencing, allowing students to experience real-world phenomena and the conceptual models that describe those phenomena simultaneously. The base technology is a camera-equipped tablet. Initial content is on force, work, and motion. The technological innovation includes, first, recognizing the components in a scene, even when the lighting is bad and people and other objects occlude some of it, and second, calculating the positions of the objects so that the virtual imagery can be transformed to match the real-world positions. Foundations for the innovation can be found in what is known about the affordances of integrating multiple representations in reasoning and problem solving and the challenges and difficulties in doing that. The aim is to overcome those challenges so that the affordances of multiple representations, each of which can communicate different things, can be taken advantage of by learners. Scientists do this regularly, and a visceral understanding of how this is done by scientists is expected to help students better understand what scientists do and what counts as evidence. Research addresses how students’ thinking processes and domain understanding are influenced by the addition of augmented reality visualizations to their hands-on scientific investigations.

There is broad agreement that the current state of educational preparedness of students in STEM areas is inadequate and that improvement in these areas is critical both for American technological leadership and for an informed, policy-making citizenry. This research works to improve science education with engaging tools that tackle one of the most difficult science educational issues: deep conceptual understanding applied to real-world situations. This project team applies an elegant technological approach to an helping learners connect the abstract science concepts they are learning about to the real world they live in. Computer vision and augmented reality will be integrated and refined in a way that addresses the needs of learners. The team will make the programs and curriculum freely and broadly available via Web-based distribution and access.