PI: Rebecca Passonneau
Pennsylvania State Univ University Park
Award Details
Development of students’ writing skills promotes critical thinking across disciplines, and professional success. Yet the past decade of the Nation’s Report Card on students’ writing points to a long-standing crisis in writing instruction that persists through post-secondary school. Students need more instruction on how to write, and instructors need more support to provide students with comprehensive, targeted feedback. This project will develop an Automated Instruction Assistant (AIA) to provide post-secondary instructors with feedback on essays while they grade them, through the application of natural language processing and machine learning techniques to the analysis of essay content and argumentation. The project will apply an iterative design process to a sequence of two argumentative essay assignments in the context of a freshman course on academic skills in a computer science department, where the enrollment is in the hundreds. It will integrate state-of-the art methods in content analysis and argument mining of text to model text meaning more deeply than in previous work. Automated support for application of educational rubrics to argumentative essays will help instructors to provide more comprehensive, standardized feedback for students, and foster transparent and supportive writing instruction.
This project will develop an AIA that assigns both a total score to an essay, and individual score dimensions, such as how well an argumentative essay articulates a major claim. The scores will be supported by pointers into the text that provide score justification. As a result, the AIA output can be linked directly to a rubric used by the instructor, which facilitates instructor reflection, training for graders, and feedback for students. The technology will integrate and extend the researchers’ previous work on content analysis and argument mining. The content analysis will take as input a small number of reference essays to generate a model of the ideas (propositions) in the domain, weighted by importance within and across essays. The argument mining will identify the propositions that play a role in an argument, and their argument relations. It will produce an argument graph in which the nodes are propositions and edges are argument relations. Integration with the content analysis will ground the propositions in the domain ideas, and make it possible to exploit the role of importance of ideas in the domain, and their prominence within an essay. Methods will include novel applications of dynamic programming and integer linear programming combined with deep learning of rich, low dimensional semantic vectors.