Technology Enabled Formative Assessment

Contributors: Mingyu Feng

Traditionally, assessments have been used to measure how much students have learned up to a particular point in time (Stiggins, 2007). This is what Rick Stiggins calls “assessment of learning” and is used to show if students are meeting standards set by the state, the district, or the classroom teacher. These summative assessments are conducted after a unit or certain time period in an effort to determine how much learning has taken place. Although Stiggins notes that assessments of learning are important if we are to ascribe grades to students and provide accountability, he urges teachers to focus more on assessment for learning. These types of assessment — formative assessments — support learning during the learning process and give the teacher information about what students are understanding so that instruction can potentially be modified to better help students.

Summative assessments typically collect assessment data oriented toward monitoring progress of schools one to multiple times a year. An annual cycle of data collection and use in planning is appropriate for an administrative system and for accountability, with longitudinal data adding value to this process (Herman & Gribbons, 2001). Large-scale assessment data are valuable indicators for administrators, policymakers, and the general public because they provide important feedback information to schools about their performance that can be used in improvement efforts. But these data are of much less use to a teacher in diagnosing her students’ learning progress and in planning appropriate instructional interventions for next week. For classroom-level instructional decision-making, cycles of formative assessment data collection and use should be extremely short, from weeks to just minutes to improve student-learning outcomes (Wiliam, 2007). Collecting formative assessment data minute-by-minute introduces the need for technology tools. These tools need to be designed to collect the data, support classroom instruction, and ideally provide guidance on what are the best pedagogical decisions, given the data, to improve learning outcomes.

Currently, researchers hold the belief that next-generation classroom technology infrastructure should be designed to support the instructional practices to improve student learning that research evidence shows to be effective. Implementation of formative assessment often requires collecting observations and other classroom-based data to support differentiated instruction. Nowadays, technology empowers the teachers and classrooms by providing a systematized and uniform process of presenting, analyzing, and communicating student results (CTB/McGraw Hill. n.d.). Formative assessment, one of the most effective instructional interventions, as evidenced by research, is a key instructional practice that should be supported by tomorrow’s classroom technologies.

Transformative Potential

Formative assessment has been the object of much attention in K-12 research and practitioner communities (Black & Wiliam, 1998a, 1998b; Boston, 2002; Roediger & Karpicke, 2006). It can be one of the most powerful ways to improve student achievement. These assessments have the potential to provide critical information to students and teachers about whether students understand the targeted concepts, and if not, what problematic or partial understandings are present instead. Teachers can then use the evidence about student understanding to decide what to do to help students progress from partial or incorrect understandings toward targeted learning goals. Black and Wiliam (1998a) reviewed 250 studies to ascertain the impact of “formative” assessment on learning. They describe impressive learning gains within the classroom especially when “formative” feedback is at the teacher-student interaction level, effect sizes of 0.40 to 0.70 — effect sizes larger than those seen for any other instructional intervention tested. Research has found that frequent use of formative assessments can improve achievement, particularly when the results are used to adjust instruction (Bergan, Sladeczek, Schwarz, & Smith, 1991; Speece, Molloy, & Case, 2003).

Formative assessments can help teachers differentiate instruction and improve student achievement (Dodge, 2009). Dodge (2009) tells of how teachers have told her that they often don’t have time to assess students because of the tight schedule for covering new content. She also discusses, that in the rush to cover more, students are actually learning less. Teachers don’t have thorough understanding of where students are on topics that have been covered, and students are unlikely to retain much of what has been “covered” in their classrooms (Dodge, 2009). Formative assessments can guide teachers in speeding up or slowing down their work to best help students.
Formative assessment has been shown to have beneficial effects for student motivation as well. Black and Wiliam (1998) and other researchers have shown that formative assessment and the feedback to students about progress and performance is motivating to students and can increase student persistence, sense of self-efficacy, and self-regulated learning (Brookhart, 1997, 2001; Stiggins, 2001b).

In Celebrating 20 Years of Research on Educational Assessment: Proceedings of the 2005 CRESST Conference, Anne Lewis says: Formative assessment is central to good instruction in several ways, including focusing learning activities on key goals; providing students feedback so they can rework their ideas and deepen their understanding; helping students develop metacognitive skills to critique their own learning products and processes; and providing teachers with systematic information about student learning to guide future instruction and improve achievement. Yet teachers typically have little preparation in assessment or how to use such information well.

Today’s teachers are responsible for an ever-growing list of instructional and administrative requirements to support learning. As a result, incorporating formative assessment for individualized learning is often ignored. Alternatively, technology enabled formative assessment may have the potential to bring formative assessment and the associated benefits to more teachers, students, and classrooms. Technology can give educators the tools that can make it easier to learn how to effectively implement and use formative assessment(CTB/McGraw Hill. n.d.). If technology enabled formative assessment can help teachers be more consistent in formatively assessing students, students maybe able to learn and retain more from their classroom experiences.

There are various technologies that provide support for formative assessments so that they can be done in an efficient manner. These technologies help educators quickly make important decisions that impact classroom-wide, school-wide and district-wide student progress.

First of all, technology has been increasingly used to support classroom summative assessments — that is, assessment of student learning for evaluation purposes. Technology can be used in the classroom to generate, analyze, and feedback high-quality student achievement data to support learning and instruction in a timely, usable fashion (Bennett, 1999; Pellegrino, Chudowsky, & Glaser, 2001). When carefully designed to align with the curriculum, standards, and large-scale tests, technology-supported classroom assessment has the potential to generate data that are relevant to the curriculum and usable in guiding classroom instruction as well as informing accountability programs (e.g., Wilson & Draney, 2004). However, such assessment systems are very rare. (Crawford, et al., 2008)
Classroom response systems, also called Clickers, have been widely used and showed capabilities for collecting data about student understandings. Using the clickers, teachers can pose multiple-choice question and collect response from students instantly. Students’ responses can be aggregated visually and shared with the class for discussion. The graphical representation provides immediate feedback to students and offers a snapshot of students’ thinking to teachers (Bransford, Brophy, & Williams, 2000; Bransford, Brown, & Cocking, 2000; Roschelle, Penuel, & Abrahamson, 2004; Zurita, Nussbaum, & Salinas, 2005).
There has been a large interest in formative assessments in K-12 Education (Olson, 2004) with many companies providing online support services. Some online testing systems (such as Renaissance Learning, www.renlearn.com) automatically grade students and provide reports. Systems such as ASSISTments (Feng, Heffernan, & Koedigner, 2009) went one step further and combines formative assessments with online learning assistance. When students respond to ASSISTments problems, they receive hints and tutoring to the extent they need them. At the same time, ASSISTments uses how individual students respond to the problems and how much support they need from the system to generate correct responses as continuous, assessment information, and provide detailed diagnostic reports for teachers to adjust their instruction accordingly. Diagnoser.com is another example of online formative assessment that focuses on diagnostic assessment and is oriented towards facet-based instruction. Facet-based assessments are one innovative approach to helping teachers diagnose students’ science understanding (Minstrell, 2001; Minstrell, Anderson, Kraus, & Minstrell, 2008). Beginning with the research on misconceptions and conceptual change, the term “facets of students’ thinking” was coined to acknowledge that not all of students’ thinking can be considered a “misconception” or error. As a student works through a set of questions associated with facets, she receives targeted feedback after responding to each item. Teachers receive real-time results of students’ answers in a report describing the facets used by each student.

Vision

Ms. Anderson plans to cover the topic “Pythagorean Theorem” today for her 8th graders. Before diving into the new topic, she asked each student to log into the online system to do a quick 5 minute formative assessment which she prepared last night. The test includes 3 questions on the topic “Square Root” that they covered a week ago, and 2 questions that ask about student’s basic understanding about the Pythagorean Theorem. Students’ answers are displayed on her screen and a summary report is generated instantly after the last student has finished the test. The report shows that about 40% of her students are delayed on mastering “Square root” while the other 60% are ready to move on. Ms. Anderson decides to hold on to the instruction on the new topic for the 40% till they are ready. She quickly split the whole class into two groups: Blue Birds, the delayed group, and Red Wings, the mastery group. Blue Birds is told to practice on the topic “Square Root” in the online tutoring system that adaptively presents problems to students based on their performance level and scaffolds the problem solving process using immediate feedback messages, worked-out examples, and illustrations. Meanwhile, Ms. Anderson starts lesson on “Pythagorean Theorem” with Red Wings. 20 minutes later, Ms. Anderson is done with her lesson. She checks on the Blue Birds and notices they have demonstrated learning over “Square Root” as shown by the formative assessment report generated based on their performance in the past 20 minutes. She then swaps the two groups to give instruction to the Blue Birds for 20 minutes while having the Red Wings to practice on the newly covered topics with support by from the online tutoring system.

Challenges

While the positive role of formative assessment has been widely accepted in the educational field, challenges persist for the implementation of formative assessment practice and maintaining the ongoing formative assessment system in schools. A few issues and concerns that should be considered are listed below as examples.

The initial challenge is one of understanding: the most effective formative assessments are embedded within the classroom and happen on a moment to moment basis. So the implementation challenge is developing formative assessment habits in teachers AND students. One of the barriers to effective assessment for learning is the classroom culture of the traditional classroom where students are compared to one another on their grades. Personal progress should be measured and acknowledged by the student themselves as we empower them in their own learning.
How to fit formative assessment into a fixed schedule without losing the long term goal? In principle, people agree that formative assessment helps teaching according to needs of children and creates more accurate ongoing leveling rather than looking at an end of year test which is often inaccurate. But staff often feels that once they have printed planning it is a document that is fixed in stone. Some of the staff getting to grips with this process tends to get lost in the process and lose sight of longer term goals. Teachers are all under a tension to get a list of topic covered in a semester. How to help them to make time to use the data from formative assessment?
Provide professional development to teachers: Formative assessment practices have the potential to increase student learning, but only where the teachers are ‘geared up’ to adjust instruction and learning activities quickly and responsively while learning is in practice. Teachers may not know how to effectively reteach to the students that demonstrated lack-of-mastery on formative assessments. Additionally, differentiation and meeting individual student needs can become difficult, as students achieve their learning goals at different paces. So professional development needs to be provided to help them understand the output of formative assessment and respond to the results appropriately.
The introduction of formative assessments needs a change in the administration policy as teachers may know formative assessment is beneficial but hands can be tied by political pressure to test, test, test, and test again.
Establish the validity of claims made from formative assessment. Not like summative assessment, there is often no strict alignment with state, district, or local standards to provide a common framework for understanding student progress.

Above are general challenges for formative assessment. As technology steps into the field of formative assessment, new concerns appear as well:

The cost of introducing the computer support system into classroom, which often includes clicker systems, laptop or desktop computers, touch pads, smart boards or other type of displaying station, etc.
The technology should be easy-to-use and highly accessible to lower the demands for teachers and students to be tech-savvy to be able to operate the support system.
The technology, ideally, should provide opportunities for intervention and provide instructional resources for teachers to use to reinforce concepts.
The data and information output from a computer system could be overwhelming sometimes because of the easiness of collecting data using computers. Designers and developers of formative assessment systems need to be careful with the types and representation of the reports such that the data is informative and understandable to teachers and can be immediately acted upon by teachers.
Computer support system has the potential to make formative assessment more efficient and more effective. But it is not a silver bullet. Teachers and students are still in the centric position to ensure the success of formative assessment. Support for customization and training becomes more essential.

Top Related Areas in Cyberlearning

Educational Data Mining
Visualization

References

Bergan, J. R., I. E. Sladeczek, R. D. Schwarz & A. N. Smith (1991). Effects of a measurement and planning system on kindergartners’ cognitive development and educational programming. American Educational Research Journal 28, 683–714.

Black, P., & Wiliam, D. (1998a). Assessment and classroom learning. Assessment in Education: Principles, Policy and Practice, 5, 7–74.

Black, P., & Wiliam, D. (1998b). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2): 139–149.

Boston, C. (2002). The concept of formative assessment. Practical Assessment, Research & Evaluation, 8(9).

Bransford, J., Brophy, S., & Williams, S. (2000). When computer technologies meet the learning sciences: Issues and opportunities. Journal of Applied Developmental Psychology, 21(1), 59–84.

Bransford, J., Brown, A., & Cocking, R. (Eds.). (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.

Brookhart, S. M. (1997). A theoretical framework for the role of classroom assessment in motivating students’ effort and achievement. Applied Measurement in Education, 10(2), 161–180.

Brookhart, S. M. (2001). Successful students’ formative and summative uses of assessment information. Assessment in Education, 8(2), 153–169.

CTB/McGraw Hill. (n.d.). Promoting student achievement using research-based assessment with formative benefits. A White Paper prepared by CTB/McGraw Hill.

Crawford, V. M., Schlager, M., Penuel, W. R., & Toyama, Y. (2008). Supporting the art of teaching in a data-rich, high performance learning environment. In E. B. Mandinach & M. Honey (Eds.), Linking data and learning (pp. 109-129). New York: Teachers College Press.

Dodge, J. (2009). 25 Quick Formative Assessments for a Differentiated Classroom. New York, NY: Scholastic Teaching Resources.

Feng, M., Heffernan, N.T., & Koedinger, K.R. (2009). User modeling and user-adapted interaction: Addressing the assessment challenge in an online system that tutors as it assesses. The Journal of Personalization Research (UMUAI journal). 19(3), 243-266.

Herman, J., & Gribbons, B. (2001). Lessons learned in using data to support school inquiry and continuous improvement: Final report to the Stuart Foundation (CSE Technical Report 525). Center for the Study of Evaluation, University of California, Los Angeles.

Minstrell, J. (2001a). Facets of students’ thinking: Designing to cross the gap from research to standards-based practice. In K. Crowley, C. D. Schunn and T. Okada (Eds.), Designing for Science: Implications for Professional, Instructional, and Everyday Science. Mahwah: Lawrence Erlbaum Associates.

Minstrell, J., Anderson, R., Kraus, P., & Minstrell, J.E. (2008). Bridging from practice to research and back: tools to support formative assessment. In J. Coffey, R. Douglas and C. Sterns (Eds.), Science Assessment: Research and Practical Approaches: NSTA Press.

Olson, L.: State test programs mushroom as NCLB Mandate Kicks. In: Education Week, 20 November,pp. 10–14 (2004)

Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.

Roediger, H. III., & Karpicke, J. D. (2006). Test-enhanced learning: taking memory tests improves long-term retention, Psychological Science, 17(3), 249–255.
Roschelle, J., Penuel, W. R., & Abrahamson, A. L. (2004). The networked classroom. Educational Leadership, 61(5), 50–54.

Speece, D.L., Molloy, D.E., & Case, L.P. (2003). Starting at the beginning for learning disabilities identification: Response to instruction in general education. Advances in Learning and Behavioral Disabilities, 16, 37-50.

Stiggins, R.J. (2007). Assessment for learning: A key to student motivation and learning. Phi Delta Kappa EDGE, 2(2), 19 pp.

Wiliam, D. (2007). Keeping learning on track: Formative assessment and the regulation of learning. In F. K. Lester, Jr. (Ed.), Second handbook of mathematics teaching and learning. Greenwich, CT: Information Age Publishing.

Wilson, M., & Draney, K. (2004). Some links between large-scale and classroom assessments: The case of the BEAR system. In M. Wilson (Ed.), Towards coherence between classroom assessment and accountability. 103rd Yearbook of the National Society for the Study of Education (pp. 132–154). Chicago: University of Chicago Press.

Zurita G., Nussbaum, M., & Salinas, R. (2005). Dynamic Grouping in Collaborative Learning Supported by Wireless Handhelds. Educational Technology & Society 8(3):149–161.