Building a Validity Argument for an Automated Writing Evaluation System (eRevise) as a Formative Assessment

Published in: Computers and Education Open (2022). doi: 10.1016/j.caeo.2022.100084

Posted on RAND.org on May 04, 2022

by Richard Correnti, Lindsay Clare Matsumura, Elaine Lin Wang, Diane Litman, Haoran Zhang

Read More

Access further information on this document at Elsevier Ltd

This article was published outside of RAND. The full text of the article can be found at the link above.

  • To be useful for formative assessment purposes, the information derived from an automated writing evaluation (AWE) system must capture meaningful features of a writing construct.
  • We present a validity argument for an AWE system—termed eRevise—that targets elementary students' text-based argumentative writing skills, specifically their ability to use text evidence.
  • We provide evidence that the feedback is understood by students and that their revisions are aligned with the automated feedback they received.
  • Teachers overall responded very positively to eRevise, with nearly all reporting that they would use eRevise again.
  • Improvement scores were higher when teachers reported substantive interactions with students about the students' interpretation of the automated feedback.

Research conducted by

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.