Making Sense of Data-Driven Decision Making in Education

Data-driven decision making (DDDM), applied to student achievement testing data, is a central focus of many school and district reform efforts, in part because of federal and state test-based accountability policies. This paper uses RAND research to show how schools and districts are analyzing achievement test results and other types of data to make decisions to improve student success. It examines DDDM policies and suggests future research in the field. A conceptual framework, adapted from the literature and used to organize the discussion, recognizes that multiple data types (input, outcome, process, and satisfaction data) can inform decisions, and that the presence of raw data does not ensure its effective use. Research questions addressed are: what types of data are administrators and teachers using, and how are they using them; what support is available to help with the use of the data; and what factors influence the use of data for decision making? RAND research suggests that most educators find data useful for informing aspects of their work and that they use data to improve teaching and learning. The first implication of this work is that DDDM does not guarantee effective decision making: having data does not mean that it will be used appropriately or lead to improvements. Second, practitioners and policymakers should promote the use of various data types collected at multiple points in time. Third, equal attention needs to be paid to analyzing data and taking action based on data. Capacity-building efforts may be needed to achieve this goal. Fourth, RAND research raises concerns about the consequences of high-stakes testing and excessive reliance on test data. Fifth, attaching stakes to data such as local progress tests can lead to the same negative practices that appear in high-stakes testing systems. Finally, policymakers seeking to promote educators’ data use should consider giving teachers flexibility to alter instruction based on data analyses. More research is needed on the effects of DDDM on instruction, student achievement, and other outcomes; how the focus on state test results affects the validity of those tests; and the quality of data being examined, the analyses educators are undertaking, and the decisions they are making.