Mar 20, 2009
The Math Science Partnership of Southwest Pennsylvania is a collaboration among K–12 school districts, higher education institutions, and other partners aimed at improving math and science education. The partnership has three goals: to enhance the quality of the math and science education workforce at all grade levels, to create sustainable partnerships between K–12 and higher education in the teaching of math and science, and to increase K–12 students' knowledge of math and science. RAND's evaluation found indications of changes that were consistent with these goals, although further analyses did not produce evidence that the partnership was directly responsible for the changes.
Created by the National Science Foundation (NSF) in 2002, the Math and Science Partnership program funds collaborative projects that pair K–12 school districts with institutes of higher education (IHEs) in an effort to improve math and science education. The Math Science Partnership of Southwest Pennsylvania (MSP) is one of approximately 140 funded by NSF nationwide. The MSP brings together 53 K–12 school districts, four IHEs, and other strategic partners in Southwestern Pennsylvania. The MSP's goals are (1) to enhance the quality of the math and science education workforce at all grade levels, (2) to create sustainable partnerships between schools and IHEs in the teaching of math and science, and (3) to increase K–12 students' knowledge of math and science. The RAND Corporation and the University of Pittsburgh were subcontracted to measure the program's progress toward these goals and the extent to which that progress could be attributed to the MSP's activities. The five-year evaluation period ended in 2008. This research brief summarizes RAND's findings regarding the MSP's progress to date. Because the program is expected to continue until 2010, these findings should be considered interim and do not constitute a final evaluation.
Overall, the evaluation found indications of change that were consistent with the MSP's goals; however, statistical analysis showed little relationship between participation in MSP activities and student achievement outcomes.
One premise underlying the MSP is that providing teachers with opportunities to deepen their content and pedagogical knowledge in the context of high-quality instructional materials will improve their preparation for teaching math and science. The MSP implemented several activities for both K–12 and IHE educators to achieve this goal, including seminars, short courses, and workshops. Activities also included opportunities for K–12 teachers to spend time at one of the four IHEs and to participate in professional development activities that brought them into contact with IHE educators. These activities exposed educators to reform-oriented teaching practices, content knowledge, pedagogical techniques, and leadership training to help disseminate ideas to colleagues. Interviews with teachers, principals, and IHE faculty suggested that these programs were having a positive impact. Educators at both the K–12 and IHE levels reported changes in leadership and instruction that were consistent with MSP goals. K–12 teachers reported that MSP activities increased their understanding of math and science concepts and how students think about math and science and also helped them change their teaching practices. They also reported that the MSP professional development was more often relevant to their needs than other development activities had been. Principals reported that the principal seminars influenced their views and behaviors. IHE faculty reported a greater emphasis on student-centered instruction and more awareness of different pedagogical techniques.
These findings suggest that the activities of the MSP may be associated with improvements in the quality of the educator workforce. However, further statistical analyses of the relationship between participation in MSP activities and these improvements did not yield conclusive evidence that the MSP was responsible for them.
The MSP is intended as a sustainable partnership that benefits both K–12 districts and IHEs. Surveys and interviews conducted by the evaluation team found indications that changes had occurred that were consistent with MSP partnership goals — specifically, greater numbers of interactions between IHE and K–12 educators, better relationships among IHEs, and changing attitudes within IHE departments. Participation in MSP activities, along with revisions to IHE courses through the teacher fellow program, helped faculty embrace a wider variety of approaches to presenting the material to their students. As a result of these changes, K–12 and IHE faculty members were optimistic that students would become more engaged in coursework and take greater responsibility for their own learning. Within some IHE departments, there was increased acceptance of a broader definition of scholarship that would include professional recognition for participating in an MSP-style partnership. This could result in sustained MSP participation by IHE faculty, beyond the grant period. Moreover, broader definitions of scholarship can also benefit IHEs in other ways, by increasing collegiality and improving the overall educational climate.
The theory underlying the MSP presumes that enhancing the preparedness of K–12 and IHE educators and building sustainable partnerships between districts and IHEs will translate into student gains in math and science. These gains are the program's ultimate goal.
Analysis of student test scores revealed improvements in student achievement during the project that were consistent with the MSP's goals. For example, science and math scores generally increased, and there were increases in the number of students who scored "proficient" or higher on math exams. However, analysis of the relationship between educator participation in the MSP and student achievement found no clear connection between the two. Moreover, math test data from non-MSP districts in Pennsylvania showed nearly identical student gains (see figure). Similar comparison data were not available for science.
If MSP participation by educators is having an effect on student learning, there are a number of possible explanations why it might not have been detected in this evaluation. For example, participation was measured by hours spent in MSP activities, an imperfect metric that does not consider potentially important factors, such as level of engagement among educators. Moreover, the math and science exams used for this analysis might not be sensitive enough to the changes in student learning that could result from the MSP intervention. Finally, teachers might have had insufficient time to translate their MSP learning into classroom teaching.
The assessment found numerous changes that suggest that MSP partners appear to be making progress toward the three MSP goals. However, statistical analysis did not find evidence that participation in the MSP was directly responsible for this progress. Although RAND's role as formal evaluator is now over, partners on the evaluation team continue to collect and analyze data that may yield further insight into the relationship between areas of progress and MSP activities.