Download Free Electronic Document

FormatFile SizeNotes
PDF file 0.5 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Brief
A student working on schoolwork in class

Photo by Rachael Hacking for Show the Good

Key Findings

  • Results paint a promising picture, suggesting that the NYC-CS had positive effects on most of the examined student academic and discipline outcomes, with some notable exceptions.
  • For some outcomes, particularly student attendance, the effects appear to be increasing over time.
  • There is some evidence that the NYC-CS supported improvements in school climate and culture in elementary and middle schools, but there is no evidence for similar effects in high schools.
  • Results have implications for policymakers and suggest directions for future research.

With the launch of the New York City Community Schools Initiative (NYC-CS) in 2014, the New York City Department of Education (NYCDOE) increased its focus on implementing a holistic strategy of education reform to address the social consequences of poverty as a way to improve student outcomes. The strategy of the NYC-CS is to organize resources in schools and share leadership among stakeholders so that academics, health and wellness, youth development, and family engagement are integrated into the fabric of each school. Although community schools are expanding across the country, New York City is implementing the strategy at a scale unmatched nationally.

What We Did

Our team of RAND researchers assessed the effects of the NYC-CS on several outcomes for elementary/middle schools and high schools. We assessed the effects along seven outcome domains (attendance, educational attainment, academic performance, disciplinary incidents, teachers' shared responsibility for student success, student connectedness to adults and peers, and family empowerment opportunities). Our comparison schools were chosen to match the community schools on preinitiative outcomes, as well as on other characteristics that are known to be related to changes in these outcomes. We leveraged an innovative quasi-experimental approach to determine whether students in the community schools were performing better than they would be doing if their schools had not been designated as community schools. This research brief summarizes the results of that effort.

What We Found

Impact of NYC-CS on Elementary and Middle School Outcomes

Outcome 2015–16 2016–17 2017–18
Proportion chronically absenta positive decrease positive decrease positive decrease
Proportion on-time progressive positive increase positive increase N/A
Average math test scores - - positive increase
Average ELA test scores - - -
Number of disciplinary incidentsa positive decrease positive decrease positive decrease
Teacher responsibility for student success - positive increase positive increase
Student connectedness to adults and peers - positive increase -
Family empowerment opportunities - - -

Impact of NYC-CS on High School Outcomes

Outcome 2015–16 2016–17 2017–18
Proportion chronically absenta positive decrease positive decrease positive decrease
Proportion graduated positive increase - positive increase
Credits accumulated positive increase positive increase positive increase
Number of disciplinary incidentsa - - -
Teacher responsibility for student success - - -
Student connectedness to adults and peers - - -
Family empowerment opportunities - - -

NOTE: Shaded cells indicate positive and statistically significant impacts. Arrows indicate whether the impact involved an increase or decrease in the outcome.

aFor these outcomes, a downward arrow suggests a reduction in the outcome among community schools, relative to the comparison schools. N/A = not available.

The figure shows the results for elementary and middle schools and high schools. The shaded cells indicate where we found that the NYC-CS had a positive and statistically significant impact in any of the three years for which we measured outcomes—2015–2016, 2016–2017, and 2017–2018. The arrows indicate whether the positive impact involved an increase or decrease in the outcome.

We found that the NYC-CS had a positive impact for students across a range of outcomes, with some notable exceptions. We found that the NYC-CS had a positive impact on student attendance in all types of schools (elementary, middle, and high schools) and across all three years (increases). We also found positive and significant effects on on-time grade progression in the two years for which we have data for elementary/middle students and on high school students' graduation rates in two of the three years. Our analyses suggest that the NYC-CS led to a reduction in disciplinary incidents for elementary and middle school students but not for high school students. Finally, we found that the NYC-CS had a positive impact on math achievement in the third and final year for elementary and middle students, but the impact estimates on English language arts (ELA) in all three years and on math achievement in the first two years were smaller and not statistically significant. For high school students, we found that NYC-CS had a positive impact on credit accumulation across all three years of the study.

Our evaluation found some evidence that the NYC-CS supported improvements in school climate and culture for elementary and middle schools—the outcomes in the bottom three rows of the table on the left. We found that teachers' reports of shared responsibility for student success increased at elementary and middle schools in the second and third years of the study. Also, we found a positive effect on students' sense of connectedness to adults and peers for elementary and middle school students, but only for the second year of the study period. Finally, we found no statistically significant impact on families' reports of engagement opportunities in elementary and middle schools, and none of the effects on the three outcomes for high schools were statistically significant.

We also examined whether estimated program impacts were being driven by certain subgroups of students or schools, including schools that had been found to be implementing key aspects of the NYC-CS program at higher levels than others (results not shown). Although all community schools experienced reductions in chronic absenteeism, community schools with higher levels of implementation of mental health programs and services saw a stronger impact on this outcome than did community schools with lower levels of mental health implementation; also, there is evidence of variation in impact for some outcomes when comparing community schools that were concurrently a part of the NYCDOE's Renewal Schools Program with those that were not. Otherwise, we did not find a consistent pattern of differential impacts to suggest that the overall pattern in estimated effects was being driven by particular subgroups of students, schools, or program implementation factors.

Implications and Directions for Future Research

The positive findings of the impact of the NYC-CS suggest that the strategy can be a promising approach to support student success in traditionally disadvantaged communities. These positive impacts are particularly important because the NYC-CS is such a large program compared with other instances of the community schools strategy that have been rigorously evaluated thus far. If other, smaller efforts that instill a whole-child, whole-school strategy for student support are to be developed, the coordinated efforts of the New York City Office of Community Schools, along with other key agencies in the city (such as the Department of Health and Mental Hygiene and the Mayor's Office for Economic Opportunity), may provide a promising template for such initiatives.

As for future research, there is a need for an explicit inquiry into the district-level strategies and processes that shape the program as a whole and are likely to affect the implementation experiences of schools. Although we see great merit in this comprehensive evaluation that considers the cumulative impact of the various NYC-CS program features, we encourage researchers to conduct focused investigations into particular community school components, such as family engagement, extended day activities, or mental health service provision. And finally, we encourage researchers to examine more long-term impacts of the community school strategy, ideally extending beyond the limited time frame used for this study.

Research conducted by

This report is part of the RAND Corporation Research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.