Four Lessons Learned from Implementing a Social and Emotional Learning Program to Enhance School Safety

In any K–12 classroom in America, chances are that at least one student is dealing directly with bullying, harassment, or another type of emotional or physical violence. In fact, the number of students affected is staggering. Recent U.S. Department of Education statistics show that in the 2017–2018 school year alone, at least 1.3 million students ages 12 to 18 were at the receiving end of threats, theft, name-calling, exclusion, shoving, and other forms of hostility. The majority (62 percent) of these victimizations occurred either in school, on school property, or on the way to school (Musu et al., 2019).

What can teachers, principals, and school district leaders do to improve safety at their schools? There is some evidence that improving the social and emotional skills of both students and educators can improve school climate and, ultimately, school safety. Evidence suggests that a safe, inclusive, and positive school climate may improve student well-being, including social, emotional, and physical health (Thapa, Guffey, and Higgins-D'Alessandro, 2013). A meta-analysis of the impact of universal social and emotional learning (SEL) programs in K–12 contexts found that these programs show promise across a multitude of schooling and behavioral outcomes (Durlak et al., 2011), including academic achievement, social and emotional skills, and reduced behavioral conduct problems. And there are many available programs and tool sets that schools can use to help improve interpersonal relationships, create a positive school climate, and build safe conditions. (For evidence reviews, see Grant et al., 2017, and Jones et al., 2017.)

One such program is Tools for Life: Relationship-Building Solutions, a classroom- and home-based program for children ages 3 through 5 and in grades 1 through 8. The program is designed to improve school climate and safety through the proactive development of students' interpersonal skills, such as relationship-building and communication, and intrapersonal skills, such as self-regulation and resiliency. With funding from a grant from the National Institute of Justice, as part of its Comprehensive School Safety Initiative, in 2016, Jackson Public School District (JPSD) in Mississippi adopted Tools for Life as one effort to address serious school safety challenges that the district was grappling with.

RAND Corporation researchers evaluated the short-term and longer-term effects of Tools for Life in randomly selected JPSD elementary and middle schools in the 2016–2017 and 2017–2018 school years. The team conducted the first districtwide randomized controlled trial of the Tools for Life program, assessing the program's implementation, costs, and impact on school climate and safety. The findings of this study suggest ways in which schools and school districts can support adoption and implementation of tools and programs to enhance students' interpersonal skills and intrapersonal skills and, ultimately, school safety.

Applying Tools for Life in the Jackson Public School District

JPSD recognized its need to address the well-being and development of students to foster a safer and more responsive learning environment in its elementary and middle schools. In 2012–2013, before Tools for Life was implemented, JPSD served 29,137 students. Most of the students were African American (97.3 percent) and economically and academically disadvantaged. On average, 23.8 percent of elementary school students, 84.9 percent of middle school students, and 69.4 percent of high school students in JPSD schools had received an office discipline referral—that is, been sent by their teacher to the principal's office for minor behavioral infractions, such as being disrespectful, or for major infractions, such as physically threatening someone or vandalizing school property.

When JPSD sought out Tools for Life, individual schools had already implemented several positive behavior intervention strategies and had taken part in restorative practice pilot projects. Despite these efforts, JPSD continued to struggle to maintain basic levels of school safety. District leaders chose Tools for Life, developed in the 1990s and implemented in more than 100 schools in Canada and the United States at the time of the study, because it offered a set of strategies that teachers integrate with classroom instruction, rather than "add on" to their classes. This was the first time JPSD employed a single program throughout all elementary and middle schools to improve school climate and safety.

Tools for Life promotes a positive school environment by building three core areas:

• Student-adult relationship: The program promotes positive and sustained adult-youth relationships through teacher-student and parent/guardian-student dialogue.
• Skills development: Teachers are given tools to build students' interpersonal and intrapersonal skills through play- and inquiry-based learning practices.
• Skills application: Students in kindergarten through grade 8 are expected to apply the skills they have learned.

As implemented in JPSD, Tools for Life consisted of multiple components. The main component was a classroom kit for teachers that included grade-specific manuals with a series of lesson plans for each grade level, along with suggested activities corresponding with each lesson. Each kit also contained a variety of supplemental materials for use during classroom activities. These included posters, bookmarks, cards, and lanyards depicting suggested problem-solving tools. In addition, each teacher was instructed to create a "calm-down" corner in the classroom, so that students would have an inviting and safe place to process their feelings. In JPSD, elementary school counselors and middle school social studies teachers delivered the Tools for Life lessons, and all other educators and staff were expected to reinforce the skills taught by drawing on the supplemental materials. Tools for Life also included home kits, which could be taken out of each school's library by parents and guardians, that provided information on key concepts and problem-solving tools.

Tools for Life provided initial and mid-year educator training sessions and an end-of-year educator conference. Specialized JPSD staff were hired to serve in supportive implementation coaching roles in schools. Tools for Life consultants also worked hand-in-hand with district implementation staff and with schools to support implementation.

JPSD leaders and Tools for Life consultants conceived the program as one that would help create a school culture in which students feel safe and can learn effectively, and in which teachers realized the importance of building relationships and interacting positively with students. JPSD leaders and Tools for Life consultants viewed the program as a preventive, proactive social and emotional skills program that would help students learn to communicate with each other and provide viable problem-solving strategies. The goal was to have students learn to self-regulate and take responsibility for making good decisions, which, in turn, would translate into better academic outcomes.

How the Study Was Conducted

To assess the degree to which Tools for Life was effective in improving school climate and school safety, the research team conducted a cluster randomized trial for JPSD students in grades 3 through 8. Twenty-three elementary and middle schools were randomly assigned to implement the program in the 2016–2017 academic year ("treatment" schools), and 22 others ("control" schools) were randomly assigned to conduct business as usual. In the second year of the study, the 22 control schools also began implementing Tools for Life.

To gain insight into the implementation of Tools for Life in JPSD, the research team analyzed data from key stakeholder interviews with district and Mississippi Department of Education officials, district-based Tools for Life coaches, and Tools for Life consultants; results from surveys administered to students in grades 3 through 8; and data from interviews with school leaders and from focus groups with instructional staff at six focal schools (three treatment schools and three control schools).

To measure the cost of implementing the program in the district, the research team collected and analyzed relevant administrative and financial data and conducted interviews with stakeholders across JPSD.

Findings and Lessons Learned

The overall study results suggest that the Tools for Life program had little impact on students' social and emotional learning, behavior, or academic performance, and that the program was relatively expensive to implement. However, several factors appear to have challenged implementation of the program and the research team's ability to discern its impacts.

It is worthwhile to examine these impeding factors and those that promoted successful implementation to draw lessons learned. These lessons may help inform the future efforts of other school districts seeking to improve school climate and safety by implementing a whole-school SEL program.

Challenge: Launching a districtwide SEL program requires considerable effort, time, resources, and commitment, which was difficult for JPSD during a time of transition.

During the time of Tools for Life implementation, JPSD was undergoing widespread administrative changes. Superintendent resignations, the dissolution of the city's formal school board at the start of the program, and a performance audit of the district by the Mississippi Department of Education likely hindered dedicated implementation. Participants in interviews and focus groups said that these stressors hindered stakeholders' buy-in of Tools for Life. One principal, for example, expressed concerns that the program would take energy and focus away from academics, and, given the intense academic review JPSD was under by the state department of education, felt that academics must take precedent. In general, principals seemed unclear how the program fit into district priorities. Overall, the timing and potential lack of clarity of messaging likely affected educators' capacity and inclination to implement Tools for Life as it was originally designed.

Lesson Learned: Make sure the time is right, and connect the program to priorities.

The readiness of the district—and each school within the district—to take on an SEL program is an important factor to consider. Sufficient support, leadership, enthusiasm among all staff and administrators are critical. A period of uncertainty and instability, as JPSD was experiencing, may not be an ideal time to launch a new program, as implementers may be stressed and feel that they are already facing competing priorities and their capacity is stretched. In addition, interview data from this study suggest that, when a district does decide to adopt a new program, educators may be more committed to it if the district clearly messages how the program fits into district or school improvement goals and situates it visibly as a priority.

Challenge: Buy-in and adoption of the program were uneven in JPSD.

There was strong evidence that some educators in JPSD viewed the program as important and fully adopted it in their classrooms. There was also evidence that buy-in and subsequent implementation varied across schools. This likely affected overall student outcomes in the study: Teachers are less inclined to implement a program they do not believe in or feel little ownership of, and students cannot be expected to develop in skills that have not been taught to them. The lack of educator buy-in in JPSD was due in part to teachers and principals perceiving SEL-related requirements as "top-down" decisions made by superintendents and school boards; many found it difficult to fully subscribe to a program they had not been consulted about. Another factor was the district's adoption of several pro-social programs in addition to Tools for Life. Principals and teachers shared concerns about "initiative fatigue"; teachers especially noted that they felt "burned out" by the many different programs cycling through the district. Finally, a significant number of parents in the district did not consent to have their student participate in the RAND study. The low student participation rate in the study may have adversely affected the overall estimation of program effects.

Lesson Learned: Secure stakeholder and implementers' buy-in before implementation.

SEL programming, like all education components, takes place in a larger school and community environment; teachers, principals, and parents can have different perspectives about the needs of the school and how to best address them. Educators and parents may also become burdened and frustrated with too many new programs that compete for their attention. JPSD may have benefited from conducting a needs assessment that included soliciting multiple stakeholders' input prior to committing to the program (see Wrabel et al., 2019). The process of engaging in such an assessment may have helped mitigate principals' and teachers' perception that programs are "not really something we need here" or are "pushed" upon them. In interviews, many teachers expressed that they did not need the Tools for Life program because they were already addressing the targeted social and emotional learning skills. A needs assessment may have surfaced and validated such perspectives. It may also have afforded opportunities for members of the school community to have rich and vulnerable conversations about where gaps may exist and where external expertise and resources may be warranted. Decisions about program adoption that take into account the perspectives of those who would be key partners in implementation and support are likely to garner stronger buy-in.

Challenge: Implementation challenges are typical in the first years of a new program; districts should be prepared.

Program lead implementers in JPSD (i.e., elementary school counselors and middle school social studies teachers) found some components of Tools for Life easy to understand and implement, but they also encountered difficulties. First, the program consisted of only 12 explicit lessons, which they perceived as too limited a number to meet district and school needs. A number of lead implementers taught two lessons per month and ran out of program-related lessons before the end of the school year. The program offered flexibility to expand or adapt lessons, but there were minimal explicit instructions on how to do so. Moreover, some teachers noted that the program lesson scripts were not clear and required extra work on their part to keep students engaged. Some counselors who served as lead implementers reported that the lessons did not work well with the large student groups they led, and middle school educators perceived the content and aesthetics of some of the program materials as "too elementary" for their students. Thus, the program, perceived by educators and lead implementers as "low dosage" and difficult to implement in their particular context, may not have been right for the district, at least not without some customization or additional supports.

Lesson Learned: Conduct a careful program selection process and take full advantage of implementation supports.

A variety of issues may underlie the implementation challenges that JPSD experienced. One could be a fundamental mismatch between the needs of the school and the selected program. Tools and guides exist to support districts and schools in selecting an effective SEL program in consideration of their context (e.g., JPSD, 2013; JPSD, 2015; Jones et al., 2017). Along with a needs assessment, such resources could have helped JPSD leaders to identify an optimal program that educators and students would engage with successfully. Another issue could be shortcomings in program materials. Program providers, as in the case of Tools for Life, sometimes are open to working with districts before implementation to tailor components of the program to district needs and to remedy potential gaps in program materials and/or to suggest ways to deepen implementation. Third, implementers may lack capacity and require ongoing support in delivering the program to their students. Educators we interviewed thought favorably of the support they received from implementation coaches and Tools for Life consultants, but they also reported not accessing such supports consistently. Principals, in particular, often found it difficult to make time to meet with consultants and coaches. Some principals reported that regular meetings between principals and coaches, as designed, may have helped them better understand implementation details and become more engaged in implementation efforts and better able to support their teachers through implementation challenges. Educators also reported that there were no forums, such as staff meetings, at which they could collaborate on Tools for Life implementation and draw on each other's expertise. Perhaps if such supports were readily available and accessed, implementers would have encountered fewer challenges.

Challenge: Not all measures are equal.

The district measures for SEL and school climate outcomes relied on students' self-reports. These measures can provide useful information about SEL and climate, but they are also subject to several potential biases. Reference bias can be a particular concern when evaluating the implementation of SEL programs; reference bias occurs when students' responses are influenced by a comparison of their own skills or experiences with those of other students. If students see their peers' SEL skills improving, they might apply a higher standard when rating their own skills. This and other limitations of self-report surveys can hinder their utility for evaluating program effects.

Lesson Learned: Choose the right measurement tools.

To support valid inferences about program effects, it can be useful to administer measures that do not rely exclusively on student self-reports. Other types of assessments of social and emotional competencies, such as teacher reports or direct skills assessments (e.g., assessments that require students to apply social and emotional competencies to situations that simulate real-world conditions), can often provide more useful information for understanding changes in students' social and emotional competencies. Similarly, data from teachers, parents, or others can provide a more comprehensive view of school climate than can be obtained through student reports alone. However, these other methods can be time-consuming or expensive to administer. (See Schweig, Hamilton, and Baker, 2019, for a review of school climate measures.) Districts and researchers need to balance the trade-offs with each type of measurement tool to determine which is most appropriate for any study.

RAND's analysis did find a positive association between students' self-reported exposure to Tools for Life materials and lessons and their self-reported social and emotional outcomes. This suggests that the program could have a greater positive impact on SEL and school safety in the district if implementation challenges are overcome. The results remind us that the readiness of a school or district and enthusiasm for a program are critical factors to assess before committing to any particular program.

References

CASEL—See Collaborative for Academic, Social, and Emotional Learning.

Collaborative for Academic, Social, and Emotional Learning, 2013 CASEL Guide: Effective Social and Emotional Learning Programs: Preschool and Elementary School Edition, Chicago, Ill., 2013.

———, 2015 CASEL Guide: Effective Social and Emotional Learning Programs: Middle and High School Edition, Chicago, Ill., 2015.

Durlak, Joseph A., Roger P. Weissberg, Allison B. Dymnicki, Rebecca D. Taylor, Kriston B. and Schellinger, "The Impact of Enhancing Students' Social and Emotional Learning: A Meta-Analysis of School-Based Universal Interventions," Child Development, Vol. 82, No. 1, 2011, pp. 405–432.

Grant, Sean, Laura S. Hamilton, Stephani L. Wrabel, Celia J. Gomez, Anamarie A. Whitaker, Jennifer T. Leschitz, Faith Unlu, Emilio R. Chavez-Herrerias, Garrett Baker, Mark Kevin Barrett, Marl Harris, and Alyssa Ramos, Social and Emotional Learning Interventions Under the Every Student Succeeds Act: Evidence Review, Santa Monica, Calif.: RAND Corporation, RR-2133-WF, 2017.

Jones, Stephanie, Katharine Brush, Rebecca Bailey, Gretchen Brion-Meisels, Joseph McIntyre, Jennifer Khan, Bryan Nelson, and Laura Stickle, Navigating SEL from the Inside Out: Looking Inside and Across 25 Leading SEL Programs: A Practical Resource for Schools and OST Providers (Elementary School Focus), Cambridge, Mass.: Harvard Graduate School of Education, March 2017.

Musu, L., A. Zhang, K. Wang, J. Zhang, and B. A. Oudekerk, Indicators of School Crime and Safety: 2018, Washington, D.C.: U.S. Department of Education, National Center for Education Statistics, and U.S. Department of Justice, Office of Justice Programs, Bureau of Justice Statistics, NCES 2019-047/NCJ 252571, 2019.

Schweig, Jonathan, Laura S. Hamilton, and Garrett Baker, School and Classroom Climate Measures: Considerations for Use by State and Local Education Leaders, Santa Monica, Calif.: RAND Corporation, RR-4259-FCIM, 2019.

Thapa, A., J. Cohen, S. Guffey, and A. Higgins-D'Alessandro, "A Review of School Climate Research," Review of Educational Research, Vol. 83, 2013, pp. 357–385.

Wilson, S., "School-Based Restorative Justice as an Alternative to Punitive Zero-Tolerance Policies," Criminal Justice. Policy and Research, 2013, pp. 1–7.

Wrabel, Stephani L., Laura S. Hamilton, Anamarie A. Whitaker, and Sean Grant, Investing in Evidence-Based Social and Emotional Learning: Companion Guide to Social Emotional Learning Interventions Under the Every Student Succeeds Act: Evidence Review, Santa Monica, Calif.: RAND Corporation, RR-2739-WF, 2019.

Available for Download

Download PDF

Additional Downloads

Includes Translations

Topics

• Educational Program Evaluation
• Elementary Education
• Social and Emotional Learning