Evaluating the Spirituality for Kids After-School Program

Research Brief

Abstract

Aiming to improve on existing research on after-school programs, RAND researchers evaluated the effect of the SFK program on a wide variety of behavioral outcomes. They found that SFK had positive effects on nearly every domain tested, and many of these effects persisted at 12-week follow-up. Notably, the effect sizes exceeded the average effect sizes found for other programs with like goals. This suggests that SFK is a successful program model. This study also provides the first causal link between spiritual development and resilience.

The United States invests considerable resources in after-school programs — in 2002, the federal government spent $3.6 billion. However, most evaluations of after-school programs focus only on academic benefits, lack rigor, and show mixed results, so how and how much these programs benefit children are open questions.

The RAND Corporation aimed to address the shortcomings of existing research in its evaluation of one promising after-school program, Spirituality for Kids (SFK). The SFK program began in Los Angeles five years ago and has since expanded to New York, Miami, and Las Vegas, as well as internationally to London, Mexico City, Panama, Israel, and Malawi. Despite its name, SFK is not a religious program. Its curriculum is based on established best practices in the resiliency literature that seek to build four areas of personal strength: social competence, problem solving, autonomy and self-efficacy, and sense of purpose.

The program has received enthusiastic support worldwide, but program effects have been documented only informally through testimonials. This RAND study is the first formal evaluation of the program’s quantitative effects on a wide variety of behavioral and attitudinal outcomes. Researchers examined the impact of the level 1 SFK after-school program, a 10-week curriculum designed to teach children how to access inner resources and build positive connections with others.

The Study Was Rigorous; Assessed Behavior Change Over Time

Nineteen program sites serving 737 children in southeast Florida were randomized to treatment or wait-list control groups. The researchers surveyed teachers and children before the program began, right after it ended, and 12 weeks after it ended. They used a well-known, validated survey instrument that asked about positive behaviors (such as adaptability, communication, leadership, social skills, and study skills) and negative behaviors (such as anxiety, depression, and attention, conduct, and learning problems). They analyzed the data using quasi-experimental methods, controlling for site variation and baseline characteristics of children.

Figure 1
SFK Effects on Improving Positive Behaviors Greatly Exceed Effects of Other Programs
SFK Effects on Improving Positive Behaviors Greatly Exceed Effects of Other Programs
  
Figure 2
SFK Effects on Reducing Negative Behaviors Equal or Exceed Effects of Other Programs
SFK Effects on Reducing Negative Behaviors Equal or Exceed Effects of Other Programs

SFK Had Significant Positive Effects

The SFK program had positive effects on virtually every domain covered by the survey. As illustrated in Figure 1, the researchers found medium to large effects (effect sizes of 0.63 to 0.80)[1] on positive behaviors. Notably, the effect sizes exceed the average effect sizes found for other after-school programs targeting similar outcomes.

As seen in Figure 2, the researchers found small to medium effects on the incidence of behavioral problems, especially attention problems, hyperactivity, and withdrawal. These effect sizes equal or exceed the average effect sizes of other programs targeting similar outcomes. There was suggestive evidence that many program effects persisted at 12-week follow-up.

SFK May Also Effect School-Related Outcomes

The SFK program affected school-related outcomes, even though SFK is not an academic intervention. For example, the program had a large effect on improving reported study skills and medium effects on reducing learning problems and attention problems. SFK’s success in improving school-related outcomes suggests that an extremely interesting follow-up study would be to examine treatment effects on grades and subsequent standardized test scores.

SFK’s Success Points to More Follow-Up Studies

Because the program continues to expand not only to new communities but to different countries and settings, the researchers recommend a follow-up study to test the replicability of the SFK model in diverse contexts and target populations. They also recommend evaluation of the level 2 and level 3 courses to test whether they can support and perhaps even build on the effects achieved after the level 1 course. Future evaluations might also seek to extend the follow-up period beyond 12 weeks.

Study Contributes to Research Literature

This evaluation not only provides SFK with the data it needs to strategize, plan, and improve, but it makes a significant contribution to the research literature. The evaluation sets standards for rigorous evaluation of after-school programs and highlights a successful program model. Finally, because of its rigorous research design, it offers the first causal link between spiritual development and resilience. square


[1] Effect sizes are calculated by dividing the estimated treatment effect by the pretest standard deviation for each scale. A positive sign is assigned to an effect size whenever the treatment group did better than the control group, and a negative sign is used whenever the control group did better. Thus, a positive effect size for a negative behavior means that the treatment group experienced a greater reduction in the behavior than did the control group.


Blue arrowRead the Full Report

This research brief describes work done for RAND Labor and Population and documented in An Outcome Evaluation of the Spirituality for Kids Program, by Nicole Maestas and Sarah Gaillot, TR-575-SFK, 2008, 70 pp., ISBN: 978-0-8330-4477-8 (Full Document).

This product is part of the RAND Corporation research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

Copyright © 2008 RAND Corporation

The RAND Corporation is a nonprofit research organization providing objective analysis and effective solutions that address the challenges facing the public and private sectors around the world. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors.

RB-9348-SFK (2008)